Parā Technica #2: The Context Engine

We’ve built machines that can speak every language yet understand none.

Large language models sound fluent—they complete our sentences and compose essays—but behind the polish lies a blind engine of probability. They predict what comes next, not why. They process the products of thought, not thought itself.

Syntax Without Sense

AAD: Mohak, in our last conversation, you described your ‘technical itch’—how these models are essentially just sophisticated predictors, sampling across a probability space. This led to the question, “Are we just higher-order predictors?“ This is a profound challenge, and to begin to answer it, I think we first need to explore the very mechanics you brought up. We often hear that these models are “next-token predictors,” but that phrase can feel a bit simplistic when you see them produce a complex essay or a poem. What’s actually happening under the hood?

Mohak: A transformer model works by looking at all the words in a prompt and figuring out how they relate to each other. It creates a kind of internal map, highlighting the crucial concepts, using this to bridge to a response based on the vast amounts of text it has ingested. It’s this ability to weigh the importance of every word in the context of every other word that makes it so powerful.

But, and this is the critical point, this entire process happens purely at the level of words and statistical patterns. It’s building a relational structure out of language, and language is how we record thought, not entirely how we think. It doesn’t know what rain feels like, what an explorer is, or what it means to feel intrepid. I know what rain feels like because I’ve experienced it. As a result, it means something to me. If I just stepped in wet from a downpour, I might resent it. If instead my mind goes to memories of running in the rain with my friends, it brings warmth. The model only knows how those words connect to other words, but none of their meaning. Transformers don’t have the world behind the word.

The Chinese Room and the Illusion of Understanding

AAD: Your description brings to mind John Searle’s Chinese Room Argument.

Searle imagines a person who knows no Chinese locked in a room with a rulebook that tells them how to respond to Chinese symbols slipped under the door. For every sequence received, the book provides one to send back.

To an observer outside, the person appears fluent and natural. Inside, there’s no understanding—only rule-following. Perfect syntax (the rules of language), zero semantics (the meaning behind it).

This is what these AI models do: they masterfully manipulate language without any true comprehension of the world it describes. Syntax is not sense, and coherence is not consciousness.

Now imagine the person outside trying to convey sarcasm or joy. Language is always incomplete. How much human meaning fits on that slip of paper? What happens to the tone, the emotion, the context between the lines, often left unexpressed? A model can reproduce a poem, but not the tremor that wrote it. What these models lack is context.

Mohak: Likewise, for all its speed, digital messaging has always been a ghost of real conversation. We’ve spent decades trying to breathe life into our texts—from crude emoticons to emoji to the perfect GIF for every reaction. Each is a clever attempt to clarify whether ‘sure’ conveys a sarcastic eye roll or a heartfelt embrace. Yet we’re still chasing the effortless understanding that comes from a single shared glance—a nuance our screens have never mastered.

The Skill for a New Era: Context Engineering

Mohak: This points to the extraordinary importance of context. We engage in small talk not because we have nothing else to say, but to share the context within which the rest of our conversation will unfold. We tell each other, “I’m tired,” or “I had a great day” to contextualize our mental state. These clues help the listener better understand us and, ultimately, better respond to us.

A key new skill that emerges when we deal with LLMs is inspired by this aspect of human interaction. We started with “prompt engineering,” to coax LLMs into giving us the answers we seek by asking precise, well-crafted questions. But now it is clear: precision matters, but context matters more—a move from prompt engineering to context engineering. If the AI has a map but no feel for the terrain, our job is to describe the terrain for it. Instead of just typing, “Write me a note to my team,” I start with the setting: “You are a trusted colleague. My team has been working on a hard deadline and has had a tough week. Many people worked through the weekend to hit the deadline. I’m worried this may be hitting morale, and the team needs inspiration. I want your help writing a note thanking the team without sounding hollow, and I want to inspire them towards our shared goals.” Suddenly, the response changes. No, the machine doesn’t understand “tired” or “demotivated,” but the added context helps it frame the response based on the data it was trained on.

I’m building a world for the AI to inhabit before it answers.

To Know AI, Know Thyself

AAD: And when you engage in worldbuilding, you are forced to articulate your own understanding of the circumstances and people involved. Good context means knowing what’s relevant, what’s ignored, and what’s valued. Context engineering, then, is a kind of forced introspection. You stop outsourcing thought and start articulating purpose. The AI becomes a high-fidelity mirror. If the reflection is muddy, the source needs clarifying. This is the new ‘garbage in, garbage out’: shallow context yields shallow meaning. It changes our relationship with technology—from operators to stewards of meaning. The irony is that in teaching the machine, we learn to understand ourselves better. A machine trained on our language ends up pushing us to define our own meaning.

Mohak: This might sound abstract, but it’s intensely practical. I see it with my daughters. My daughter sometimes comes to me crying, unable to say why. I can’t help until she can describe what hurts. The moment she names it, “She said I wasn’t her friend anymore,” we can solve her problem together. Context transforms emotion into a solvable experience.

We’re doing the same thing with these models. We bring our confusion and expect clarity, but we forget to give the model context. In that sense, we are simply crying to our tools.

AAD: That plea for context is so often at the heart of what we call wisdom. Take the Bhagavad Gita, which opens on the brink of war. Arjuna faces an inevitable fight against his own kin. Paralyzed by the horror, his will collapses and his bow slips from his grasp. He is engulfed in desolation. Krishna’s response is a profound act of context engineering. He rebuilds Arjuna’s world with a new reality: reframing Arjuna’s identity from that of a mortal man to an eternal soul, his duty from a personal burden to an instrument of Divine will, and even his very being into one engrossed in the Divine. Krishna’s context shift transforms paralysis into purpose, action into offering, despair into devotion. By providing context he ultimately grounded him in meaning.

The Context Engine

In the age of AI, humans may not remain the best creators of output, but we’ll remain unique creators of context and meaning. Machines can produce sentences; only we can decide which ones matter.

The future of work will likely belong to context engines—people who understand and reflect on goals, constraints, personalities, values, and timing. The power won’t lie in knowing what to ask, but in knowing why to ask it.

Take a music producer. The artist provides melody and lyrics, the engineer manages levels—but the producer listens for meaning: how it feels. They decide when to leave silence and when to let imperfection breathe, to bring listeners to tears. That’s context work—shaping not the notes, but the world around them.

Or picture a great team leader. They don’t just assign tasks—they tune into exhaustion, mood, timing. Their real craft is reading the room, not writing the plan. That’s context engineering in human form.

Imagine a world filled with people who reflect, strive to understand, and provide context to each other as much as to these models. A better world—and a more human one.

Questions to Carry

• As you worldbuild for the AI, what core principles—values, assumptions, emotions—form your foundation?

• When the AI’s reflection is “muddy,” what unarticulated part of your thinking is the source of distortion?

• In the “forced introspection” of teaching the machine, what have you uncovered about your own unsaid purpose?

• If shallow context yields shallow meaning, how often do our conversations with people suffer the same?