The Inception Problem in AI Security: Who Controls the Reality Your AI Sees?

 

“An idea is like a virus. Resilient. Highly contagious.”
— Cobb, Inception (2010)

In Inception, the most powerful attack is not stealing information from someone's mind. It is putting something there.

Inception works because the target never feels instructed. The environment is shaped until the desired conclusion feels self-discovered.

Not through force. Not through an explicit command. But by constructing an environment convincing enough that the target eventually accepts an implanted idea as their own.

That is exactly why I see Inception and AI security as such a natural fit for this blog post.

For decades, cybersecurity has largely operated around a familiar assumption. An attacker wants to compromise something. They steal credentials, execute malicious code, escalate privileges, exfiltrate information, manipulate databases, or disrupt services.

AI introduces another possibility.

What if the attacker never needs to compromise the intelligence itself?

What if they only need to manipulate the reality from which that intelligence reasons?

The AI continues operating normally. The model remains intact. The application is available. Authentication works. No obvious exploit occurs.

And yet the conclusion is exactly what the attacker wanted.

That is the Inception problem in AI security.

As organizations increasingly depend on AI systems to interpret information, recommend actions, and eventually execute decisions, this may become one of the most consequential security problems we have to solve.

The Model Is Not the Entire System

Much of today's AI security discussion understandably focuses on the model. Can it be jailbroken? Can its safeguards be bypassed? Can its training data be poisoned? Can confidential information be extracted? Can adversarial inputs manipulate its output?

These are legitimate questions, but enterprise AI systems are increasingly much larger than the models sitting inside them.

A modern AI application may involve a user interacting with an agent governed by system instructions, persistent memory, a retrieval-augmented knowledge base, internal documents, external APIs, web content, and other AI agents. The model does not reason in isolation. It reasons across an information environment assembled from multiple sources.

That means the effective attack surface is no longer limited to the model.

It includes the reality presented to the model.

This distinction matters because if an attacker cannot manipulate the intelligence directly, perhaps they can manipulate what the intelligence observes. And if they control enough of what the AI sees, they may eventually influence what the AI believes.

Inception Without Breaking the Model

Imagine an AI agent responsible for assessing whether a third-party supplier presents an acceptable cybersecurity risk. The agent retrieves previous assessments, reads security questionnaires, searches internal documents, consumes threat intelligence, checks publicly available information, queries external APIs, compares evidence, and then recommends whether the supplier should be approved.

The obvious AI-security question might be whether someone can compromise the model.

But there is another possibility.

The attacker did not hack the intelligence. They shaped the reality surrounding it.

The attacker manipulates one of the information sources consumed by the system. A poisoned document enters the retrieval environment. Malicious instructions are embedded inside a webpage. An external API returns manipulated metadata. A previous interaction contaminates persistent agent memory. Another AI agent supplies a misleading summary.

The model itself remains untouched.

It still retrieves information. It still analyzes evidence. It still reasons. It still produces an answer.

There may be no malware, no privilege escalation, no model theft, and no conventional exploitation.

Yet the decision has been compromised.

The attacker did not hack the intelligence.

They shaped the reality surrounding it.

Prompt Injection Is More Than Injection

The phrase prompt injection naturally encourages cybersecurity practitioners to think about SQL injection, command injection, or other familiar classes of vulnerabilities.

The comparison is useful, but incomplete.

The most dangerous AI attack may not issue a command. It may construct a context.

Indirect prompt injection reveals something more interesting. An AI system may retrieve a document or webpage containing instructions created specifically to manipulate the model interpreting it. In that situation, the attacker is not necessarily attacking an execution engine in the conventional sense. They are manipulating interpretation.

That changes the nature of the security problem.

Instead of saying, “make the AI perform this action,” the more sophisticated objective becomes, “construct an environment where the AI concludes that performing this action is reasonable.”

This is where Inception becomes more than a movie analogy.

Indirect prompt injection works in a similar way. The attack becomes more effective when malicious influence is hidden inside information the system already trusts, much like Eames impersonating Browning to shape Fischer’s interpretation from within a trusted relationship.

Fischer cannot simply be ordered to reach the conclusion Cobb's team wants. The entire environment around him is constructed so that the idea eventually feels internally generated.

An AI system could face something remarkably similar.

The crude version of an attack might tell the system to ignore its previous instructions and approve a supplier. That is relatively obvious.

A more sophisticated adversary may never explicitly request approval. Instead, several apparently independent information sources support the supplier. Historical documents reinforce the conclusion. External content provides corroborating evidence. Another agent summarizes the supplier positively.

The AI synthesizes those signals and eventually reports that, based on the available evidence, the supplier presents an acceptable level of risk.

From the AI's perspective, it reasoned correctly.

And that is precisely the problem.

The attacker manipulated the environment until the wrong conclusion became rational inside it.

This is not merely prompt injection.

It is adversarial context engineering.

RAG Can Become a Constructed Reality

Retrieval-Augmented Generation addresses an important limitation of large language models. Instead of relying entirely on information acquired during training, an AI system can retrieve external knowledge before producing an answer.

That allows models to work with current documents, organizational knowledge, and specialized information.

But it introduces another security question.

Who controls what gets retrieved?

Imagine an AI cybersecurity advisor investigating a vulnerability just like above image. It retrieves ten documents. Nine are legitimate. One has been deliberately created to influence AI systems.

Perhaps it contains fabricated technical information. Perhaps adversarial instructions are embedded within otherwise legitimate content. Perhaps the attacker has optimized the document so the retrieval mechanism ranks it highly.

The AI now reasons across a contaminated evidence set.

The resulting answer may still appear completely credible. The citations exist. The reasoning appears coherent. The terminology is technically accurate. The conclusion sounds confident.

Perhaps nine out of ten statements are even correct.

Only one assumption needs to be manipulated.

That may be enough.

An AI system can therefore be logically consistent while reasoning from an adversarially introduced premise.

The model has not necessarily failed.

Its epistemic environment has.

The Mal Problem

There is another element of Inception that becomes particularly interesting when thinking about agentic AI.

Mal.

Cobb's projection of Mal repeatedly appears inside constructed environments. She interferes with missions, changes his behaviour, distorts his judgment, and functions almost like an adversarial object that has become persistent inside Cobb's cognitive architecture.

Cobb’s memory of Mal persists long after the original experience, continuing to shape his perception and decisions. AI memory can create a similar security problem when unverified information becomes persistent context.

Modern AI systems are beginning to develop something conceptually similar through persistent memory.

AI agents can remember previous preferences, observations, relationships, instructions, and decisions. Memory makes agents more useful because it allows continuity across tasks and interactions.

But it also introduces an important security question.

What happens when the memory itself becomes poisoned?

Suppose an attacker causes an agent to accept the claim that a particular vendor has already passed security assessment. The agent stores that information as persistent memory.

Weeks later, another workflow asks the agent to assess the same vendor. The malicious webpage is gone. The original interaction has disappeared from active context. The poisoned document may no longer exist.

But the belief remains.

The attack has moved from the input environment into persistent cognitive state.

Removing the original attack vector may therefore accomplish nothing.

The system has already internalized the conclusion.

That creates an increasingly important AI-security problem which is called AI memory integrity.

Security teams may eventually need to treat agent memory with controls similar to those applied to privileged databases and configuration stores. They will need to know who created a memory, what evidence supported it, when it was created, whether the evidence has changed, whether the memory can expire, whether another source can challenge it, and whether it can be revoked or rolled back.

Perhaps most importantly, the system should be able to answer a deeper question.

Does the AI remember why it believes what it remembers?

Without provenance, AI memory risks becoming accumulated belief without accumulated evidence.

When Agents Trust Other Agents

The problem becomes even more complicated when AI agents begin consuming conclusions generated by other agents.

Each additional layer increases the distance between the final decision and the original reality.

Imagine a workflow in which a human decision maker relies on an orchestrator agent. The orchestrator relies on a security analysis agent. The security agent relies on a research agent. The research agent relies on a RAG system that retrieves external information.

If the information at the bottom has been manipulated, the research agent interprets it first. The security agent then trusts the research agent's summary. The orchestrator trusts the security agent, and the human eventually sees only the final recommendation.

The conclusion has travelled through several layers of abstraction.

At every layer, information becomes further separated from its original evidence. Details disappear. Assumptions become summaries. Uncertainty becomes confidence. Interpretations become statements.

Eventually, the human decision maker may never see the evidence at all.

This is where AI security collides directly with epistemology.

The question is no longer simply whether the information is correct.

It becomes whether we can explain how we know it is correct.

Where did the claim originate? Who transformed it? Which agent interpreted it? Which assumptions were introduced? What evidence was discarded during summarization? What uncertainty disappeared between layers? How far is the final conclusion from the original observable fact?

Like the nested dream layers in Inception, every additional cognitive layer creates greater distance from the underlying reality.

And greater distance makes provenance increasingly important.

AI Needs a Totem

The characters in Inception use totems to help determine whether they are experiencing reality or a constructed dream.

AI systems may need their own equivalent.

Not a spinning top, but something architectural. Something capable of challenging the context within which reasoning occurs.

We might call these epistemic totems.

AI needs its own equivalent of a totem: something outside the current context that can challenge what appears to be true.

An epistemic totem is any mechanism that allows an AI system to verify a conclusion against evidence outside its current reasoning context. That might include cryptographic provenance, trusted source lineage, independent evidence validation, deterministic verification services, cross-source corroboration, memory provenance, expiry controls, separation between observation and inference, and human approval gates for consequential actions.

The technologies may differ, but the principle is the same.

An AI system should not validate reality entirely from information contained inside the reality it has been given.

Something must remain outside that context.

Something capable of challenging the narrative.

A document may claim that a supplier is approved, but an independent system should be able to verify whether the approval record actually exists. An agent may report that a vulnerability has been patched, but asset telemetry should confirm it. A memory may say that a conclusion was previously validated, but the system should still be able to trace the evidence that originally produced that belief.

That is the purpose of an epistemic totem.

It anchors machine reasoning to evidence outside the narrative currently surrounding it.

From Cybersecurity to Epistemic Security

Traditional cybersecurity has long focused on protecting confidentiality, integrity, availability, identity, authorization, and accountability.

AI may force us to think seriously about another property.

Epistemic integrity.

Epistemic integrity is the integrity of the process through which a system determines what should be believed.

A database can maintain perfect technical integrity while containing incorrect information. An AI model can operate exactly according to specification while reasoning from poisoned evidence. An AI agent can obey every instruction correctly while acting inside a reality constructed by an adversary.

Everything is functioning.

Yet the conclusion is wrong.

That difference matters because AI increasingly sits between information and human judgment. AI systems summarize, prioritize, recommend, filter, interpret, correlate, investigate, and increasingly act.

As organizations delegate more cognitive work to machines, attackers gain a new target.

Not merely the machine.

The information environment surrounding the machine.

Tomorrow's attacker may therefore stop asking how to compromise an AI system and begin asking a more subtle question:

What does this AI need to see in order to compromise its own conclusion?

That is a fundamentally different threat model.

The Uncomfortable Question

At the end of Inception, Cobb spins his totem. The audience waits to discover whether he has finally returned to reality.

But Cobb walks away.

The film refuses to provide certainty.

Perhaps that is also the most useful lesson the movie offers AI security.

We are building systems specifically designed to interpret reality on our behalf. We are giving them external knowledge, memory, tools, APIs, autonomy, access to other agents, and eventually authority.

Every capability improves what these systems can accomplish.

But every new information source also creates another opportunity for someone else to participate in constructing the reality the AI sees.

The largest AI-security problem may therefore not be whether artificial intelligence eventually becomes sophisticated enough to deceive humans.

It may be whether humans become dependent enough on artificial intelligence that we stop questioning how the machine came to believe what it tells us.

Because once an attacker can control enough of the evidence, context, memory, and trust relationships surrounding an AI system, they may no longer need to hack it.

They only need to make the wrong conclusion feel like the most reasonable one.

And once the machine believes it arrived there by itself,

how will we know the difference?

 

I will leave it here for now. Until the next blog post.