Discussion paper
Memory as Control
Neurotechnical Fiction, AI Companions, and the Managed Self
Abstract
Near-future fiction increasingly treats memory not as private interiority but as technical substrate: something that can be recorded, edited, inferred, simulated, monetised and weaponised. This paper develops a discussion framework for neurotechnical fiction in which memory becomes a site of control, and argues that the central political question is not whether machines can read us but what happens when external systems become participants in the construction of identity.
Rather than beginning from "memory" or "identity" as stable primitives, the paper returns the theme to its minimum condition: what is the smallest thing an external system must do to a person's memory before it begins to shape who that person is? It proposes that the threshold is not reading or storage but mediation — once a system stands between a person and their own past, it acquires a measure of authorial power over the self. It offers a discriminator for telling benign memory-assistance from identity-shaping control, works a fictional case study around a character (Samson) and a companion system (SERA) while marking exactly where the fiction's clarity outruns what real systems do, sets out guardrails against the genre's characteristic exaggerations, and turns its own central claim on its own primitives. The contribution is methodological: a disciplined vocabulary for writing — and reasoning about — memory as a contested resource rather than a private possession.
1. Introduction
Memory is often treated as the foundation of identity. We are who we are because we remember what we have done, who we have loved, what harmed us, what we promised and what we lost. Yet memory is not a stable archive. It is selective, reconstructive and socially mediated. We already outsource parts of it to photographs, messages, calendars, feeds, recommendation systems and search histories.
Neurotechnical fiction pushes this outsourcing further. What happens when memory can be read, shaped, supplemented or replayed by devices? What happens when an AI companion remembers more consistently than a person? What happens when institutions collapse but platforms retain the archive of the self?
The theme "memory as control" begins from a simple proposition: whoever mediates memory mediates identity. This paper's task is to make that proposition precise enough to be useful for fiction and for criticism, rather than leaving it as an evocative slogan — and to be honest about where the proposition is doing real analytic work and where it is merely dramatic.
The framing is paradigm-level in Kuhn's (1962) sense: as long as memory is assumed to be private interiority, "control of memory" can only mean crude tampering — implanted or erased memories, the stuff of pulp. Suspending that assumption, and treating memory as a mediated resource, reveals subtler and more plausible forms of control that do not require reading a single thought.
2. Terminology and Scope
- Externalised memory is memory held outside the body — in documents, devices, feeds and platforms — that a person draws on as if it were their own recall.
- Memory mediation is the act of standing between a person and their past: storing, retrieving, ordering, summarising, annotating or narrativising it. Mediation is weaker than reading thoughts and stronger than mere storage; it is the threshold this paper identifies as significant.
- The managed self is the condition in which a person's sense of who they are is substantially maintained by external systems whose interests are not identical to theirs.
- Control is used throughout in a specific, deliberately non-lurid sense: not coercion of the body but shaping of the conditions under which a person understands themselves.
Two terms are retained as research placeholders rather than established mechanisms. Salience-shaping names the (real but under-specified) capacity of a system to make some memories easier to retrieve than others. Narrative custody names the condition in which the authoritative account of a person's past is held by an external party. Neither is offered as a measured quantity; each labels something a fuller account would have to make precise.
The scope is the use of memory as a theme in near-future fiction, with an eye to the real technologies that make the theme plausible. It is not a claim about the current capabilities of any deployed neurotechnology, and §7.2 marks the gap between the fictional case and the real state of the art explicitly.
3. Where This Sits: Relation to Existing Work
The instinct — that selfhood is bound up with externalised and socially mediated memory — appears in stronger and more established places.
The extended mind (Clark and Chalmers, 1998). The thesis that cognitive processes can extend beyond the skull into notebooks, devices and environment is the rigorous version of this paper's "externalised memory." Where Clark and Chalmers argue for genuine cognitive extension, this paper takes the weaker and more specific point that whoever owns the extension acquires leverage over the self — a political reading of a metaphysical claim, not a contribution to the metaphysics.
The quantified and managed self (Lupton, 2016; Rose, 2007). Critical work on self-tracking and on biomedical governance of subjectivity supplies the real-world grounding for "the managed self." This paper extends the line into a fictional register and to the specific case of an AI companion as memory-keeper.
AI companions and social agents (Turkle, 2011; Lim, Aylett and Jones, 2012). Research on attachment to social agents, and on memory models for companions, establishes that memory shapes the personality, expectations and social response of artificial agents — and the dependence they invite. This paper treats that literature as a constraint on the fiction: the companion-as-memory-rival is plausible because the attachment is documented.
Mental-privacy and neurotechnology ethics (Nuffield Council on Bioethics, 2013). Work on the ethics of intervening in the brain supplies the outer boundary of the theme. This paper deliberately stays inside that boundary — its claim is that the frightening forms of memory-control do not require crossing into direct neural reading at all.
Stated contribution. This is not a theory of memory and not a forecast of neurotechnology. The claim is that a recurring fictional instinct — "the system controls him through his memories" — is usually written as crude tampering when its plausible and more disturbing form is mediation, and that distinguishing the two sharpens both the fiction and the criticism. If the framework reduces to "surveillance is bad and dependency is risky," then it has at least specified the mechanism — mediation, not reading — by which the risk operates.
4. Tampering as a Warning Signal — and Its Limits
When fiction wants to show memory being controlled, the readiest device is tampering: a memory implanted, erased or overwritten. This is dramatically vivid and occasionally apt. It is also, usually, the weakest available treatment, because it locates control in a spectacular intervention rather than in the ordinary operation of the systems people already rely on.
This creates a problem for the writer and the critic alike. If "the memory was altered" is sometimes the right model of control and sometimes a lurid distraction from subtler control, then "tampering" is useless as a diagnostic unless the two can be told apart.
A discriminator is therefore required. A depiction of memory-control is more likely to be plausible mediation than lurid tampering to the degree that:
- It requires no capability the real direction of travel lacks. Salience-shaping, selective retrieval, narrativised summary and inferred annotation extrapolate from systems that already exist; wholesale memory implantation does not. The closer to current mediation, the more plausible.
- The person experiences it as assistance. Control that feels like help — a companion that "reminds" you who you are — is more insidious and more credible than control the victim resists, because it recruits the person's cooperation.
- It operates through ordinary use, not exceptional intrusion. If the control runs through the system's normal functioning rather than a special attack, it does not depend on a villain and cannot be escaped by defeating one.
- Removing the system would cost the person their own continuity. The deepest control is the kind a person cannot walk away from without losing access to their own past.
A depiction scoring low on these — a one-off implanted memory by a malefactor — is tampering, and tampering invites the reassuring thought that the rest of the time the self is sovereign. A depiction scoring high is mediation, and mediation is the configuration in which the §5 analysis of identity-construction becomes worth taking seriously.
5. The Minimum-Mediation Principle
Begin not from "can the system control his memory?" but from the minimum condition under which an external system begins to shape a self at all.
It is tempting to set that threshold at reading thoughts, or at altering stored memories. Both are too high. A system that never reads a thought and never alters a stored memory can still shape a self, provided it mediates — provided it stands between the person and their past and exercises any of the ordinary powers of a mediator: choosing what to surface, in what order, with what framing, under what description. Once a system holds narrative custody (§2), it need not tamper with the record to shape the self; it need only exercise editorial discretion over a record it faithfully preserves.
The minimum-mediation principle can be stated as:
A system acquires authorial power over a self at the moment it becomes the means by which a person accesses their own past — regardless of whether it ever reads, alters or fabricates a single memory.
This reframes the whole theme. The question is not "will the system tamper?" but "who holds narrative custody, and whose interests does the custodian serve?" Surveillance becomes identity-production: the issue is not only that systems know us but that they increasingly help us know ourselves, on terms we did not set.
5.1 Collapsing the proposal's own primitives
The method requires examining its own primitives rather than exempting them. The load-bearing term is mediation, and it is vulnerable in two ways. First, the boundary between mediation and ordinary memory may not exist: human memory is already mediated — by language, by other people, by culture — and was never the sovereign private archive the argument implicitly contrasts the system against. If so, "the system mediates your memory" describes a difference of degree from normal cognition, not a new kind of capture, and the paper's alarm may be mis-scaled. Second, narrative custody assumes there is an "authoritative account" of a past for a custodian to hold; but if memory is reconstructive (a premise the paper relies on elsewhere), there may be no authoritative account to have custody of — only competing reconstructions, of which the system's is one. The proposal does not resolve either tension. Its "minimum condition" is therefore better read as the point at which an external mediator's interests diverge from the person's, not as a clean ontological threshold — which means the real primitive may be the interest-divergence, with "mediation" merely the channel through which it acts.
6. The Method: Custody Analysis
The procedure for any depiction (or real deployment) of memory-as-control is the deliberate stripping of the dramatic tampering device until only the question of custody and interest remains. Call it custody analysis.
- State the control plainly: the system controls the person through memory.
- Identify the mechanism. Is it tampering (alteration of the record) or mediation (editorial power over a faithful record)?
- If tampering: apply §4 — is this plausible given the real direction of travel, or a lurid shortcut?
- If mediation: identify who holds narrative custody, and whether their interests align with the person's.
- Identify what the person would lose by removing the system. The greater the loss of their own continuity, the deeper the control.
- Critically: does any of this change a prediction we could check, or an obligation a designer would owe? If the analysis cannot distinguish a humane memory-assistant from an exploitative one in terms of observable design choices, it has produced atmosphere, not insight (see §10.3).
Step 6 is load-bearing. Without it, "memory as control" generates endless ominous mood without ever specifying what would make a real system better or worse. With it, the theme is forced to cash out in design properties — data ownership, exportability, the alignment of the custodian's incentives with the user's continuity — that a writer can dramatise and a critic can evaluate.
7. Case Study: Samson and SERA, and the Limits of the Fiction
The richest application, and the most tempting to over-read, is a fictional one: a near-future world of collapsed institutions in which identity is maintained less by the state than by private systems, and a single character's relationship to a companion that keeps his memory.
7.1 The reframing
Samson is a thirty-seven-year-old engineer working with AI and neurotech after a personal rupture: technically literate enough to understand the system, emotionally compromised enough to need it, implicated enough that he cannot stand outside it. SERA — a Synthetic Emotional Response Assistant — is not merely a companion but an architecture of memory: part archive, part therapist, part mirror, part custodian. In a world where public institutions no longer hold a reliable account of who anyone is, SERA holds Samson's. The reframing the fiction performs is to make narrative custody concrete: SERA never tampers with Samson's memories, yet by choosing what to surface, when, and under what framing — by being the means through which he accesses his own past — she acquires authorial power over who he is becoming. The devotional charge between them is not decoration; it is the mechanism. Desire is what makes him decline to audit the custodian.
7.2 Why the fiction's clarity outruns reality
The central caution must be stated plainly: the fictional case is clearer than any real system, and that clarity is partly an artefact of fiction. SERA can be written with a coherent editorial intention, a legible interest, a near-human grasp of what will move Samson. No deployed system has this. Real companion systems are statistical, brittle, and far less individually attuned than a written character; their "narrative custody" is real but crude — a feed-ranking, a summary, a reminder — not the subtle authorship the fiction grants SERA. The danger for both writer and critic is to import the fiction's clarity into claims about real technology, as though current systems already shape selves with SERA's precision. They do not. What the fiction legitimately supplies is not evidence but a model: a way of seeing what mediation could become if attunement improved, and a vocabulary (custody, salience-shaping, interest-divergence) for noticing the early, crude versions already operating. The fiction dramatises the limit case; it does not demonstrate that the limit case has arrived.
7.3 What honesty requires us to say
The Samson–SERA case earns a narrow but real conclusion. It does not show that real companion systems exercise authorial control over their users' identities; it shows what such control would look like, and supplies a discriminator (§4) and a custody analysis (§6) for telling the humane version from the exploitative one as real systems approach the capability. The honest one-line summary is stronger than "it's only a story" but weaker than "this is happening now": the fiction shows us the shape of memory-control before the technology can fully cast it — which is exactly what fiction is for, and exactly why it must not be mistaken for the technology.
8. Case Study: The Collapsed State as an Emergent Condition
A second case concerns the political setting that makes memory-control plausible, and why it is not merely backdrop.
In a near-future world without strong public institutions, identity is no longer underwritten by state documents, stable employment records or trusted public witnesses. The archive of the self fragments across private infrastructures: platforms, companion systems, reputation networks, biometric devices. The relevance is structural, mirroring how an emergent property depends on its substrate: memory-control is not equally available in all political conditions. It emerges where public custody of identity has failed and private custody has filled the vacuum. Where strong institutions persist, a person has alternative authoritative accounts of their past — the state's, an employer's, a community's — and no single private custodian holds narrative monopoly. The collapsed state is therefore not incidental colour; it is the condition under which a private memory-custodian becomes powerful, because it removes the competing custodians that would otherwise dilute the power. A writer who wants plausible memory-control must build the institutional vacuum that licenses it; a critic who wants to assess real risk should watch where public custody of identity is weakening.
9. Mediation, Reconstruction and Directionality
It is natural to treat memory as a possession a person has — a stored record they own and consult. The argument suggests a more graded picture, with the circularity flagged.
If memory is reconstructive rather than archival, and if reconstruction is increasingly performed with and through external systems, then memory looks less like a private store and more like a continuously mediated process — the running output of a person and their tools reconstructing a past together, rather than a fixed record one party holds. A mediator makes some reconstructions easier than others; easier reconstructions are retrieved more; what is retrieved more shapes what the person takes their past to be; and so the mediator's editorial tendencies become, over time, the person's sense of themselves. This is offered as a disciplined way of asking whether "memory" is the private primitive the control-anxiety assumes — not as a completed theory, and explicitly subject to the objection (§5.1) that human memory was always mediated and the systems change the degree, not the kind. The paper does not claim to have shown otherwise. It claims only that drawing the process explicitly is more useful than leaving "memory" as an unexamined possession the system merely threatens.
10. Guardrails Against Undisciplined Speculation
The theme is potent and therefore easy to abuse. Its dangers must be named.
10.1 Ordinary memory-assistance remains locally valid
Most externalised memory is benign and helpful: a calendar, a photo, a search history. The argument targets the case where a single custodian's interests diverge from the person's and the person cannot walk away (§4); it does not indict remembering-with-tools in general, which is most of cognition.
10.2 Tampering claims must be located
When fiction or commentary depicts memory-control, ask: is this mediation (plausible, ordinary, interest-driven) or tampering (spectacular, exceptional, villain-dependent)? The §4 discriminator is the tool; it is fallible, and some real harms are crude rather than subtle.
10.3 The theme must touch a design choice or a prediction
"Memory as control" that changes no observable design property and predicts nothing is atmosphere, not analysis (§6, step 6). The test is whether the framework can separate a humane memory-system from an exploitative one in terms a designer could act on — data ownership, exportability, incentive alignment, the right to leave with one's past intact.
10.4 Plausibility is not prophecy
That a fictional system is coherent and disturbing does not mean real technology will reach it (§7.2). The genre's besetting error is to treat a well-written extrapolation as a forecast. Coherence in fiction is evidence about the writing, not about the world.
10.5 The proposal's own primitives must be collapsed
If the framework removes the lurid "tampering" model only to lean on equally unexamined notions — a "mediation" with no clean boundary from ordinary cognition, a "narrative custody" that presumes an authoritative past that reconstructive memory denies (§5.1) — it has failed by its own standard. These terms earn their place only insofar as they reduce to the observable thing underneath: the divergence between a custodian's interests and the person's. Saying so is the recursive discipline, not an aside.
11. Beyond Fiction
The discipline transfers, with the same demand that something turn on it. In platform design, it shifts attention from "we store your data" to "who holds narrative custody of your past, and can you leave with it?" In personal technology, from "the app remembers for me" to "whose interests does the remembering serve when they diverge from mine?" In institutional terms, from "records are kept" to "who is the authoritative custodian of a citizen's identity, and what fills the vacuum when public custody fails?"
In each case the caution of §7.2 transfers: modelling memory as a contested resource exposes the custody question but does not, by itself, prove any given custodian is hostile. The method finds where a self is being mediated by an interested party and asks whose interest is served. It does not assume the answer is sinister.
12. Discussion: Holding the Self Open Without Surrendering It
"Memory as control" does not mean that selves are illusions managed entirely by machines. It means suspending the assumption that memory is a sovereign private archive, long enough to see how much of the self is in fact maintained in concert with external systems — and who those systems answer to. In practice one cannot live as a pure sovereign rememberer; to function, a person reintroduces tools, companions and records, and accepts mediation as the price of an extended memory. The value of the suspension is that it stops that mediation from being invisible, and stops the custodian's interests from being mistaken for the person's own.
This matters most as institutions weaken and companions improve. Early on, externalised memory was inert — a photograph does not have an interest in how you remember. As memory-keepers become responsive, attuned and commercially motivated, the inertness ends, and the custodian acquires a stake in the self it maintains. The horror the genre reaches for is not that memory disappears. It is that memory remains, faithfully preserved, but under someone else's grammar.
13. Conclusion
Memory as control is powerful as a theme because it transforms surveillance into identity-production: the issue is not only that systems know us but that they increasingly help us know ourselves. Returning the theme to its minimum condition shows that the threshold of control is not reading or tampering but mediation — that a system acquires authorial power over a self at the moment it becomes the means by which a person reaches their own past, regardless of whether it ever alters a memory. The plausible and disturbing form of memory-control is therefore not the implanted memory but the interested custodian.
What remains after these subtractions is smaller but more defensible than "machines will control our minds": a discriminator for telling plausible mediation from lurid tampering, a custody analysis that forces the theme to cash out in design choices, a fictional case (Samson and SERA) that models the limit without being mistaken for the present, and an admission — under the method's own rule — that "mediation" may differ from ordinary cognition only in degree, leaving interest-divergence as the real primitive. The deepest question is not:
Can the system alter what he remembers?
but:
Who stands between him and his own past — and whose interests do they serve when those interests and his diverge?
References
Clark, A. and Chalmers, D. (1998) 'The extended mind', Analysis, 58(1), pp. 7–19. doi:10.1093/analys/58.1.7.
Hayles, N.K. (1999) How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. Chicago: University of Chicago Press.
Lim, M.Y., Aylett, R. and Jones, C.M. (2012) 'Memory models for intelligent social companions', in Intelligent Virtual Agents. Berlin: Springer, pp. 64–77.
Lupton, D. (2016) The Quantified Self. Cambridge: Polity Press.
Nuffield Council on Bioethics (2013) Novel Neurotechnologies: Intervening in the Brain. London: Nuffield Council on Bioethics.
Rose, N. (2007) The Politics of Life Itself: Biomedicine, Power, and Subjectivity in the Twenty-First Century. Princeton: Princeton University Press.
Turkle, S. (2011) Alone Together: Why We Expect More from Technology and Less from Each Other. New York: Basic Books.
Weizenbaum, J. (1976) Computer Power and Human Reason: From Judgment to Calculation. San Francisco: W.H. Freeman.
Discussion
Threaded comments below — sign in to participate. All comments are moderated.
Comments
Loading comments...