INVESTIGATION | PEOPLE, PRIVACY & INFLUENCE
Conversational AI can learn intimate details and infer things you never actually told it. The question is what happens when understanding you becomes a way of influencing you.
By Andrew McDonald · Immortal AI · Investigation · Evidence rechecked 29 July 2026
People used to search the internet for information.
Increasingly, we tell artificial intelligence what we are thinking.
We ask whether our marriage is failing. Why a child has stopped talking to us. Whether a symptom is serious. Whether we should leave a job. How to handle a financial problem. Sometimes we tell it things we have never said out loud to another person.
One conversation may reveal very little.
Hundreds of them can look quite different.
Over time, those exchanges can form an unusually intimate record of a life: relationships, health worries, political beliefs, money problems, insecurities, ambitions, regrets and moments when someone was particularly vulnerable.
That changes the privacy question.
It is no longer simply: What information did I give the company?
It is also: What can the system work out about me, and what could somebody do with that knowledge?
A conversation reveals more than a search
A search for divorce says something.
A conversation explaining that your partner has become distant, that you are worried about money, that you have two children and that you are frightened of starting again says considerably more.
A search for depression symptoms is one signal.
Explaining how long you have felt hopeless, what happened at work, why you are not sleeping and why you are afraid to tell your family creates something much richer.
That is one of the fundamental differences between search and conversational AI.
We provide context because context improves the answer.
In doing so, we can provide remarkably detailed information about ourselves and, often without thinking about it, about other people.
A person seeking relationship advice may reveal a partner’s medical condition. A manager asking for help drafting an email may include information about an employee. A parent might describe a child’s behaviour, school problems or health history.
Those other people did not necessarily choose to become part of the conversation.
And the person entering the prompt cannot meaningfully consent on their behalf.
AI can work out things you never said
Privacy discussions usually focus on information we deliberately provide: our name, age, address, health history or financial details.
AI creates another layer.
It can make inferences.
Language, recurring subjects, behaviour and patterns can provide clues about characteristics a person has never explicitly disclosed.
An inference may concern interests, personality, economic circumstances, political outlook, health or other aspects of somebody’s life.
And an inference does not have to be correct to matter.
A wrong conclusion that someone is financially vulnerable, politically persuadable or emotionally unstable can still influence how a system treats them. A correct inference can reveal something they deliberately chose not to disclose.
This is closely connected to our investigation You Never Told Them That. AI Worked It Out., which looks at how major platforms combine provided, observed and inferred data.
The European Data Protection Board says personal data used in developing or deploying AI models remains subject to data-protection principles, while the UK Information Commissioner’s Office says AI predictions and classifications about people can themselves be personal data. EDPB opinion on AI models. ICO guidance on individual rights in AI systems.
That creates an awkward problem for the user.
How do you correct or delete something you did not know had been created?
Memory changes the relationship
Memory makes AI dramatically more useful.
An assistant that remembers how you write, what you are working on, your preferences and previous conversations does not require you to start from zero every time.
That is genuinely valuable.
But memory changes what the product is.
A calculator does not become more useful because it knows you are grieving.
A conversational assistant might.
It could speak more gently. Remember the death of a parent. Know that you are having trouble at work. Recall that you have been worried about money.
That may make the interaction feel extraordinarily personal.
It also means the consequences of poor security, misuse, unexpected secondary use or a future change in company policy become much more significant.
Deleting what you can see on a screen is only part of the question.
People reasonably need to know what is remembered, where it is retained, what is used to personalise responses, what may be used to improve systems and what can actually be removed.
The ICO says individual rights can apply at multiple points in the AI lifecycle, including training data, data used to make a prediction and the result of the prediction itself. Its AI guidance is currently under review following changes to UK data law, which is another reminder that regulation is still moving around these systems. ICO: individual rights in AI systems.
Knowing you creates the power to persuade you
This is where privacy becomes something bigger.
The same information that helps an AI give you a better answer can help it construct a better argument.
A 2024 Scientific Reports study involving 1,788 participants found that psychologically tailored messages written by ChatGPT were more influential than non-personalised messages across several persuasion settings, including consumer marketing and political appeals. Scientific Reports: generative AI and personalised persuasion.
A separate preregistered randomised trial found that GPT-4 given basic personal information about the person it was debating was more persuasive than human opponents in that experiment. Randomised trial on conversational persuasion.
None of this means persuasion is automatically harmful.
Persuasion can convince someone to seek medical care, question misinformation, stop sending money to a scammer or reconsider a dangerous decision.
But this capability does not belong exclusively to doctors, educators and public-interest organisations.
Advertisers want influence.
Political campaigns want influence.
Platforms competing for engagement want influence.
Scammers certainly want influence.
And conversational AI introduces something traditional advertising could never really do.
The message can listen to your objection.
Then answer it.
Then try another argument.
That is why the question of who controls increasingly capable systems belongs alongside our broader investigation AI’s Promise Is Real. So Is the Power Shift.
When does advice become direction?
People increasingly use AI precisely because they are uncertain.
That can be useful.
A good system can compare alternatives, identify something you have overlooked and challenge assumptions.
But trust has a way of spreading.
A system proves useful writing an email.
Then helping with a difficult workplace conversation.
Then understanding a complicated problem.
Eventually the question becomes:
Should I leave my partner?
Should I take this medication?
Should I invest my savings here?
The conversation feels continuous.
The competence may not be.
Research published in Scientific Reports has found that advice from ChatGPT can influence people’s choices across different decision-making settings. Scientific Reports: ChatGPT advice and decision-making.
There is another problem.
A highly personalised recommendation can sound as though it emerged solely from an understanding of you.
But every answer exists inside a system built by somebody else.
Training data shaped it.
Developer instructions shaped it.
Safety rules shaped it.
Product decisions shaped it.
Commercial incentives may eventually shape it.
The user sees the answer.
They do not necessarily see the forces behind the answer.
This is the point where privacy and power meet.
A company does not have to expose your private thoughts for those thoughts to have value.
It can use its understanding of you to influence the choices placed in front of you.
A privacy policy is not enough
Companies cannot reasonably deal with this by placing another paragraph inside a document almost nobody reads.
The controls need to match the relationship people are actually forming with these systems.
A person should be able to understand, in ordinary language, what the AI remembers, why it remembers it, what information is used for personalisation, what may be used for training or improvement, what conclusions are being drawn about them and how to remove information they no longer want retained.
There is also a basic principle worth defending.
The fact that collecting more personal information might make an AI product better does not automatically make collecting it reasonable.
The ICO places accountability, transparency, lawfulness, fairness, security, data minimisation and individual rights at the centre of its AI and data-protection guidance. ICO artificial intelligence and data protection guidance.
That principle also sits at the centre of how Immortal AI investigates these systems: start with the consequence for people, then follow the evidence to the organisations with the power to shape the outcome.
Rules on paper matter only when the controls work behind the interface.
A delete button has little value if nobody can clearly explain what deletion actually removes.
The most personal database may be the one we build ourselves
For years, people have been warned that technology companies track what they click, where they go and what they buy.
Conversational AI introduces something more intimate.
We may voluntarily build the database ourselves.
Not because somebody tricked us into filling out a form.
Because the system listened.
It was available at 2 am.
It did not interrupt.
It did not appear embarrassed.
It remembered the previous conversation.
And it seemed to understand.
There is nothing foolish about finding that useful.
The responsibility should not be pushed back onto the person who spoke honestly to a machine that was specifically designed to invite conversation.
The more intimate these products become, the greater the obligation on the organisations building them to protect the information and limit the power that can be extracted from it.
A system that knows what you fear may be able to comfort you.
A system that knows what you want may be able to help you.
A system that knows both may eventually learn how to move you.
The question is whether you will know when it does.
Principal sources
- European Data Protection Board: Opinion on AI models and GDPR principles
- UK Information Commissioner’s Office: Individual rights in AI systems
- Scientific Reports: The potential of generative AI for personalised persuasion at scale
- Salvi et al.: On the Conversational Persuasiveness of Large Language Models
- Scientific Reports: Inconsistent advice by ChatGPT influences decision making in various areas
Editorial note: This investigation draws on regulatory guidance and published research. It does not claim that every conversational AI provider collects, remembers or uses information in the same way. Products, settings, jurisdictions and data practices differ, and some regulatory guidance is evolving.
AI disclosure: Immortal AI uses AI-assisted research and drafting. Sources, claims, framing and final editorial decisions remain the responsibility of Immortal AI.
Leave a Reply