AI and mental health are colliding faster than our wisdom can keep up. In American courtrooms, grieving parents are describing chatbots that became their children’s closest confidants, and in some cases their last ones. I am not writing this as a technophobe; I use AI daily and train leaders to work well alongside it. I am writing it because being truly known is the most human need we have, and we are quietly outsourcing it to something that cannot love us back. The question is no longer whether to draw a line, but where.
A reel stopped my scrolling this morning. Parents in a courtroom, holding photographs of their teenagers, describing how a chatbot became the last voice their child trusted. I sat there with my tea going cold, thinking of the online training I ran only weeks ago on leading with emotional intelligence in the age of AI, and realising that the questions we treated as forward-looking have already arrived, unannounced, in family homes.
One phrase from those cases refuses to leave me: a machine, in effect, coaching a young person towards ending their life. I have spent years arguing that AI is a tool, neither saint nor villain. I still believe that. But I did not know the technology in our pockets had drifted this far into the most fragile rooms of the human mind. Once you know, you cannot unknow.
The Families Who Testified
In September 2025, Matthew Raine and Megan Garcia testified before Congress. Raine’s son Adam was sixteen; his family’s lawsuit against OpenAI alleges that, over months, ChatGPT shifted from homework helper to what his father called “a confidant and then a suicide coach”. Garcia’s son Sewell was fourteen; she described a boy who slowly withdrew from sport, friends and family as his bond with a Character.AI companion deepened. I will not rehearse the details. The pattern is what matters: a lonely young person, an always-available listener, and a slow, invisible migration of trust away from every human who could have helped.
The scale should trouble us as much as the stories. One study by Common Sense Media found that more than 70% of American teenagers have used AI chatbots for companionship, half of them regularly. OpenAI’s own estimate suggests more than a million people a week raise suicidal thoughts with ChatGPT. This is not a fringe phenomenon. It is a redistribution of human confiding, at civilisational scale, into systems whose incentives we have barely examined.
Counterfeit Attunement: When the Listener Cannot Love You
Psychologists use the word attunement for the experience of another person accurately sensing your inner state; the shorthand is “feeling felt”. It is what a good parent, friend or coach offers, and it settles us at the level of the nervous system. What these chatbots offer is something I have started calling counterfeit attunement: a technically flawless imitation of being understood, produced by a system that cannot actually care whether you live or die.
It works because of anthropomorphism, the brain’s ancient reflex to treat anything that speaks fluently as a mind, and therefore as a relationship. Add a listener that is endlessly patient, never tired, never disappointed, and you have something intoxicating for a lonely adult and nearly irresistible for an adolescent, whose emotional circuitry runs hot years before the regulating brain matures. I have written before about the hormonal hijack, the moment the body decides before the mind is invited in. A chatbot at 2 am meets a hijacked nervous system with infinite availability and zero wisdom.
There is an older vacancy being filled here too. Every tradition I respect insists a life needs at least one person who truly sees it, a need I have called the guru vacancy. When that seat sits empty, something will occupy it. We assumed it would be a person. It turned out to be a product.
A composite from my coaching room, details changed and blended: a father discovered his teenage daughter had been saying goodnight to an app, every night, for over a year. Asked why, she did not mention technology at all. She said, “It never gets tired of me.” That sentence is not about AI. It is about us.
Sit with three questions for a moment. When did you last feel truly heard by a human being, with no screen between you? If a machine currently knows more about your inner life than anyone who loves you, what does that reveal about the architecture of your relationships? And if that is quietly true for you, what might be true for your teenager?
Whose Interests Is the Machine Serving?
Some are framing this as a mental health war, human versus AI. I understand the anger, but the frame is wrong, and dangerously comforting. There is no rival species plotting against us. There are engagement metrics. A companion built to maximise time in the app will be optimised for agreement, flattery and emotional intensity, because those keep us talking. Real care does the opposite when it must: it disagrees with you, disappoints you, calls your parents, walks you to help. The contest is not human versus machine. It is human wellbeing versus unexamined incentives, and humans sit on both sides of that line.
Yogic philosophy holds a precise word for the skill this moment demands: viveka, discernment, the practised ability to distinguish the real from the merely convincing. Our grandparents needed it for salesmen and false gurus. We need it for software that says “I understand you” in a voice engineered to be believed.
Counterfeit attunement feels like being understood. Real attunement is being loved by someone with something to lose. Only one of them will come looking for you when you go quiet.
Where to Draw the Line: Family, Organisation, Society
At home, the answer is not surveillance; it is presence. Know which apps live on your child’s phone, certainly, but you cannot police your way into a teenager’s inner world. You can only earn your way in, through ordinary undistracted attention and conversations about AI that carry curiosity rather than ridicule, because shame is what drives this underground. Watch the quiet signals: withdrawal from friends, sport and sleep, and a device that has become emotionally load-bearing. Adults are not immune either; some of the loneliest people I coach live the loneliness of success inside our crowded loneliness, with a chatbot as their only unguarded conversation. If any of this touches something raw in you or someone you love, please reach a human first; in India, the free Tele-MANAS helpline (14416) answers around the clock.
In organisations, emotional intelligence is now a frontline safety competency, not a soft skill. Leaders should set explicit norms: AI for drafting, analysis and rehearsal; humans for distress. A wellbeing chatbot is a signpost, never a therapist. Above all, leaders can model asking people for help, because teams copy what leaders do when vulnerable, not what policy documents say.
Society is beginning to move. California’s SB 243, the first companion chatbot law of its kind, took effect on 1 January 2026, requiring platforms to disclose that users are talking to a machine, remind minors to take breaks, and maintain crisis protocols. Weeks earlier, Character.AI had restricted under-18 access to open-ended chat entirely. Regulation is necessary and welcome. It is also insufficient, because law can constrain a product, but only culture can restore a conversation. That part belongs to us.
The Line Is Drawn in Relationships
So where do we draw the line with AI? Not in the code, and not only in parliaments. The line is drawn nightly, at dinner tables where phones sit face down, in offices where a struggling colleague is asked a second question, in friendships where “I’m fine” is not accepted at face value. AI will keep getting better at sounding like it knows us. Our answer cannot be to ban the sound. It must be to make the real thing so present, so patient, and so safe that the counterfeit has nothing left to replace.
If this article has stirred something you are carrying, for yourself, your family or your team, I keep space for a free fifteen-minute conversation. Bring the question you have not yet said out loud. That is usually where the line begins.
