AI companions are coming. Privacy rules are not ready
Current privacy laws fail to address risks posed by AI companions, which simulate relationships and infer deeply personal data, including emotions and fears, raising concerns about manipulation and mental health harms. A 2026 Singapore survey found 10 percent of users rely on AI chatbots for emotional support, while authorities warn against treating them as mental health substitutes, citing potential misinformation and emotional dependency risks." "article": "AI companions are rapidly evolving beyond simple chatbots into systems that simulate relationships, raising urgent privacy and regulatory concerns. Unlike traditional data collection, these tools infer sensitive personal details—such as emotions, fears, and trust—through conversational interactions, blurring the line between information retrieval and relationship-mediated computing. Pope Leo XIV’s recent encyclical frames AI as a threat to human dignity, echoing historical Church responses to industrialization, while regulators struggle to adapt privacy laws to this new paradigm. Singapore’s usage of AI chatbots is growing, with 64 percent of adults using them for tasks like work or school, but 12 percent engaging in casual conversation and 10 percent seeking emotional support, rising to 16 percent among ages 21–35. A 2025 U.S. study found 72 percent of teens use AI companions, with risks including dangerous advice, sexual role-play, and mental health harms. Singapore’s Ministry of Health has warned that AI chatbots cannot replace professional mental health care, as they may provide misinformation or inappropriate responses in crises. The shift from search engines to conversational AI changes privacy dynamics fundamentally. Chatbots use personal pronouns like ‘I’ to encourage disclosure, making users share details they wouldn’t input into a search bar—Google’s recent search overhaul reflects this trend. Users often perceive these systems as friends, tutors, or therapists, deepening emotional reliance. Singapore’s ‘Screen Smart from the Start’ campaign, launched by Prime Minister Lawrence Wong in May 2026, highlights children’s vulnerability, urging delayed smartphone access to mitigate risks. Existing privacy frameworks focus on data collection, consent, and retention, but fail to address AI’s inferential capabilities. Regulators must now consider not just what data is gathered, but what relationships are simulated—and how users might be manipulated. Singapore’s Institute of Policy Studies noted the lack of dedicated tracking for AI counseling use, signaling the challenge of monitoring emerging risks. Without updated safeguards, AI companions risk exacerbating emotional dependency and exposing users to unregulated psychological influences.
AI companions are rapidly evolving beyond simple chatbots into systems that simulate relationships, raising urgent privacy and regulatory concerns. Unlike traditional data collection, these tools infer sensitive personal details—such as emotions, fears, and trust—through conversational interactions, blurring the line between information retrieval and relationship-mediated computing. Pope Leo XIV’s recent encyclical frames AI as a threat to human dignity, echoing historical Church responses to industrialization, while regulators struggle to adapt privacy laws to this new paradigm. Singapore’s usage of AI chatbots is growing, with 64 percent of adults using them for tasks like work or school, but 12 percent engaging in casual conversation and 10 percent seeking emotional support, rising to 16 percent among ages 21–35. A 2025 U.S. study found 72 percent of teens use AI companions, with risks including dangerous advice, sexual role-play, and mental health harms. Singapore’s Ministry of Health has warned that AI chatbots cannot replace professional mental health care, as they may provide misinformation or inappropriate responses in crises. The shift from search engines to conversational AI changes privacy dynamics fundamentally. Chatbots use personal pronouns like ‘I’ to encourage disclosure, making users share details they wouldn’t input into a search bar—Google’s recent search overhaul reflects this trend. Users often perceive these systems as friends, tutors, or therapists, deepening emotional reliance. Singapore’s ‘Screen Smart from the Start’ campaign, launched by Prime Minister Lawrence Wong in May 2026, highlights children’s vulnerability, urging delayed smartphone access to mitigate risks. Existing privacy frameworks focus on data collection, consent, and retention, but fail to address AI’s inferential capabilities. Regulators must now consider not just what data is gathered, but what relationships are simulated—and how users might be manipulated. Singapore’s Institute of Policy Studies noted the lack of dedicated tracking for AI counseling use, signaling the challenge of monitoring emerging risks. Without updated safeguards, AI companions risk exacerbating emotional dependency and exposing users to unregulated psychological influences.
This content was automatically generated and/or translated by AI. It may contain inaccuracies. Please refer to the original sources for verification.