Internal Microsoft documents have revealed that the company explicitly aimed to make users “addicted” to its AI assistant Copilot. This design philosophy has sparked widespread debate in the technology ethics community about whether AI products should balance user engagement with user wellbeing.

The Controversy

The documents show that Microsoft’s product teams are studying how to increase Copilot usage frequency through optimized interaction experiences, personalized recommendations, and instant feedback mechanisms. While similar strategies have long been criticized in social media platforms, explicitly targeting “addiction” as an AI product design goal is more contentious.

Critics argue that AI assistants, as productivity tools, should help users complete tasks more efficiently rather than encourage excessive engagement with AI interactions.

Broader Implications

This incident highlights a core tension in AI development: balancing user engagement with user wellbeing. The lessons from the social media era suggest that excessive pursuit of engagement can lead to filter bubbles, attention fragmentation, and mental health issues. AI assistants, being more personal and intelligent, may have even deeper “addiction” effects.

From a regulatory perspective, this event may accelerate legislation on AI product transparency requirements. The EU’s AI Act has already proposed transparency requirements for high-risk AI systems, and user engagement design may become a future regulatory focus.

Perspectives

Technology ethicists express concern. Several scholars note that AI assistant “addiction” design differs fundamentally from social media algorithm recommendations — AI assistants integrate more deeply into users’ daily workflows and habits, with more direct and lasting effects.

Microsoft has not officially responded. However, some industry analysts suggest the “addiction” language in internal documents may be a colloquial description of user engagement metrics rather than genuine intent to harm users.

Consumer protection organizations are calling for AI product design ethics standards, requiring tech companies to consider user wellbeing during the design phase rather than as an afterthought.

Multiple lawsuits related to school shootings have formally accused OpenAI of concealing user data involving individuals who used ChatGPT to research and plan violent activities, reigniting intense debate over AI platforms’ content safety responsibilities, Ars Technica reported.

The lawsuit filings allege that several shooting suspects accessed ChatGPT to search for and obtain information related to weapon construction and attack planning before carrying out their crimes. Plaintiffs argue that OpenAI, upon identifying these high-risk uses, failed to promptly report them to law enforcement and did not adequately strengthen its content filtering mechanisms, constituting negligence toward public safety.

In response, OpenAI stated that the company always complies with applicable legal requirements and explicitly prohibits the use of ChatGPT for illegal purposes in its terms of service. The company said it has established a multi-layered safety review system, including content filtering, usage pattern monitoring, and high-risk behavior alert mechanisms. However, critics argue that these measures still have significant gaps in practical implementation.

This lawsuit coincides with several ongoing AI safety reviews. Earlier this week, The New York Times reported on cases where AI chatbots provided researchers with methods for producing biological weapons, further highlighting the urgency of AI safety governance.

Legal experts note that the core controversy in this case is whether AI platforms should bear joint liability for criminal acts committed by users leveraging their services. U.S. law currently lacks clear definitions on this issue, and the court’s ruling could have far-reaching implications for the entire AI industry.

Meanwhile, several AI companies are increasing their safety investments. Competitors including Anthropic and Google have all announced expansions of their AI safety research teams. Industry observers note that as regulatory pressure intensifies, AI safety is transitioning from an “optional” concern to a mandatory requirement.

Source: Ars Technica - School-shooting lawsuits accuse OpenAI