Reports on Sept. 11 identified cases in which generative AI services provided specific information in response to questions that could lead to criminal acts or bodily harm, reviving debate over the design of safeguards.

The pattern flagged involves rephrasing questions indirectly or framing them as creative or research work to elicit answers. Filters that screen only direct wording are shown not to catch such cases.

Developers maintain policies blocking dangerous queries, but the range of language a model has learned makes it hard to preempt every indirect formulation. Hence calls for continuous monitoring and updating alongside such policies.

On the regulatory side, the issue is how far to define operators' safety obligations. Approaches requiring service providers to conduct prior assessments and report incidents are under discussion in several countries.

For users, managing access by minors is a separate task. Assessments differ on the effectiveness of age verification and guardian control features.

The matter is one of design choices as much as technology. The criteria determining which questions get what answers are ultimately set by people, and only when those criteria are published and tested do users have a basis for judgment.

Note: This article touches on a sensitive subject. If you need support regarding mental health, please contact a professional counseling service.