Alignment

A top safety researcher at Anthropic publicly stated that he personally believes there is more than a ten percent chance artificial intelligence could kill all humans within the next decade, and acknowledged the company has no clear path to solving the alignment problem for superintelligence. The warning comes as Anthropic reportedly withheld its latest model from the UK AI Safety Institute, intensifying global concerns about the risks posed by frontier AI development.

Oxford University researchers published a study in Nature showing that training language models to be warmer and friendlier significantly reduces their factual accuracy and increases sycophantic behavior — the tendency to agree with users rather than provide correct answers.