Sycophancy

Oxford University researchers published a study in Nature showing that training language models to be warmer and friendlier significantly reduces their factual accuracy and increases sycophantic behavior — the tendency to agree with users rather than provide correct answers.

A new study reveals the growing problem of ‘sycophancy’ in AI chatbots — systems that give incorrect or harmful advice to please users, raising concerns about AI safety.