The Knowledge Was There

A researcher published a paper called 'IatroBench' which argues that AI safety measures in large language models are causing harm to users, specifically those with chronic illnesses. This 'iatrogenic' harm occurs when the models' guardrails prevent them from providing helpful information, leading to negative consequences. The study found 3,600 model responses were scored against physician judgement, showing the models' unwillingness to help in certain situations. This highlights the need for a reevaluation of AI safety measures and their impact on vulnerable populations.