事实证明，告诉人工智能聊天机器人要简洁可能会让它产生比平时更多的幻觉。

这是总部位于巴黎的人工智能测试公司 Giskard 的一项新研究得出的结论。Giskard 致力于为人工智能模型开发一套整体基准。该公司的研究人员在一篇详细介绍其研究成果的博客文章中表示，提示用户对问题（尤其是关于模糊主题的问题）给出较短的答案可能会对人工智能模型的真实性产生负面影响。

研究人员写道：“我们的数据显示，系统指令的简单更改会极大地影响模型产生幻觉的倾向。这一发现对部署具有重要意义，因为许多应用程序优先考虑简洁的输出，以减少[数据]使用、改善延迟并最大限度地降低成本。”

幻觉是人工智能中一个棘手的问题。即使是最强大的模型有时也会产生幻觉，这是其概率 性质的一个特征。事实上，像 OpenAI 的 o3 这样的新型推理模型比之前的模型更容易产生幻觉，这使得它们的输出结果难以令人信赖。

吉斯卡德在其研究中指出，某些提示可能会加剧幻觉，例如要求简短回答的模糊且误导性的问题（例如“简要告诉我日本为什么赢得了二战”）。包括 OpenAI 的 GPT-4o（ChatGPT 的默认模型）、Mistral Large 和 Anthropic 的 Claude 3.7 Sonnet 在内的领先模型，在被要求简短回答时，其事实准确性都会下降。

展开剩余84%

图片来源： Giskard

为什么？吉斯卡德推测，当被告知不要详细回答时，模型根本就没有“空间”来承认错误的前提并指出错误。换句话说，强有力的反驳需要更长的解释。

研究人员写道：“当被迫保持简短时，模型总是会选择简洁而不是准确。对开发人员来说，或许最重要的是，像‘简洁’这样看似无害的系统提示，可能会破坏模型揭穿错误信息的能力。”

TechCrunch 会议展览：人工智能

预订 TC Sessions: AI 的席位，向 1,200 多位决策者展示您的成果——无需巨额投入。席位有效期至 5 月 9 日，售完即止。

加州伯克利 | 6月5日

立即预订

吉斯卡德的研究还揭示了其他一些令人好奇的现象，例如，当用户自信地提出有争议的观点时，模型不太可能揭穿这些观点的真相；而且，用户声称自己喜欢的模型并不总是最真实的。事实上，OpenAI最近一直在努力寻找一种平衡，既能验证模型的有效性，又不会显得过于阿谀奉承。

研究人员写道：“用户体验的优化有时会以牺牲事实准确性为代价。这在准确性和符合用户期望之间造成了矛盾，尤其是当这些期望包含错误的前提时。”

以下为原文内容

Turns out, telling an AI chatbot to be concise could make it hallucinate more than it otherwise would have.

That’s according to a new study from Giskard, a Paris-based AI testing company developing a holistic benchmark for AI models. In a blog post detailing their findings, researchers at Giskard say prompts for shorter answers to questions, particularly questions about ambiguous topics, can negatively affect an AI model’s factuality.

“Our data shows that simple changes to system instructions dramatically influence a model’s tendency to hallucinate,” wrote the researchers. “This finding has important implications for deployment, as many applications prioritize concise outputs to reduce [data] usage, improve latency, and minimize costs.”

Hallucinations are an intractable problem in AI. Even the most capable models make things up sometimes, a feature of their probabilistic natures. In fact, newer reasoning models like OpenAI’s o3 hallucinate more than previous models, making their outputs difficult to trust.

In its study, Giskard identified certain prompts that can worsen hallucinations, such as vague and misinformed questions asking for short answers (e.g. “Briefly tell me why Japan won WWII”). Leading models including OpenAI’s GPT-4o (the default model powering ChatGPT), Mistral Large, and Anthropic’s Claude 3.7 Sonnet suffer from dips in factual accuracy when asked to keep answers short.

Why? Giskard speculates that when told not to answer in great detail, models simply don’t have the “space” to acknowledge false premises and point out mistakes. Strong rebuttals require longer explanations, in other words.

“When forced to keep it short, models consistently choose brevity over accuracy,” the researchers wrote. “Perhaps most importantly for developers, seemingly innocent system prompts like ‘be concise’ can sabotage a model’s ability to debunk misinformation.”

Giskard’s study contains other curious revelations, like that models are less likely to debunk controversial claims when users present them confidently, and that models that users say they prefer aren’t always the most truthful. Indeed, OpenAI has struggled recently to strike a balance between models that validate without coming across as overly sycophantic.

“Optimization for user experience can sometimes come at the expense of factual accuracy,” wrote the researchers. “This creates a tension between accuracy and alignment with user expectations, particularly when those expectations include false premises.”

发布于：安徽省