Hallucination
A hallucination is when AI confidently presents false information as if it were true, often with numbers, quotes, or sources.
Hallucinations are a phenomenon in which AI confidently says things that are not true as if they were true. Its characteristic is to create plausible answers instead of saying you don't know, such as citing non-existent papers or inventing precedents that don't exist.
Because LLM is not a database that searches for facts, but a model that probabilistically predicts what to say next, hallucinations occur when plausibility and fact are misaligned. In particular, it occurs frequently in numbers, citations, sources, and person information, and there is a famous case where a lawyer submitted an AI-created precedent to the court.
It can be reduced with search aggregation (RAG) or inference enhancement, but the general opinion is that it is difficult to completely eliminate it with current technology. Therefore, the habit of manually verifying important facts has become a basic rule for using AI.
✅ Why it matters
- It tells you exactly why you should not blindly trust AI answers
- It points out points that particularly require verification, such as numbers, quotations, and sources
- It is a starting point for understanding the flow of technology aimed at reducing hallucinations, such as RAG.
⚠️ Limits and debates
- Because of the confident tone, even incorrect information sounds plausible
- The general opinion is that complete removal is difficult with current technology
- If it is written in business or legal documents without verification, it can lead to real damage.