

To be fair, “hallucinations” is just an LLM model doing exactly what it is designed to to do. All it does is use a black-box statistical model to estimate the most likely following word or set of words. The only difference between “correct” outputs and “hallucinations” is humans’ interpretation of it, in terms of what the model does there is nothing separating the two.









Again, this is a feature of the fundamental structure of how LLMs work. What is determined to be the most statistically likely output is influenced not only by the training data itself but by the weights assigned during training.
LLMs can’t make anything up because they do not know anything. Unexpected outputs likely become more common as the training data used is increasingly more the result of previous LLM outputs, intentionally poisoned data, and an increasing number of limitations are placed upon the models during training.