Then you don’t understand how a modern LLM based ‘AI’ functions and I don’t blame you. It’s extra confusing because putting in data to the thing is called ‘training’ and the marketing materials say it’s artificial intelligence. So why can’t we just train our artificial intelligence to do better?
Well first of all, because an LLM isn’t intelligent at all. That’s just a term we use, that’s applied to a lot of stuff. A few lines of code in a video game so an enemy avoids your shots is called AI. A simple decision tree based system is called AI. A lot of things have the term AI slapped on, which aren’t intelligent at all. The same applies to LLM based chat bots, they get called AI but contain no form of intelligence inside.
So what is an LLM exactly then? Simply put it’s a machine that predicts the next word based on the words that came before. It’s been fed a whole bunch of text from the internet, books and any other source they could get their hands on. With this data they create a model, which given a bunch of words poops out what the next word would most likely be. In practice there’s a lot more to it, but this is the core of the thing. And what we learned was if you create a model large enough, you can feed it a lot of text and it will happily supply a bunch of text that follows. Putting this in a chat format, you can ask it a question and it will give an answer.
So the name LLM stands for large language model, like I said it’s a large model, which means it has been trained on a lot of data and knows the relation between a lot of words. The language part is because the model is specifically for natural language. It’s source data is natural language and the output is natural language. The training is the part where they feed it all of the data.
OK, why does this then mean we can’t train it not to lie anymore? Because the core of the system is predicting which words come next. The LLM system doesn’t know what the meaning of words are, it doesn’t understand anything. It’s just putting together a jigsaw puzzle and slotting in the pieces where they fit. It generates text because it’s internal calculations result in those words being likely based on the previous words. So when asked a question, it will most likely return a properly formatted and grammatically correct answer. There is however no relation between the answer and the truth. It literally hallucinates every answer it gives and because all the source data that was put into it contained hopefully a lot of truths, the answer has a chance of also being true. But it has a chance of not being true as well and if the source data didn’t contain something similar enough to the question, all bets are off and the answer has a high likelihood of not being true.
So what to do to fix it? Early on it was thought to increase the amount of data put into the model and increase the amount of resources the model can use. So let it “know” more and feed it more data. This helps to avoid the questions not being in the source data, or the model not recognizing the question as similar enough. So it should help reduce the wrong outputs right? Alas it turned out not to be. This helps a little bit, but the amount of effort gets exponentially greater and the results only get mildly better. More source data also meant more noise in the data, more truthful answers to a question, but also more false answers. It turned out especially when the model was fed output from earlier models, this messes up the end result.
To get it to behave properly, one would have to feed an infinite amount of data into it. And that data simply isn’t there. All of the good quality data has already been collected and put into it. So this is about as good as it gets. AI companies are going the pump in more resources route, but they are fast running into diminishing returns.
This is a really short and simplified explanation. There is a lot more to it and people are making entire careers in this field. But the core principle is solid. These systems only put in words that seem to fit, true or not. This is the fundamental functionality of the system, so it will always be prone to hallucinations.
So when AI companies tell in their marketing: “Just look at where we were a few years ago and where we are now, imagine where we will be in a couple of years!”. Hopefully you now know to take this with a lot of doubt. They are running into hard limits. Infinite growth isn’t a thing and past results are not a good indication of future results. They need a really big breakthrough, otherwise this technology will mostly fail.
I literally based what I said on seeing papers and video essays by students using generative AI to perform specific tasks, and overcoming this issue. It’s not just about the data it is “learning” but also about how you “reward” it for doing what you intend it to do. Let it figure out how to win at a game, and it will cheat until you start limiting how it is allowed to win.
when you use reinforcement learning to punish the ai for saying “the sky is magenta”, you’re training it to “don’t say the sky is magenta”. You’re not training it to “don’t lie”. What about the infinite other ways the answer could be wrong though?
Then either you and/or those kids don’t understand the tech they are using.
Sure you can use reinforcement training to improve or shape a model in the way you would want it to be. However, as I said, the model doesn’t know what is true and what is not true. That data simply isn’t there and can’t ever be there. So training the model ‘not to lie’ simply isn’t a thing, it doesn’t “know” it’s lying, so it can’t prevent or control lies or truths.
Lets say you create a large dataset and you define in that data set whether something is true or false. This would be a pretty labour intensive job, but possible perhaps (setting aside the issue of truth often being a grey area and not a binary thing). If you instruct it only to re-iterate what is defined as true in the source data, it then loses all freedom. If you ask it a slightly different question that isn’t in the source data, it simply won’t have the data to know if the answer is true or false. So just like the way it currently functions, it will output an answer that seems true. An answer that logically fits after the question. It likes putting in those jigsaw pieces and the ones that fit perfectly must be true right? No, they have just as big of a chance of being totally false. Just because the words seem to fit, doesn’t mean it’s true. You can instruct it not to output anything unless it knows it is true, but that limits the responses to the source data. So you’ve just created a really inefficient and cumbersome search tool.
This isn’t an opinion thing or just a matter of improving the tech. The data simply isn’t there, the mechanisms aren’t there. There is no way an LLM can still do what it does and also tell the truth. No matter how hard the marketing machines are working to convince people it is an actual artificial intelligence, it is not. It’s a text prediction engine and that’s all it will ever be.
Current AI models have been trained to give a response to the prompt regardless of confidence, causing the vast majority of hallucinations. By incorporating confidence into the training and responding with “I don’t know”, similar to training for refusals, you can mitigate hallucinations without negatively impacting the model.
If you read the article, you’ll find the “destruction of ChatGPT” claim is actually nothing more than the “expert” making the assumption that users will just stop using AI if it starts occasionally telling users “I don’t know”, not any kind of technical limitation preventing hallucinations from being solved, in fact the “expert” is agreeing that hallucinations can be solved.
You’ve done a lot of typing and speak very confidently, but ironically, you seem to have only a basic understanding of how an LLM works and how they are trained, and are just parroting talking points that aren’t really correct.
Then you don’t understand how a modern LLM based ‘AI’ functions and I don’t blame you. It’s extra confusing because putting in data to the thing is called ‘training’ and the marketing materials say it’s artificial intelligence. So why can’t we just train our artificial intelligence to do better?
Well first of all, because an LLM isn’t intelligent at all. That’s just a term we use, that’s applied to a lot of stuff. A few lines of code in a video game so an enemy avoids your shots is called AI. A simple decision tree based system is called AI. A lot of things have the term AI slapped on, which aren’t intelligent at all. The same applies to LLM based chat bots, they get called AI but contain no form of intelligence inside.
So what is an LLM exactly then? Simply put it’s a machine that predicts the next word based on the words that came before. It’s been fed a whole bunch of text from the internet, books and any other source they could get their hands on. With this data they create a model, which given a bunch of words poops out what the next word would most likely be. In practice there’s a lot more to it, but this is the core of the thing. And what we learned was if you create a model large enough, you can feed it a lot of text and it will happily supply a bunch of text that follows. Putting this in a chat format, you can ask it a question and it will give an answer.
So the name LLM stands for large language model, like I said it’s a large model, which means it has been trained on a lot of data and knows the relation between a lot of words. The language part is because the model is specifically for natural language. It’s source data is natural language and the output is natural language. The training is the part where they feed it all of the data.
OK, why does this then mean we can’t train it not to lie anymore? Because the core of the system is predicting which words come next. The LLM system doesn’t know what the meaning of words are, it doesn’t understand anything. It’s just putting together a jigsaw puzzle and slotting in the pieces where they fit. It generates text because it’s internal calculations result in those words being likely based on the previous words. So when asked a question, it will most likely return a properly formatted and grammatically correct answer. There is however no relation between the answer and the truth. It literally hallucinates every answer it gives and because all the source data that was put into it contained hopefully a lot of truths, the answer has a chance of also being true. But it has a chance of not being true as well and if the source data didn’t contain something similar enough to the question, all bets are off and the answer has a high likelihood of not being true.
So what to do to fix it? Early on it was thought to increase the amount of data put into the model and increase the amount of resources the model can use. So let it “know” more and feed it more data. This helps to avoid the questions not being in the source data, or the model not recognizing the question as similar enough. So it should help reduce the wrong outputs right? Alas it turned out not to be. This helps a little bit, but the amount of effort gets exponentially greater and the results only get mildly better. More source data also meant more noise in the data, more truthful answers to a question, but also more false answers. It turned out especially when the model was fed output from earlier models, this messes up the end result.
To get it to behave properly, one would have to feed an infinite amount of data into it. And that data simply isn’t there. All of the good quality data has already been collected and put into it. So this is about as good as it gets. AI companies are going the pump in more resources route, but they are fast running into diminishing returns.
This is a really short and simplified explanation. There is a lot more to it and people are making entire careers in this field. But the core principle is solid. These systems only put in words that seem to fit, true or not. This is the fundamental functionality of the system, so it will always be prone to hallucinations.
So when AI companies tell in their marketing: “Just look at where we were a few years ago and where we are now, imagine where we will be in a couple of years!”. Hopefully you now know to take this with a lot of doubt. They are running into hard limits. Infinite growth isn’t a thing and past results are not a good indication of future results. They need a really big breakthrough, otherwise this technology will mostly fail.
I literally based what I said on seeing papers and video essays by students using generative AI to perform specific tasks, and overcoming this issue. It’s not just about the data it is “learning” but also about how you “reward” it for doing what you intend it to do. Let it figure out how to win at a game, and it will cheat until you start limiting how it is allowed to win.
when you use reinforcement learning to punish the ai for saying “the sky is magenta”, you’re training it to “don’t say the sky is magenta”. You’re not training it to “don’t lie”. What about the infinite other ways the answer could be wrong though?
Then either you and/or those kids don’t understand the tech they are using.
Sure you can use reinforcement training to improve or shape a model in the way you would want it to be. However, as I said, the model doesn’t know what is true and what is not true. That data simply isn’t there and can’t ever be there. So training the model ‘not to lie’ simply isn’t a thing, it doesn’t “know” it’s lying, so it can’t prevent or control lies or truths.
Lets say you create a large dataset and you define in that data set whether something is true or false. This would be a pretty labour intensive job, but possible perhaps (setting aside the issue of truth often being a grey area and not a binary thing). If you instruct it only to re-iterate what is defined as true in the source data, it then loses all freedom. If you ask it a slightly different question that isn’t in the source data, it simply won’t have the data to know if the answer is true or false. So just like the way it currently functions, it will output an answer that seems true. An answer that logically fits after the question. It likes putting in those jigsaw pieces and the ones that fit perfectly must be true right? No, they have just as big of a chance of being totally false. Just because the words seem to fit, doesn’t mean it’s true. You can instruct it not to output anything unless it knows it is true, but that limits the responses to the source data. So you’ve just created a really inefficient and cumbersome search tool.
This isn’t an opinion thing or just a matter of improving the tech. The data simply isn’t there, the mechanisms aren’t there. There is no way an LLM can still do what it does and also tell the truth. No matter how hard the marketing machines are working to convince people it is an actual artificial intelligence, it is not. It’s a text prediction engine and that’s all it will ever be.
Current AI models have been trained to give a response to the prompt regardless of confidence, causing the vast majority of hallucinations. By incorporating confidence into the training and responding with “I don’t know”, similar to training for refusals, you can mitigate hallucinations without negatively impacting the model.
If you read the article, you’ll find the “destruction of ChatGPT” claim is actually nothing more than the “expert” making the assumption that users will just stop using AI if it starts occasionally telling users “I don’t know”, not any kind of technical limitation preventing hallucinations from being solved, in fact the “expert” is agreeing that hallucinations can be solved.
You’ve done a lot of typing and speak very confidently, but ironically, you seem to have only a basic understanding of how an LLM works and how they are trained, and are just parroting talking points that aren’t really correct.