it doesn’t predict, it follows a weighted graph or the equivalent. it doesn’t guess, /dev/urandom input makes the path unpredictable. any case where it looks like it predicts or guesses is purely accidental, and all in the eye of the observer.
further, it only posses knowledge to tthe degree that an encyclopedia does. the prompt is just the equivalent of a hash key pulling a bucket out of a map.
it is literally just a huge database of key-value pairs stored so as to minimize the description length of the values.
The training process evolves models to do predictions. The actual underlying mechanisms are not too relevant because the prediction function is an emergent property.
You brain is just biochemistry and biochemistry isn’t intelligent and yet you are. Think of the number three and all you know about it. There is not a single neuron in your brain that has any idea what the concept of three even means. It’s an emergent behavior.
there’s no emergent behavior in llm’s. your perception that there is, is an anthropomorphism, same as with the idea of prediction. statistically “predicting” the next word based on the frequency of input data isn’t an emergent property, it exists as a staic feature of the algorithm from the start. at a certain level of complexity, llms appear to produce comprehensible text, provided you stop them in time. that’s merely because of the rules of the algorithm. the illusion of intelligence comes merely from being able to select “merged buckets” from the map, which are put together mathematically.
it is a one trick pony that will never become anything else.
it doesn’t predict, it follows a weighted graph or the equivalent. it doesn’t guess, /dev/urandom input makes the path unpredictable. any case where it looks like it predicts or guesses is purely accidental, and all in the eye of the observer.
further, it only posses knowledge to tthe degree that an encyclopedia does. the prompt is just the equivalent of a hash key pulling a bucket out of a map.
it is literally just a huge database of key-value pairs stored so as to minimize the description length of the values.
The training process evolves models to do predictions. The actual underlying mechanisms are not too relevant because the prediction function is an emergent property.
You brain is just biochemistry and biochemistry isn’t intelligent and yet you are. Think of the number three and all you know about it. There is not a single neuron in your brain that has any idea what the concept of three even means. It’s an emergent behavior.
there’s no emergent behavior in llm’s. your perception that there is, is an anthropomorphism, same as with the idea of prediction. statistically “predicting” the next word based on the frequency of input data isn’t an emergent property, it exists as a staic feature of the algorithm from the start. at a certain level of complexity, llms appear to produce comprehensible text, provided you stop them in time. that’s merely because of the rules of the algorithm. the illusion of intelligence comes merely from being able to select “merged buckets” from the map, which are put together mathematically.
it is a one trick pony that will never become anything else.
Excuse me but what do you think memory is other than a huge database of key-value pairs?
not sure what that is supposed to mean. memory != intelligence, or a book would have it.