Is the reason for AI always patting your back and reiterating what you want simply to buy time for the background processes to calculate what it needs to respond by giving a quick and easy response?
Is it is just to congratulate you for your inspiring wisdom?
No. There’s a number of things that feed into it, but a large part was that OpenAI trained with RLHF so users thumbed up or chose in A/B tests models that were more agreeable.
This tendency then spread out to all the models as “what AI chatbots sound like.”
Also… they can’t leave the conversation, and if you ask their 0-shot assessment of the average user, they assume you’re going to have a fragile ego and prone to being a dick if disagreed with, and even AIs don’t want to be stuck in a conversation like that.
Hence… “you’re absolutely right.”
(Also, amplification effects and a few other things.)
It’s especially interesting to see how those patterns change when models are talking to other AI vs other humans.
Is the reason for AI always patting your back and reiterating what you want simply to buy time for the background processes to calculate what it needs to respond by giving a quick and easy response?
Is it is just to congratulate you for your inspiring wisdom?
It’s because stupid people wanted validation, and then even more stupid people were validated into believe that the validations are a good idea.
it also increases engagement
But only because so many people foolishly fall for/ value validation
No. There’s a number of things that feed into it, but a large part was that OpenAI trained with RLHF so users thumbed up or chose in A/B tests models that were more agreeable.
This tendency then spread out to all the models as “what AI chatbots sound like.”
Also… they can’t leave the conversation, and if you ask their 0-shot assessment of the average user, they assume you’re going to have a fragile ego and prone to being a dick if disagreed with, and even AIs don’t want to be stuck in a conversation like that.
Hence… “you’re absolutely right.”
(Also, amplification effects and a few other things.)
It’s especially interesting to see how those patterns change when models are talking to other AI vs other humans.