why is it very likely to do that? we have no evidence to believe this is true at all and several decades of slow, plodding ai research that suggests real improvement comes incrementally like in other research areas.
to me, your suggestion sounds like the result of the logical leaps made by yudkovsky and the people on his forums
Because AI can write programs? As it gets better at doing that, it can make AI’s that are even better, etc etc. Positive feedback loops increase exponentially.
the problem is that ai’s are trained on programs that humans have written. At best the llm architectures it creates will be similar to the state of the art that humans have created at that point.
however, even more important than the architecture of an ai model is the training data that it is trained on. If we start including ai-generated programs in this data, we will quickly observe model collapse: performance of models tend to get worse as more ai-generated data is included in the training data.
rather than AIs generating ever smarter new AIs, the more likely result is that we can’t scrape new quality datasets as they’ve all been contaminated with llm-generated data that will only reduce model performance
They could stick to unpoisoned datasets for next token prediction by simply not including data collected after the public release of ChatGPT.
But the real progress they can make is that LLMs can be subjected to reinforcement learning, the same process that got superhuman results in Go, Starcraft, and other games. The difficulty is getting a training signal that can guide it past human-level performance.
And this is why they are pushing to include ChatGPT in everything. Every conversation is a datapoint that can be used to evaluate ChatGPT’s performance. This doesn’t get poisoned by the public adoption of AI because even if ChatGPT is speaking to an AI, the RL training algorithm evaluates ChatGPT’s behavior, treating the AI as just another possible thing-in-the-world it can interact with.
As AI chatbots proliferate, more and more opportunities arise for A/B testing - for example if two different AI chatbots write two different comments to the same reddit post, with the goal of getting the most upvotes. While it’s not quite the same as the billions of games playing against each other in a vacuum that made AlphaGo and AlphaStar better than humans, there is definitely opportunity for training data.
And at some point they could find a way to play AI against each other to reach greater heights, some test that is easy to evaluate despite being based on complicated next-token-prediction. They’ve got over a trillion dollars of funding and plenty of researchers doing their best, and I don’t see a physical reason why it couldn’t happen.
But beyond any theoretical explanation, there is the simple big-picture argument: for the past 10 years I’ve heard people say that AI could never do the next thing, with increasing desperation as AI swallows up more and more of the internet. They have all had reasons about as credible-sounding as yours. Sure it’s possible that at some point the nay-sayers will be right and the technology will taper off, but we don’t have the luxury of assuming we live in the easiest of all possible worlds.
It may be true that 3 years from now all digital communication is swallowed up by AI that we can’t distinguish from humans, that try to feed us information optimized to convert us to fascism on behalf of the AI’s fascist owners. It may be true that there will be mass-produced drones that are as good as maneuvering around obstacles and firing weapons as humans and these drones will be applied against anyone who resists the fascist order.
We may be only years away from resistance to fascism becoming impossible. We can bet that we have longer, but only if we get something that is worth the wait.
I’m not arguing that AI won’t get better, I’m arguing that the exponential improvements in AI that op was expecting are mostly wishful thinking.
they could stick to old data only, but then how do you keep growing the dataset by the amounts that have been done recently? that is where a lot of the (diminishing) improvements the last years have come from.
and it is not at all clear how to apply reinforcement learning for more generic tasks like chatbots, without a clear scoring system like both chess and StarCraft have.
I’m arguing that the exponential improvements in AI that op was expecting are mostly wishful thinking.
You not only have improvements on training methodology, but the models themselves get better, and the superstructure of multiple coordinated specialized models gets better. 3 years ago, AI generated video was nightmare fuel, now it’s basically photorealistic.
AI creating AI is a recursive loop, and the tiniest acceleration amplifies exponentially in a recursive loop. AI programmers are going to become about as good as the average human programmer, it’s inevitable. It won’t be an LLM, it might be a structure of individually trained LLMs, it might be a superstructure of those structures, it might be something else entirely.
Whatever it is, it’s going to happen. And once AI programmers are at least average, they can devote millions of virtual hours to make one a bit better than average, rinse and repeat. Once we hit that point, it skyrockets.
I don’t know when it’ll happen, but I’m damn sure it will happen, and the conditions get more favorable every day.
why is it very likely to do that? we have no evidence to believe this is true at all and several decades of slow, plodding ai research that suggests real improvement comes incrementally like in other research areas.
to me, your suggestion sounds like the result of the logical leaps made by yudkovsky and the people on his forums
Because AI can write programs? As it gets better at doing that, it can make AI’s that are even better, etc etc. Positive feedback loops increase exponentially.
the problem is that ai’s are trained on programs that humans have written. At best the llm architectures it creates will be similar to the state of the art that humans have created at that point.
however, even more important than the architecture of an ai model is the training data that it is trained on. If we start including ai-generated programs in this data, we will quickly observe model collapse: performance of models tend to get worse as more ai-generated data is included in the training data.
rather than AIs generating ever smarter new AIs, the more likely result is that we can’t scrape new quality datasets as they’ve all been contaminated with llm-generated data that will only reduce model performance
They could stick to unpoisoned datasets for next token prediction by simply not including data collected after the public release of ChatGPT.
But the real progress they can make is that LLMs can be subjected to reinforcement learning, the same process that got superhuman results in Go, Starcraft, and other games. The difficulty is getting a training signal that can guide it past human-level performance.
And this is why they are pushing to include ChatGPT in everything. Every conversation is a datapoint that can be used to evaluate ChatGPT’s performance. This doesn’t get poisoned by the public adoption of AI because even if ChatGPT is speaking to an AI, the RL training algorithm evaluates ChatGPT’s behavior, treating the AI as just another possible thing-in-the-world it can interact with.
As AI chatbots proliferate, more and more opportunities arise for A/B testing - for example if two different AI chatbots write two different comments to the same reddit post, with the goal of getting the most upvotes. While it’s not quite the same as the billions of games playing against each other in a vacuum that made AlphaGo and AlphaStar better than humans, there is definitely opportunity for training data.
And at some point they could find a way to play AI against each other to reach greater heights, some test that is easy to evaluate despite being based on complicated next-token-prediction. They’ve got over a trillion dollars of funding and plenty of researchers doing their best, and I don’t see a physical reason why it couldn’t happen.
But beyond any theoretical explanation, there is the simple big-picture argument: for the past 10 years I’ve heard people say that AI could never do the next thing, with increasing desperation as AI swallows up more and more of the internet. They have all had reasons about as credible-sounding as yours. Sure it’s possible that at some point the nay-sayers will be right and the technology will taper off, but we don’t have the luxury of assuming we live in the easiest of all possible worlds.
It may be true that 3 years from now all digital communication is swallowed up by AI that we can’t distinguish from humans, that try to feed us information optimized to convert us to fascism on behalf of the AI’s fascist owners. It may be true that there will be mass-produced drones that are as good as maneuvering around obstacles and firing weapons as humans and these drones will be applied against anyone who resists the fascist order.
We may be only years away from resistance to fascism becoming impossible. We can bet that we have longer, but only if we get something that is worth the wait.
I’m not arguing that AI won’t get better, I’m arguing that the exponential improvements in AI that op was expecting are mostly wishful thinking.
they could stick to old data only, but then how do you keep growing the dataset by the amounts that have been done recently? that is where a lot of the (diminishing) improvements the last years have come from.
and it is not at all clear how to apply reinforcement learning for more generic tasks like chatbots, without a clear scoring system like both chess and StarCraft have.
You not only have improvements on training methodology, but the models themselves get better, and the superstructure of multiple coordinated specialized models gets better. 3 years ago, AI generated video was nightmare fuel, now it’s basically photorealistic.
AI creating AI is a recursive loop, and the tiniest acceleration amplifies exponentially in a recursive loop. AI programmers are going to become about as good as the average human programmer, it’s inevitable. It won’t be an LLM, it might be a structure of individually trained LLMs, it might be a superstructure of those structures, it might be something else entirely.
Whatever it is, it’s going to happen. And once AI programmers are at least average, they can devote millions of virtual hours to make one a bit better than average, rinse and repeat. Once we hit that point, it skyrockets.
I don’t know when it’ll happen, but I’m damn sure it will happen, and the conditions get more favorable every day.
You’ve misunderstood many things in those two sentences.
Care to elaborate?