The point of integrating AI into games is to provide further diversity within the game.
Think Skyrim. By default you’re limited to 3-4 discussion options, right? Imagine now, if you will, that you could just… type in anything, including emotional markers, and have the characters respond interactively to the statement and tone. No longer are you bound by limited dialogue in RPGs.
visual generative AI will just spice up the visuals - hopefully. Things like repetitive textures and such will disappear as the game generates brand new textures for each grid element. Or create tons of background characters without the need to specify them. The list goes on.
I dream of AI being used to generate cool, theme appropriate NPCs for an indie dev that doesn’t have an army of talented writers at their disposal.
But honestly let’s face it, that’s never what they’ll aim for. It’s all about not paying people, for these assholes. They’ll happily fire their talented writers of they can use AI instead, never once wondering who’s gonna feed the AI in the first place.
And so it is a matter of principle to say no to AI in general, because you know they’ll always pick the worst possible way to use it.
You have optimism. It could be used for good like you describe, but most likely will be used by EA to cut artists out, slash costs, make as much profit as possible and suffocate the company until the private equity owners move on to the next company.
No, that’s exactly what this is about. They came right out and said as much. It won’t work, but they’ll cause a lot of damage in the process of failing.
Which is a different article about a (somewhat) unrelated topic.
Using AI for development is already out there, and you can’t put that genie back in the bottle. As an engineer I’m already using it in my daily work for tons of things - I’ve built separate agents to do a number of things:
read work tickets, collate resources, create a work plan, do the initial footwork (creating branches, moving tickets to the right states, creating Notion document with work plan and resources)
read relevant changes in UI design documents and plan + execute changes (still needs some manual review but e.g. with Android Jetpack Compose, it makes 90-95% of the needed work and requires minimal touch-up)
do structural work - boilerplates, etc.
write unit and integration tests, and already working out a UI test automation agent
do code reviews on changes, document them, and write appropriate commit messages
do PR reviews - I still review them myself but an extra eye is always helpful
guess what, AI didn’t replace me, it just allowed me to focus on actually thinking up solutions instead of doing hours of boilerplate stuff.
AI isn’t the enemy in software development. Companies who think they can replace engineers with AI. Middle managers will sooner be on that date, as they were mostly useless anyway.
A project costs X amount because of the standard template of pay per time unit Y multiplied by timeframe in time unit Z.
Simply said if you have 100 people working on the project, that costs 100Y per hour. If the project takes 6 months (approx. 960 hours), you multiply the two and get that your costs are 96000Y.
Now the two ways to reduce this is to either reduce the number of employees, with AI you can get rid of maybe 2/3, reducing the expenses to 32000Y…
Or since AI speeds up almost every workflow by about 8 to 10 times, you can keep all the people, but cut down project time from 6 months to about 2 months, which doesn’t just reduce the expenses by the same 2/3 but also increases potential profits for the same 6 month period by 200%, as instead of one product you’re releasing three.
Cutting jobs ain’t the only way to reduce costs with AI.
Your experience counts for jack shit. There is zero evidence that AI is substantially improving efficiency. There is some that suggests its effect is negative.
Great for you. You did say “almost every workflow”. How many workflows exist beyond your own lived experience? Do you work on games, do you know all the workflows there? Citation absolutely fucking needed.
I have worked on games, and have a good understanding of the workflows involved.
You’ll obviously still need to do the creative parts manually (and should!) but the majority of the work involving the engine core build and the specific game coding, that can all be sped up borderline exponentially.
But I’m glad that someone with absolutely no understanding of the topic does their best to call out those who do show some experience on the topic just because they don’t get a neatly pre-chewed and pre-digested reply detailing all the information they lack and are unwilling to look it up themselves. As a next step would you like me to cut your steak up and feed it to you byte by byte, or tuck you in at night?
Procedural generation with appropriate constraints and a connected game that stores and recalls what’s been created can do this far better than a repurposed LLM. It’s hard work on the front end but you have a much better idea of what the output will be vs. hoping the LLM “understands” and remembers the context as it goes.
I agree that the results will be different, and certainly a very narrowly trained LLM for conversation could have some potentials if it has proper guardrails. So either way there’s a lot of prep beforehand to make sure the boundaries are very clear. Which would work better is debatable and depends on the application. I’ve played around with plenty of fine tuned models, and they will get off track contextually with enough data. LLMs and procedural generation have a lot in common, but the latter is far easier to manage predictable outputs because of how the probability is used to create them.
It would be interesting if NPCs could react believably to free form input from the player, but that seems unlikely to be reliable. I’m not sure how you’d map the text to in game actions.
Like, you could have the LLM respond with “ok I’ll let you have my horse if you leave in the morning, and solve the zombie problem” but it’s not trivial to make the game world respond to that. Not even counting it breaking out into “sorry as an LLM I can’t …”
I’ve seen prototypes of RPGs where you could freeform talk to NPCs and I pretty quickly lost enthusiasm for the idea after seeing it in action.
It didn’t feel like a DnD game where you’re maneuvering a social conflict with the DM or other players, it felt more like the social equivalent of jumping up on a table where an NPC couldn’t get to you and stabbing them in the face.
If I wanted to talk to NPCs at length I’d just type into a fucking chat bot. I play games to experience the developers story and vision, not endlessly prompt an npc.
I think your experience will be a common one for players.
LLM can already do tool calling, emotion metadata output and so on. It would take minimal effort for a well tuned model to also output things like facial expressions, body language, hand and body movements and so on.
The tool calling part might be doable (though I’ve personally struggled to get it working passably with local models), but if the goal is to tell a compelling story or create an interesting experience, especially if doing so in a very open ended way, that isn’t trivial. The only LLM based game I’ve personally played that seemed good was mostly on rails and partially scripted, most of them just aren’t very interesting to play, because the model doesn’t have a good idea where it’s going with anything and is often not very creative, the stuffy personality of the instruct model seems to infect the dialogue and apparent thought process of the characters. For a specific example I’d recommend watching streams of the game Suck Up, which has a genuinely cool concept and solid execution, but you can see people being frustrated running into its limitations as something to interact with creatively.
I’ve tried a couple times to start game projects involving LLMs, and get the feeling that there is a lot of exploration that needs to be done into what can be done well and where that intersects with what is actually fun. Kind of don’t expect EA to be the one to do that.
I don’t think it would be easy to map free form text to game behavior. Not just like “make the NPC smile” but complex behavior like “this NPC will now go to this location and take this action”. That seems like it would be very error prone at best.
How do you think most game scripting engines work?
Nowadays game engines don’t rely on strictly speaking hardcoded behaviour, but rather are themselves just a scripting environment to execute a specific format of code.
Skyrim is still the perfect example because it gives you the ability to literally do anything in the world, via a scripting language.
Instructing NPCs to behave in a specific way is also done through these scripts. And LLMs - especially coding fine-tuned ones which could be tied into the execution chain - can easily translate things like <npcpacesaround> to specific instructions so the NPC walks up and down at a specific distance or in a circle or whatever you want it to do.
You’re seriously over-estimating the work it takes on even crappy, but modern engines to get certain things to happen. Especially when it comes to things that are already dynamically scripted. Like NPCs.
LLM generated code is notoriously bad. Like, “call this function that doesn’t exist” is common. Maybe a more specialized model would do better, but I don’t think it would ever be completely reliable.
But even aside from that, it’s not going to be able to map the free form user input to behavior that isn’t already defined. If there’s nothing written to handle “stand on the table and make a speech”, or “climb over that wall” it’s not going to be able to make the NPC do that even if the player is telling them too.
But maybe you’re more right than I am. I don’t know. I don’t do game development. I find it hard to imagine it won’t frequently run into situations where natural language input demands stuff the engine doesn’t know how to do.
Alright I did read further and damn, you just keep going on being wrong, buddy!
Yes, you can fucking do “stand on the table and make a speech” work. You know how? By breaking it up into detailed steps (pun intended), something that LLMs are awesome at!
For example in this case the LLM could query the position and direction of the table compared to the NPC and do the following:
plan a natural path between the two points (although the game engine most likely already has such a function)
make the NPC follow that path
upon path end, it will instruct the NPC to step onto the table via existing functions (Skyrim pretty much has all these base behaviours already coded, but the scripting engine should also be able to modify the skeleton rig of an NPC directly, which means the LLM can easily write it)
then the script can initiate dialogue too.
I’ve asked Perplexity (not even one of the best coding agents out there, it’s mistake ratio is around 5%), and within seconds it spit out a full on script to identify the nearest table or desk, and start talking. You can take a look here. And while my Papyrus is a bit rusty, it does seem correct on even the third read-through - but that’s the fun part, one does not need trust the AI, as this script can be run through a compiler or even a validator (which let’s be honest is a stripped down compiler first stage) to verify it isn’t faulty, which the LLM can then interact with and iterate over the code based on the compiler feedback (which would point out errors).
now mind you this is the output of an internet-enabled, research oriented LLM that hasn’t been fine-tuned for Papyrus and Skyrim. With some work you could probably get a 0.5B local model that does only natural language to Papyrus translation, combined with a 4B LLM that does the context expansion (aka what you see in the Perplexity feed, my simple request being detailed step by step) and reiteration.
You’d also be surprised just how flexible game engines are. Especially freeroaming, RPG style engines. Devs are usually lazy so they don’t want to hardcore all the behaviours, so they create ways to make it simple for game designers to actually code those behaviours and share between units. For example, both a regular object (say, a chair) and a character type object (such as an NPC) will have a move() function that moves them from A to B, but latter will have extra calls in that function that ensure the humanoid character isn’t just sliding to the new position but taking steps as it moves, turns the right direction and so on. Once all these base behaviours are available, it’s super easy to put them together. This is precisely why we have so many high quality Skyrim mods (or in general for Bethesda games).
And again, code quality in LLMs has come a VERY long way. I’m a software engineer by trade, and I’d say somewhere between 80-90% of all the code I write is actually done by AI. I still oversee it, review what it does, direct it the right way when it does something silly, but those aren’t as minor functionalities as we’re talking here. I’ve had AI code a full on display driver for a microcontroller, with very specific restrictions, in about 4 hours (and I’d argue 2 of that was spent with running the driver and evaluating the result manually then identifying the issue and working out a solution with the LLM). In 4 hours I managed to do what otherwise would’ve taken me about a week.
Now imagine that the same thing only needs to do relatively small tasks, not figure out optimal data caching and updating strategies tied to active information delivery to the user with appropriate transformation into UI state holders.
Yes, you can fucking do “stand on the table and make a speech” work. You know how? By breaking it up into detailed steps (pun intended), something that LLMs are awesome at!
My intended point was the LLM at run time taking user input wouldn’t be able to do “make a speech” if the game engine doesn’t have that concept already encoded. And if the game is presented as “take user input and respond believably” then users are going to ask for stuff the engine can’t do. Maybe there’s no animations for climbing. Maybe they did some shortcuts and the graphics look bizarre when stuff is elevated.
I wasn’t talking about Skyrim specifically.
But also you’re being unpleasant in this exchange, so you can win.
Okay I won’t even read past the first paragraph because you’re so incredibly wrong that it hurts.
First generation LLMs were bad at writing long batches of code, today we’re on the fourth (or by some metric, fifth) generation.
I’ve trained LLM agents on massive codebases that resulted in <0.1% fault ratio on first pass. Besides, tool calling is a thing, but I guess if I started detailing how MCP servers work and how they can be utilised to ensure an LLM agents doesn’t do incorrect calls, you’d come up with another 2-3 year old argument that simply doesn’t have a foot to stand on today.
It’s very obvious in this thread that you have hands on experience and many others do not. 20+ years professional SWE here, a majority of it applied ML/big data/etc. LLMs are really bad at many things but specifically using them as a natural language layer over NPC interactions would be relatively easy and seems like a great use case honestly.
That’s… not what this is about?
The point of integrating AI into games is to provide further diversity within the game.
Think Skyrim. By default you’re limited to 3-4 discussion options, right? Imagine now, if you will, that you could just… type in anything, including emotional markers, and have the characters respond interactively to the statement and tone. No longer are you bound by limited dialogue in RPGs.
visual generative AI will just spice up the visuals - hopefully. Things like repetitive textures and such will disappear as the game generates brand new textures for each grid element. Or create tons of background characters without the need to specify them. The list goes on.
I dream of AI being used to generate cool, theme appropriate NPCs for an indie dev that doesn’t have an army of talented writers at their disposal.
But honestly let’s face it, that’s never what they’ll aim for. It’s all about not paying people, for these assholes. They’ll happily fire their talented writers of they can use AI instead, never once wondering who’s gonna feed the AI in the first place.
And so it is a matter of principle to say no to AI in general, because you know they’ll always pick the worst possible way to use it.
You wish, this will be about money.
You have optimism. It could be used for good like you describe, but most likely will be used by EA to cut artists out, slash costs, make as much profit as possible and suffocate the company until the private equity owners move on to the next company.
No, that’s exactly what this is about. They came right out and said as much. It won’t work, but they’ll cause a lot of damage in the process of failing.
Which is a different article about a (somewhat) unrelated topic.
Using AI for development is already out there, and you can’t put that genie back in the bottle. As an engineer I’m already using it in my daily work for tons of things - I’ve built separate agents to do a number of things:
guess what, AI didn’t replace me, it just allowed me to focus on actually thinking up solutions instead of doing hours of boilerplate stuff.
AI isn’t the enemy in software development. Companies who think they can replace engineers with AI. Middle managers will sooner be on that date, as they were mostly useless anyway.
How do they reduce costs with AI if not by eliminating jobs?
By improving the cadence of projects.
A project costs X amount because of the standard template of pay per time unit Y multiplied by timeframe in time unit Z.
Simply said if you have 100 people working on the project, that costs 100Y per hour. If the project takes 6 months (approx. 960 hours), you multiply the two and get that your costs are 96000Y.
Now the two ways to reduce this is to either reduce the number of employees, with AI you can get rid of maybe 2/3, reducing the expenses to 32000Y…
Or since AI speeds up almost every workflow by about 8 to 10 times, you can keep all the people, but cut down project time from 6 months to about 2 months, which doesn’t just reduce the expenses by the same 2/3 but also increases potential profits for the same 6 month period by 200%, as instead of one product you’re releasing three.
Cutting jobs ain’t the only way to reduce costs with AI.
Citation fucking needed.
My own fucking experience. Which I’ve already explained in detail above.
Your experience counts for jack shit. There is zero evidence that AI is substantially improving efficiency. There is some that suggests its effect is negative.
And your comment also counts for jackshit since you provided no evidence of your claims, not even your own experience.
So you can fuck right off, buddy.
Great for you. You did say “almost every workflow”. How many workflows exist beyond your own lived experience? Do you work on games, do you know all the workflows there? Citation absolutely fucking needed.
I have worked on games, and have a good understanding of the workflows involved.
You’ll obviously still need to do the creative parts manually (and should!) but the majority of the work involving the engine core build and the specific game coding, that can all be sped up borderline exponentially.
But I’m glad that someone with absolutely no understanding of the topic does their best to call out those who do show some experience on the topic just because they don’t get a neatly pre-chewed and pre-digested reply detailing all the information they lack and are unwilling to look it up themselves. As a next step would you like me to cut your steak up and feed it to you byte by byte, or tuck you in at night?
Procedural generation with appropriate constraints and a connected game that stores and recalls what’s been created can do this far better than a repurposed LLM. It’s hard work on the front end but you have a much better idea of what the output will be vs. hoping the LLM “understands” and remembers the context as it goes.
Sorry but procedural generation will never give you the same result as a well tuned small LLM can.
Also there’s no “hoping”, LLM context preservation and dynamic memory can be easily fine-tuned even on micro models.
I agree that the results will be different, and certainly a very narrowly trained LLM for conversation could have some potentials if it has proper guardrails. So either way there’s a lot of prep beforehand to make sure the boundaries are very clear. Which would work better is debatable and depends on the application. I’ve played around with plenty of fine tuned models, and they will get off track contextually with enough data. LLMs and procedural generation have a lot in common, but the latter is far easier to manage predictable outputs because of how the probability is used to create them.
It would be interesting if NPCs could react believably to free form input from the player, but that seems unlikely to be reliable. I’m not sure how you’d map the text to in game actions.
Like, you could have the LLM respond with “ok I’ll let you have my horse if you leave in the morning, and solve the zombie problem” but it’s not trivial to make the game world respond to that. Not even counting it breaking out into “sorry as an LLM I can’t …”
I’ve seen prototypes of RPGs where you could freeform talk to NPCs and I pretty quickly lost enthusiasm for the idea after seeing it in action.
It didn’t feel like a DnD game where you’re maneuvering a social conflict with the DM or other players, it felt more like the social equivalent of jumping up on a table where an NPC couldn’t get to you and stabbing them in the face.
If I wanted to talk to NPCs at length I’d just type into a fucking chat bot. I play games to experience the developers story and vision, not endlessly prompt an npc.
I think your experience will be a common one for players.
Yes it is trivial.
LLM can already do tool calling, emotion metadata output and so on. It would take minimal effort for a well tuned model to also output things like facial expressions, body language, hand and body movements and so on.
The tool calling part might be doable (though I’ve personally struggled to get it working passably with local models), but if the goal is to tell a compelling story or create an interesting experience, especially if doing so in a very open ended way, that isn’t trivial. The only LLM based game I’ve personally played that seemed good was mostly on rails and partially scripted, most of them just aren’t very interesting to play, because the model doesn’t have a good idea where it’s going with anything and is often not very creative, the stuffy personality of the instruct model seems to infect the dialogue and apparent thought process of the characters. For a specific example I’d recommend watching streams of the game Suck Up, which has a genuinely cool concept and solid execution, but you can see people being frustrated running into its limitations as something to interact with creatively.
I’ve tried a couple times to start game projects involving LLMs, and get the feeling that there is a lot of exploration that needs to be done into what can be done well and where that intersects with what is actually fun. Kind of don’t expect EA to be the one to do that.
I don’t think it would be easy to map free form text to game behavior. Not just like “make the NPC smile” but complex behavior like “this NPC will now go to this location and take this action”. That seems like it would be very error prone at best.
How do you think most game scripting engines work?
Nowadays game engines don’t rely on strictly speaking hardcoded behaviour, but rather are themselves just a scripting environment to execute a specific format of code.
Skyrim is still the perfect example because it gives you the ability to literally do anything in the world, via a scripting language.
Instructing NPCs to behave in a specific way is also done through these scripts. And LLMs - especially coding fine-tuned ones which could be tied into the execution chain - can easily translate things like
<npc paces around>to specific instructions so the NPC walks up and down at a specific distance or in a circle or whatever you want it to do.You’re seriously over-estimating the work it takes on even crappy, but modern engines to get certain things to happen. Especially when it comes to things that are already dynamically scripted. Like NPCs.
LLM generated code is notoriously bad. Like, “call this function that doesn’t exist” is common. Maybe a more specialized model would do better, but I don’t think it would ever be completely reliable.
But even aside from that, it’s not going to be able to map the free form user input to behavior that isn’t already defined. If there’s nothing written to handle “stand on the table and make a speech”, or “climb over that wall” it’s not going to be able to make the NPC do that even if the player is telling them too.
But maybe you’re more right than I am. I don’t know. I don’t do game development. I find it hard to imagine it won’t frequently run into situations where natural language input demands stuff the engine doesn’t know how to do.
Alright I did read further and damn, you just keep going on being wrong, buddy!
Yes, you can fucking do “stand on the table and make a speech” work. You know how? By breaking it up into detailed steps (pun intended), something that LLMs are awesome at!
For example in this case the LLM could query the position and direction of the table compared to the NPC and do the following:
I’ve asked Perplexity (not even one of the best coding agents out there, it’s mistake ratio is around 5%), and within seconds it spit out a full on script to identify the nearest table or desk, and start talking. You can take a look here. And while my Papyrus is a bit rusty, it does seem correct on even the third read-through - but that’s the fun part, one does not need trust the AI, as this script can be run through a compiler or even a validator (which let’s be honest is a stripped down compiler first stage) to verify it isn’t faulty, which the LLM can then interact with and iterate over the code based on the compiler feedback (which would point out errors).
now mind you this is the output of an internet-enabled, research oriented LLM that hasn’t been fine-tuned for Papyrus and Skyrim. With some work you could probably get a 0.5B local model that does only natural language to Papyrus translation, combined with a 4B LLM that does the context expansion (aka what you see in the Perplexity feed, my simple request being detailed step by step) and reiteration.
You’d also be surprised just how flexible game engines are. Especially freeroaming, RPG style engines. Devs are usually lazy so they don’t want to hardcore all the behaviours, so they create ways to make it simple for game designers to actually code those behaviours and share between units. For example, both a regular object (say, a chair) and a character type object (such as an NPC) will have a move() function that moves them from A to B, but latter will have extra calls in that function that ensure the humanoid character isn’t just sliding to the new position but taking steps as it moves, turns the right direction and so on. Once all these base behaviours are available, it’s super easy to put them together. This is precisely why we have so many high quality Skyrim mods (or in general for Bethesda games).
And again, code quality in LLMs has come a VERY long way. I’m a software engineer by trade, and I’d say somewhere between 80-90% of all the code I write is actually done by AI. I still oversee it, review what it does, direct it the right way when it does something silly, but those aren’t as minor functionalities as we’re talking here. I’ve had AI code a full on display driver for a microcontroller, with very specific restrictions, in about 4 hours (and I’d argue 2 of that was spent with running the driver and evaluating the result manually then identifying the issue and working out a solution with the LLM). In 4 hours I managed to do what otherwise would’ve taken me about a week.
Now imagine that the same thing only needs to do relatively small tasks, not figure out optimal data caching and updating strategies tied to active information delivery to the user with appropriate transformation into UI state holders.
My intended point was the LLM at run time taking user input wouldn’t be able to do “make a speech” if the game engine doesn’t have that concept already encoded. And if the game is presented as “take user input and respond believably” then users are going to ask for stuff the engine can’t do. Maybe there’s no animations for climbing. Maybe they did some shortcuts and the graphics look bizarre when stuff is elevated.
I wasn’t talking about Skyrim specifically.
But also you’re being unpleasant in this exchange, so you can win.
Okay I won’t even read past the first paragraph because you’re so incredibly wrong that it hurts.
First generation LLMs were bad at writing long batches of code, today we’re on the fourth (or by some metric, fifth) generation.
I’ve trained LLM agents on massive codebases that resulted in <0.1% fault ratio on first pass. Besides, tool calling is a thing, but I guess if I started detailing how MCP servers work and how they can be utilised to ensure an LLM agents doesn’t do incorrect calls, you’d come up with another 2-3 year old argument that simply doesn’t have a foot to stand on today.
lol if you had read the rest of my post you would have seen I admitted you might be right. But go off, I guess.
It’s very obvious in this thread that you have hands on experience and many others do not. 20+ years professional SWE here, a majority of it applied ML/big data/etc. LLMs are really bad at many things but specifically using them as a natural language layer over NPC interactions would be relatively easy and seems like a great use case honestly.