Ok, you had a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
Ok, you had a moderately complex math problem you needed to solve. You gave the problem to 6 LLMS all paid versions. All 6 get the same numbers. Would you trust the answer?
No, because they don’t do math. If the LLM calls a script to do the math and just formats the input it might get accurate results consistently… but you just invented a machine to press calculator buttons for you at that point which is hilariously energy inefficient. This is unacceptable from a cost and reliability standpoint. If you’re familiar with enterprise reliability metrics you’d weep at the thought of a multistage process where each step had a single 9 and no visibility to underlying model tuning that can change outputs in wildly unexpected ways.