If you look at efficacy though on academic tests or asking it some fact question and you compare that to asking a random person instead of always getting the ‘right’ answer, which we expect computers/calculators to do, would LLMs be comparable or better? Surely someone has some data on that.

E: It looks like in certain domains at least LLMs beat out human counterparts. https://stanfordmimi.github.io/clin-summ/