Smaller, cheaper AI can do most jobs. Do employers need LLMs?

Running smaller models on local devices instead of in the cloud could cut the cost of generating AI output by roughly 80%, Nicholson said, but almost no large organization is set up to make that kind of granular, task-by-task decision yet.

Nicholson pointed to how AI providers already ask customers to make that kind of call, without giving them much to go on.

“Anthropic today gives you a drop down menu and sort of says, okay, well for most everyday tasks, select this one, for more complex tasks, select this one. And it’s like, well, gosh, I don’t know. I wanted to write a draft of a blog post. This one’s going to cost you 4 cents and this one’s going to cost you 39 cents. We don’t have a good way of conceptualizing that yet,” he said.

Most everyday tasks, he argued, don’t call for the most powerful AI model on the market.

“If all I’m doing is walking down to the corner store, do I need to drive the minivan and burn all that gas? And the answer is no, you don’t,” Nicholson said.