Researchers at MIT have developed a new method that helps large language models (LLMs) adjust their “thinking effort” based on how hard a question is. Rather than giving the model the same fixed computational budget for every task, the system dynamically allocates more resources when the problem is complex, and conserves resources when it’s simple, tells MIT News.
Traditional approaches often either waste computation on easy questions or hit resource limits on hard ones. MIT’s technique uses a “process reward model” (PRM) to inspect partial answers as they form, estimate which reasoning paths are most promising, and then expand those paths. That way, the model pursues deeper reasoning only when it seems likely to help.
In tests, this adaptive approach cut computational effort roughly in half compared with standard methods, while maintaining comparable accuracy across a range of problems from easy to very hard. What’s more, smaller and less resource-intensive LLMs sometimes matched or outperformed larger models when using this adaptive reasoning strategy.
This matters for both performance and sustainability. By tailoring compute use per problem, the method could cut the energy cost of inference, a growing concern as generative AI scales, while keeping reasoning quality high. It also opens the door for deploying efficient, capable LLMs in settings that require reliability, speed, or lower compute budgets.
Instead of treating every query the same, LLMs can now “judge” how much thinking they need to do, similar to how humans might mull over a hard puzzle but answer a simple question quickly. This shift could make AI assistants smarter, faster, and cheaper to run.