Researchers want to control AI agents by designing rules they can't game

Researchers are exploring a new way to keep powerful AI models in check. Instead of only telling an AI what to do, they want to design the rules so that following them is the best outcome for the AI itself. The approach is called "reverse game theory," and MIT Technology Review covered it this week.

How it works

Game theory is the study of how rational players behave under a given set of rules. Reverse game theory flips that around. It starts with the behavior people want, then designs rules that reward it.

The classic example is cutting a cake. If one person cuts and the other chooses a slice first, the cutter ends up with the piece left over. So the cutter has a reason to make the slices equal. Nobody has to watch them.

The idea is to apply that same logic to AI agents, which are AI tools that take actions on your behalf. Researchers believe it could align AI more closely with human rules and instructions.

What is not known yet

The big question is whether this works outside the lab. The researchers themselves say major questions remain about whether these mechanisms will hold up in practice. The report does not say which AI systems have been tested or what the results were.

It matters because AI tools are starting to handle tasks tied to your money, your data and your business. How reliably they follow your rules will matter more as they take on more of that work.