AI analysis tools still get most business advice wrong, survey finds

AI tools that study company data and suggest what to do are getting better, but they still miss more often than they hit. A new report finds that 48% of insights from these tools led to actions that improved the business. A year ago, respondents put that figure at 37%. So progress is real, but more than half still fail to help.

What the report found

The report comes from Insights, the custom content arm of MIT Technology Review, not its editorial staff. It looks at analytics agents. These are AI programs that study business data, produce findings and, in some cases, take actions based on them. Most respondents praise the quality of descriptive insights, which explain what happened. Reviews of prescriptive insights, which recommend what to do, are mixed.

Why the advice goes wrong

The executives surveyed named three main problems:

• Missing business context was the top cause of weak insights, cited by 55%.
• Fragmented or inaccessible data was cited by 45%.
• At 52% of companies, agents produce too many insights to review and act on.

Missing context means the tool does not fully understand how a company works or what its numbers mean. Flawed insights that trigger wrong actions can cause reputational or financial damage, the report warns.

Too much to check

Only 24% of organizations can evaluate every insight their agents produce. Most review them case by case, and that work is currently done mostly by people. More output does not help if no one has time to judge it.

What the best performers do

A small group the report calls insight leaders sees more than 70% of agent insights lead to improvements. Fewer of them say their agents struggle to reach context. Across all companies, the top priorities are stronger AI and data governance (60%) and adding a semantic layer (45%). A semantic layer is a shared set of definitions that tells software what business terms mean. The experts interviewed called it crucial.

What to do

If you let an AI tool suggest or take actions, check its advice before acting on it. Make sure it knows how your business defines its key numbers. The report points to better context and clear rules, rather than bigger AI models, as the way to improve results.