A new Medium series explains why a fraud-spotting AI can score 98% on accuracy and still catch none of the fraud. Here is what to ask before you trust one.
A Medium article, Part 1 of a series called "AI Metrics for Mere Mortals," makes a point worth knowing before you buy any AI tool. A model can be 98% accurate and still catch no fraud at all.
Accuracy counts every right answer. Fraud is rare, so most cases are honest ones. A model that flags nothing gets all of those right and still posts a high score. As a simple illustration, picture 100 payments where 2 are fraud. A tool that flags none is right 98 times and useless at its job.
The easy answers pile up the score. They never mattered in the first place.
Anyone who is told a tool is "98% accurate" can use these questions. That includes business owners picking fraud software and people who rely on AI to protect their money.
The full article text was not available, so the better measures the author recommends are not named here. The material also does not say how many parts the series will have or when the next one appears. The series is described as a guide to judging AI tools with better measures.
Before trusting an AI tool, ask what it actually catches, not just how accurate it says it is.
Fake PayPal emails and texts, some posing as a security report, try to get you to tap a link and type your login. Checking your account directly avoids the trap.
Read more: Got a fake PayPal message? Skip its link and check your account yourselfVisa tested 22 AI models on realistic shopping tasks. They passed just over half, and often bought the wrong thing anyway.
Read more: Visa tested 22 AI shoppers, and most still make costly mistakesDARPA wants AI to hunt for design flaws in stock, bond, energy and cloud markets, so enemies cannot use them to cause economic damage that reaches households.
Read more: Pentagon wants AI to find weak spots in stock and energy markets
