OpenAI opens a much faster, cheaper AI for quick yes-or-no choices

OpenAI has made its Decisions API available to everyone. It is a fast, cheap AI tool built to pick between options, such as sorting a request into the right category. It costs $0.10 per million input tokens, with no charge for the answers it sends back. Tokens are the small chunks of text an AI reads. The release came Thursday, after an announcement at OpenAI's developer day on September 29.

What it does

Decision models act like super sorters. You give them a set of options, and they quickly choose one and rate how confident they are. Developers found these models can replace larger chatbot-style AI in many workflows, running faster and at much lower cost. OpenAI did not build a new model for this. It took ChatGPT Luna and reconfigured it to run faster and cost less.

What changed

The big gain is speed. OpenAI's Nikunj Handa said responses shrank from one to three seconds down to about 100 to 180 milliseconds. The public version also adds voice and visual abilities. OpenAI says it also worked to cut delays in regions around the world by placing its computing resources closer to users.

• Price: $0.10 per million input tokens, no output-token charges
• Speed: about 100 to 180 milliseconds per response
• New: voice and visual modalities
• Early testers: Box, Datadog, Ramp and Speak

Early results

Speak, which builds AI language tutors, cut the time between actions from 1.4 seconds to 0.13 seconds, according to its CTO, Andrew Hsu. OpenAI also used the product to make its Dots voice assistant respond faster. Handa said OpenAI's model may stand out on images, where he said Luna is showing better calibration, meaning its confidence ratings line up better with how often it is right.

A crowded race

OpenAI is following TypeSafe, a startup founded by a former OpenAI researcher. Its Jev model created this category, and TypeSafe is now valued at $7.6 billion. On two public rankings, the latest Jev model placed seventh and third. OpenAI's model placed eighth and fourteenth. Other models, including Liquid's D1 Ultra and Base Lab's Elephant-1, tend to rank higher. Jev is still the most widely used on OpenRouter.

Nobody has a clear answer yet on what makes a good decision model. Speed, cost, accuracy and confidence ratings matter differently for each job. TypeSafe's CEO, Diogo Almeida, urges people to test models on their own work rather than trust rankings.

What happens next

Handa said OpenAI may train its models specifically for this kind of work later. He also floated a goal of about 60 milliseconds, which could allow a check on every frame of a video. For now, any business that sorts or routes lots of requests can try the tool and compare it with what it uses today.