Anthropic's new AI model finished 41.3 percent of real freelance jobs to a paying client's standard, up from 2.5 percent a year ago. Here is what that means.
A new AI model from Anthropic called Fable 5.2 can now finish about four in ten real freelance jobs well enough that a paying client would accept the work. It scored 41.3 percent on the Remote Labor Index, a test built by Scale Labs and the Center for AI Safety. A year ago, the best system managed 2.5 percent.
The test measures how often AI agents, which are programs that carry out tasks on their own, complete whole jobs rather than single skills. When it launched about a year ago, the top system, Manus, finished 2.5 percent. Fable 5.2 now completes more than 16 times as many.
The jump is steep even against recent results. Fable 5, an older model in the same family, led in July at about 16 percent. Fable 5.1 sits just under 18 percent.
The test uses 240 completed projects from the freelance platform Upwork, done by 358 verified professionals. Together the jobs are worth about $144,000. The average project costs $632.60 and takes a human freelancer just under 29 hours. Jobs range from video editing to CAD design, game development and data analysis.
The AI agents work in a virtual workspace with common software such as Blender, GIMP and Audacity, plus AI tools for images, audio and video. Trained human experts then open the files and compare the result with what the human freelancer delivered. Researchers tried using another AI to do this checking. It failed, because AI reviewers are fooled by work that looks polished but is flawed.
Scale Labs says Fable 5.2 gained most in editing audio and music, designing architecture and interior spaces, and making 3D models. It also handles long, complex projects that earlier top models often failed. Nearly 50 of its successes were jobs no AI agent in the test had solved before.
It still struggles with industrial product design and animation. When agents fail, the researchers say they usually hand in amateur-quality work, unfinished results, broken files or contradictory designs.
If you hire freelancers for editing, design or data work, some of that work may now be within reach of AI. Still, the best system fails on roughly six in ten jobs in this test. Results need a careful check by someone who knows the work, because flawed files can look fine at a glance.
AI can now finish some real freelance work to a client's standard, but it still fails most jobs in this test, so check what it delivers.
A leaked EU draft would make AI shopping helpers ask for fresh approval when they are paid to push a deal. Shoppers and sellers could see more prompts.
Read more: EU draft law would make AI shopping helpers ask before some purchasesAnthropic says it signed about $10 billion more in cloud commitments with Microsoft and Google, which shows how costly AI is to run.
Read more: Anthropic adds about $10 billion in cloud deals with Microsoft and GoogleSalesforce says its AI sales agents give reps back about six hours a week, but the numbers come from a survey of only 24 of its own customers.
Read more: Salesforce survey: AI sales helpers save reps about six hours a week
