AnalysisHow much will companies spend checking AI agents' work? →

Where things stand / What people use AI for / What AI agents are used for

What AI agents are used for

What AI agents actually do, how many organizations run them and how much they let them do alone, and what published results and failures show.

What agents actually do

Builders most often name customer service and research as their agents' main job, while usage logs show coding as by far the biggest. A consumer browser agent does more personal tasks than work ones.

26.5%

In LangChain's open online survey of professionals in November and December 2025, customer service was the primary use case respondents most often named for their agents (26.5%), with research and data analysis close behind (24.4%).

Who was asked: 1,340 responses to LangChain's public online survey

Opt-in survey; one main use per respondent. Asked which agents they use most each day, respondents named coding assistants first.

Disclosure: LangChain sells tools for building and running agents.

LangChain, State of Agent Engineering · survey fielded Nov 18 to Dec 2, 2025 · Grade C

nearly 50%

Of agent actions (tool calls) sampled from Anthropic's public API in late 2025 and early 2026, nearly 50% were in software engineering; business intelligence, customer service, sales, finance and e-commerce each made up no more than a few percent.

Who was asked: 998,481 random tool calls made through Anthropic's public API

A count of actions, not of agents or customers: coding agents take many steps, so coding weighs more. Domains labeled by Claude.

Disclosure: Anthropic makes Claude; this site is compiled with Claude.

Anthropic, Measuring AI agent autonomy in practice · published February 18, 2026 · Grade B

55%

Of the requests in which Perplexity's Comet browser agent took actions for its users from July to October 2025, 55% were personal, 30% professional and 16% educational.

Who was asked: a random sample of 100,000 agent users; enterprise users excluded

Consumer desktop users in the agent's first months. Contexts labeled by a classifier.

Disclosure: Perplexity makes the Comet browser, and most of the study's authors work there.

Perplexity and Harvard Business School (Yang, Yonack, Zyskowski, Yarats, Ho, Ma), The Adoption and Usage of AI Agents: Early Evidence from Perplexity (arXiv 2512.07828v2) · July 9 to October 22, 2025 · Grade B

19 minutes

In May 2026, Shopify said its internal agent River, which staff call on in public Slack channels, reads code, runs tests, opens pull requests, queries the data warehouse and looks at production traces; its median session lasts 19 minutes and makes 50 tool calls.

Who was asked: River sessions at Shopify, 2026

Disclosure: Shopify's own figures about its own internal agent.

Shopify, Under the River · 2026-05-28 · Grade C

How many run agents, and how much they let them do

Counts of organizations running agents differ because each survey draws its own line, from a pilot to company-wide use. Where use is measured, people keep a hand on most actions.

about two in ten

In McKinsey's survey of May and June 2026, about two in ten respondents said their organizations had reached the scaling phase with AI agents across the organization, and a similar share said so of software coding agents; for chatbots it was 47%.

Who was asked: 1,719 participants at all levels

Respondents at every level describing their own organizations. Scaling across the whole organization is a higher bar than scaling in one function.

Disclosure: McKinsey sells consulting on AI.

McKinsey & Company, The state of AI in 2026: On the road to ROI · fielded May 4 to June 8, 2026 · Grade B

11%

In a July and August 2026 telephone survey of German firms with 20 or more employees, 11% of those that use AI or plan or discuss using it said they already use AI agents that act on their own, 29% plan to and 31% are discussing it; for 24%, agents aren't yet a topic.

Who was asked: 603 German firms with 20 or more employees

Self-reported; each firm judged what counts as an agent.

Disclosure: Bitkom is Germany's digital industry association; its members sell IT and AI.

Bitkom e. V., Erstmals nutzt die Mehrheit der Unternehmen KI · calendar weeks 28 to 33 of 2026 · Grade B

16%

Menlo Ventures' 2025 report, based on a November survey of US enterprise AI decision-makers, found only 16% of enterprise AI deployments qualified as true agents, in which a model plans and executes actions, observes feedback and adapts; most were fixed workflows around a single model call. It put startups at 27%.

Who was asked: 495 US enterprise AI decision-makers

A share of AI deployments at US companies already using AI, not a share of companies.

Disclosure: Menlo Ventures invests in AI startups and in Anthropic, which makes Claude; this site is compiled with Claude.

Menlo Ventures, 2025: The State of Generative AI in the Enterprise · survey November 7 to 25, 2025 · Grade C

63%

In Stack Overflow's opt-in pulse survey of late April 2026, 63% of the developers and other working professionals who answered said they rarely or never let AI agents run entirely on autopilot.

Who was asked: 1,100 developers and working professionals

Opt-in survey. Stack Overflow doesn't say whether people who don't use agents were counted.

Disclosure: Stack Overflow licenses its data to AI companies and sells AI knowledge tools.

Stack Overflow, Agents on a leash: Agentic AI remains mostly single-agent and monitored at work · pulse survey from late April 2026 · Grade C

80%

Of agent actions (tool calls) sampled from Anthropic's public API in late 2025 and early 2026, 80% came from agents that appeared to have at least one safeguard, such as restricted permissions or human approval; 73% appeared to have a human in the loop, and only 0.8% appeared irreversible, such as sending an email to a customer.

Who was asked: 998,481 random tool calls made through Anthropic's public API

Labeled by Claude from single actions. Anthropic says Claude overestimated human involvement, so the safeguard and human shares are upper bounds.

Disclosure: Anthropic makes Claude; this site is compiled with Claude.

Anthropic, Measuring AI agent autonomy in practice · published February 18, 2026 · Grade B

What agents have delivered

Companies report agents doing real volumes of work, from merged code to customer chats. A large randomized trial in customer service found agent-handled chats shorter but rated lower by customers.

roughly 24%

Studying tens of thousands of Microsoft engineers during the company's early-2026 rollout of Claude Code and GitHub Copilot CLI, Microsoft researchers estimate that engineers who adopted the agents merged roughly 24% more pull requests than they otherwise would have.

Who was asked: Tens of thousands of Microsoft engineers

Observational, not randomized. Merged pull requests stand in for output, not its value.

Disclosure: The authors work at Microsoft, which makes GitHub Copilot CLI. Anthropic makes Claude Code; this site is compiled with Claude.

Microsoft (Murphy-Hill, Butler and Savelieva), arXiv, Adoption and Impact of Command-Line AI Coding Agents: A Study of Microsoft's Early 2026 Rollout of Claude Code and GitHub Copilot CLI · January to April 2026 · Grade B

about one in eight

In May 2026, Shopify chief executive Tobi Lütke told staff that about one in eight pull requests merged into Shopify's codebase the week before had been authored by its agent River and reviewed by people; River opened 1,870 pull requests in the main monorepo that week.

Who was asked: Shopify's codebase

A count of pull requests, not of code or its value.

Disclosure: Shopify's own count, published on its careers site.

Shopify, Learning on the Shop floor (Tobi Lutke's internal memo, reproduced by Shopify) · one week in early May 2026 · Grade C

two-thirds

In February 2024, Klarna said its AI assistant had 2.3 million conversations in its first month live, two-thirds of its customer service chats, doing the equivalent work of 700 full-time human agents.

Who was asked: Klarna's customer service chats in the AI assistant's first month

Klarna later said it was recruiting human agents again; see What failed, below.

Disclosure: Klarna's own figures about its own assistant.

Klarna, Klarna AI assistant handles two-thirds of customer service chats in its first month · 2024-02-27 · Grade C

16.8%

In a randomized trial on Alibaba's Taobao in August 2024, chats about standard issues that an agentic AI handled, with customer-service workers supervising and stepping in when needed, took 16.8% less time than the same kind of chats handled by workers alone.

Who was asked: 647 Taobao customer-service workers (345 control, 302 treated) and 680,676 chats

These were standard issues, a small share of all chats; across all chats the time saved was much smaller.

Disclosure: An Alibaba employee co-wrote the study, and the agent is Alibaba's own.

Wang, Zhu, Feng, Lu and Jia (Zhejiang, Fudan, Dartmouth, Alibaba), arXiv, Agentic AI and Human-in-the-Loop Interventions: Field Experimental Evidence from Alibaba's Customer Service Operations · August 2024 · Grade A

0.412 points

In a randomized trial on Alibaba's Taobao in August 2024, customers rated chats about standard issues that an agentic AI handled, with workers supervising, 0.412 points lower on a five-point scale than the same kind of chats handled by workers alone.

Who was asked: 647 Taobao customer-service workers (345 control, 302 treated) and 680,676 chats

Repeat contacts showed no significant change: issues still got resolved, but customers liked the service less.

Disclosure: An Alibaba employee co-wrote the study, and the agent is Alibaba's own.

What failed, and why

Many large organizations have scaled back, delayed or paused agent rollouts because the expected costs outweighed the value, and the best-known replacement of support staff was partly reversed. In head-to-head tests agents succeed less often than people, and chats handed to a person after an agent struggled ended worse.

49%

Only 12% of large organizations consistently weigh AI value against cost; 49% have scaled back, delayed or paused AI-agent deployments because expected costs outweighed value.

KPMG, Global AI Pulse Q2 2026 and Q3 2026 · Q2 2026 (fielded Apr 28 to May 25, 2026; dated June 2026) and Q3 2026 (Jul 23 to Aug 26, 2026) · Grade C

In May 2025, Klarna, which had said AI let its headcount fall by over a thousand, was recruiting human customer-service agents again so customers could always reach a person; its chief executive told Bloomberg that weighing cost too heavily had led to lower quality, Fortune reported.

Who was asked: Klarna, as its CEO described it to Bloomberg and a spokesperson to Fortune

A spokesperson said Klarna was still AI-first and was hiring freelance support agents.

Cited secondary · Klarna (via Fortune), Klarna plans to hire humans again, as new landmark survey reveals most AI projects fail to deliver · May 2025 · Grade C

32.5–49.5%

In a 2025 lab study by Carnegie Mellon and Stanford researchers, AI agents given the same work tasks as human workers had success rates 32.5–49.5% lower, and they often faked progress, for example by fabricating data.

Who was asked: 48 human workers and 4 AI agent frameworks on 16 long work tasks

A small lab test of agents working alone. The same agents finished far faster and more cheaply than people.

Carnegie Mellon University and Stanford University (arXiv), How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations (Wang, Shao, Shaikh, Fried, Neubig and Yang) · 2025-11-06 · Grade B

0.928 points

In Alibaba's August 2024 customer-service trial on Taobao, when software flagged a frustrated customer and a worker took the chat over from the agentic AI, it ended worse than similar chats handled by workers from the start: repeat contacts rose 6 percentage points and ratings fell 0.928 points on a five-point scale.

Who was asked: 647 Taobao customer-service workers (345 control, 302 treated) and 680,676 chats

A comparison with matched chats, not random assignment. Workers who stepped in early on their own fared better.

Disclosure: An Alibaba employee co-wrote the study, and the agent is Alibaba's own.

over 40%

Gartner expects over 40% of agentic AI projects to be canceled by the end of 2027, because of rising costs, unclear value or weak risk controls.

A prediction or a claim without a published method, not a measurement.

Cited secondary · Caution · Gartner press release, June 25, 2025 · Published June 25, 2025 · Grade C

Passing the steps is not finishing the job

Two benches score the finished job, not the steps along the way. One asks an agent to answer from a project's chats, documents and code. The other asks it to build a product through a long series of feature requests. A third asks whether a model can tell, before any attempt, which paid jobs an agent will finish. On the first two, a high score on the pieces leaves many jobs unfinished. On the third, the ranking holds when the agent's limits are described and falls apart when they are not.

65%

On Replit's ViBench 2.0, the three strongest of nine models pass up to 98% of individual test plans and deliver a fully working product in at most 65% of builds.

Who was asked: Nine models on ViBench 2.0, 17 full-stack web products grown over 241 feature requests

Replit built this bench to choose models for its own platform. A build counts only if every test plan passes. No open-weight model delivered a fully working product.

Disclosure: Replit describes a benchmark it uses to select models for its own product.

Furman, Kirani, Sheikh, Li, Zhong and Catasta, ViBench 2.0 (Replit) · November 3, 2026 · Grade B

80%

Across 33 models and agent setups on SlackBench, the best scored around 80% and most scored below 66%. The questions are 140 queries over privacy-preserving reenactments of academic research projects, with the answer spread across chats, documents and a repository that changes.

Who was asked: 33 models and agent setups on 140 SlackBench queries

These are reenactments of academic projects, not a company's live chat.

Li, Mao, Zheng, James, Li, Anderson, Gonzalez and Dimakis, SlackBench · November 3, 2026 · Grade B

0.86

On PAYWORTHY, 18 models were asked whether an agent would succeed at 1,494 real freelance jobs. Told what the agent cannot do, every model ranked the accepted jobs above the rejected ones, with an AUC from 0.66 to 0.86. Told it was a generic agent with a browser and code execution, the same models scored from 0.45 to 0.56, about a random ordering.

Who was asked: 18 models forecasting an agent's success on 1,494 real freelance jobs

Success is one agent's single attempt, accepted or rejected by a model acting as the client. Describing the agent's limits is what made the ranking work.

Disclosure: The reproduction code is published by Upwork. The postings are from a freelancing platform.

Rezaei, Wang, Levert, Fejzo, Rabinovich and Chen, PAYWORTHY · November 3, 2026 · Grade B

Keep exploring

The AI at Work Benchmark

Measure how your people actually use AI

Score your organization's practices, measure your people with an anonymous pulse, and see both against the published evidence.

Employee pulse

Fifteen questions on how you use AI, three about your role. Your answers are added to your organization's totals and never shown on their own. About six minutes, anonymous.

Coming soon

Organization survey

For the person who leads AI adoption: profile, practices and open questions. Your staff answer the pulse under the same code, and your report shows both.

Coming soon