Bad news, star employees: You're not the ones who'll benefit the most from AI
Getty Images
- Consultants using AI completed tasks faster and produced higher-quality results than those without, according to a new study.
- The greatest gains were seen by below-average performers using AI, per the study's authors.
- A previous study showed that AI assistance had a weak impact on higher-skilled customer service workers.
A recent study conducted at management consulting firm Boston Consulting Group, or BCG, found that consultants who ranked below average benefitted the most from using the technology.
Conversely, those who did better saw fewer gains.
The study, published Thursday, involved 758 BCG consultants globally — or about 7% of the company's individual contributor-level consultants. They were divided into two groups: one with access to GPT-4, the model that currently powers ChatGPT, and another group without.
Before the experiment, the staffers were tested to gauge their average level of performance, classifying them into "bottom-half" and "top-half" skilled participants.
They were then assigned a series of practical consulting tasks for a fictional shoe company and had their performance graded by human and AI raters.
The greatest gains were seen by below-average performers using AI, whose average performance improved by 43%.
Their above-average counterparts only saw an average performance increase of 17% from using AI.
The study also found that those who used AI completed their tasks faster and produced higher-quality results than those without access.
However, it also underscored the risks of over-reliance on AI.
Consultants with access to AI performed up to 20% worse when presented with tasks that were beyond the AI's grasp. In these cases, the AI would present misleading yet plausible responses.
The study was conducted by researchers from Harvard, MIT, the University of Warwick, and the University of Pennsylvania, along with the Boston Consulting Group.
Ethan Mollick, a Wharton professor and a co-author of the study, wrote in a post on X: "AI worked like a leveler: lower performers had the biggest gains."
—Ethan Mollick (@emollick) September 16, 2023
Another study — published in April — examined customer service workers at a Fortune 500 company and found that highly-skilled workers saw a "close to 0%" productivity boost with an AI assistant.
In contrast, lower-skilled agents saw a productivity boost of 35%.
To be sure, generative AI, as it currently exists, remains prone to convincingly presenting wrong statements as fact. The fallout of these errors can be severe — ranging from butchered obituaries to worse health outcomes for patients.
The study's authors and BCG did not immediately respond to requests for comment from Insider, sent outside regular business hours.