Business
New Study Questions Return on Investment for Companies Using Generative AI
A recent MIT study found that 95% of firms that adopt generative AI have seen no return on their investment.
While the study has its fair share of critics, it has raised alarms when it comes to the business benefits around a multi-billion-dollar advancement in the tech industry.
Amazon recently eliminated 14,000 roles in a company-wide reorganization effort in which AI was listed as one of the principal reasons. The company faced scrutiny as many debate whether the integration of AI into a company’s workflow can really do the job of actual employees.
The MIT study suggests that generative AI has not yet reached the capability needed to improve a company overall, let alone replace jobs.
“Research found limited layoffs from GenAI, and only in industries that are already affected significantly by AI,” the study found. “There is no consensus among executives as to hiring levels over the next 3-5 years.”
Although adoption of the technology is high, “transformation is rare. Only 5% of enterprises have AI tools integrated in workflows at scale and 7 of 9 sectors show no real structural change.”
Joel Shapiro, professor of data analytics at Northwestern University, remains skeptical of the study, saying that measuring return on investment for AI integration is difficult.
“If we are talking about huge transformations of the way a company works, ROI is very hard to measure; it’s very hard to capture and it’s very hard to, you know, quantify,” Shapiro said.
For companies that are looking to integrate generative AI into their workflow as a mode of survival amid competition, he believes, “there’s no good way to measure ROI around survival.”
Shapiro added that companies often use AI in a narrow way. For example, as a tool to review and manage contracts, or in the grocery industry to manage their items better. For those smaller uses, it’s easier to measure ROI because there are more specific goals they are trying to achieve, he said.
Shapiro is also skeptical about the study’s focus on only generative AI technology, which is the use of large language models to produce original content. That does not include the totality of AI tools that have already been implemented in different sectors.
Steven Keith Platt, director of analytics at the Lab for Applied AI at Loyola University’s Quinlan School of Business, echoed Shapiro’s skepticism around what an AI bubble might look like.
“There’s a lot of inflation in stock prices, right, Oracle, Nvidia and those prices are pretty elevated, but it’s a lot different when the dot-com bubble happened,” Platt said. “The biggest difference is there’s a ton of adoption of AI right now and a lot of companies are clamoring and asking for help on implementation.”
He notes that the companies involved in AI now, as opposed to the dot-com bubble, are significantly larger, which will mitigate the harm on the market if there is a potential crash.
Platt added that companies have already had AI integration into their workflows for years, before the arrival of generative AI, that have proven to be successful.
Ben Zhao, professor of computer science at the University of Chicago, believes AI in tech has gotten to a point where it’s almost reaching widespread adoption; however, people are realizing the limitations of what it can actually do.
“That coupled with a tremendous amount of actual capital investment in the market and, you know, in financial markets that’s where you have potential risk for a real bubble burst.” Zhao said.
With the emerging development of AI, many have critiqued the businesses that are developing AI training models with copyrighted material. There are also ethical concerns regarding misinformation and the environmental toll it takes to produce enough computing power for AI computing at scale.
Zhao has testified in favor of guardrails on AI development, so protections can be made that ensure intellectual property.
“Companies feel like they must embrace it full on or else they’re going to miss out,” Zhao said. “...That type of pressure has led not just AI developers, but also the application developers and society at large to really think about this as a must. And along the way we sort of pushed aside issues like copyright, consent, licensing, we’ve thrown away issues of right of publicity for personal images.”
