Stanford HAI’s annual report highlights rapid adoption and growing accessibility of powerful AI systems

Researchers from Stanford University early today published the latest edition of their annual AI Index Report, detailing the growing influence of artificial intelligence technologies on our society and the global economy.

The Stanford Institute for Human-Centered Artificial Intelligence, known as Stanford HAI, has been publishing its annual reports on the state of the AI industry since 2017. This year’s edition, the eighth, is the “most comprehensive” report to date, spanning more than 430 pages. The authors say it’s arriving at a critical juncture as the influence of AI across society rapidly accelerates with the emergence of increasingly capable and sophisticated AI systems.

The report contains an in-depth analysis of the latest new AI models and development techniques, along with a detailed look at the evolving landscape of AI hardware, new estimates on the cost of AI inference, and studies on some of the hottest new industry trends. It also looks at the expanding role of AI in areas such as science and medicine, and the growing emphasis on responsible AI practices.

Advances made, but challenges persist

According to Stanford HAI’s analysis, AI developers continued to make great strides in overall performance over the last year, with record-breaking scores on key benchmarks including the MMLU, GPQA and SWE-bench, which were established in 2023 to test the limits of advanced AI systems.

Beyond these achievements, the last year also saw significant advances in areas such as AI-generated video and AI agents, which are autonomous systems that can perform tasks on behalf of people with minimal supervision.

The report says that the U.S. is still the nation to beat in terms of top-performing models, but worryingly for that country, China is closing the gap fast. In 2024, U.S.-based organizations published 40 “notable AI models” compared to just 15 from Chinese firms and three from Europe. However, those Chinese models have made some impressive strides, achieving close to parity with their U.S. counterparts on key benchmarks such as MMLU and HumanEval

One reason for China gaining ground is that AI systems are becoming increasingly efficient, meaning they’re more affordable to develop and therefore more accessible. The report found that the inference cost for a system matching the performance of OpenAI’s GPT-3.5 has fallen by 280-times over the last two years, while in terms of hardware, costs declined 30% in the last 12 months.

AI models are also consuming 40% less energy than a year ago, while the emergence of more powerful “open-weights” models that can be easily customized for different use cases is also lowering the barrier to entry. That may explain why regions such as the Middle East, Southeast Asia and Latin America were able to launch their first powerful LLMs in the last year.

However, the report notes that even the top AI developers are still struggling to match the capabilities of real humans in terms of AI’s reasoning skills. While learning-based systems that generate and verify hypotheses are performing well on tasks like the International Math Olympiad, such systems still struggle with “logic-heavy tasks” such as arithmetic and planning, which limits their adoption.

Growing investment and enterprise adoption

The advances in AI continue to be fueled by billions of dollars in investment, and the lion’s share of that cash comes from America. In the U.S., private investors threw more than $109 billion at AI startups and projects, almost 12 times the $9.3 billion invested in China, and 24 times as much as the U.K.’s $4.5 billion. Within the AI industry, generative AI continues to attract the most cash at $33.9 billion in private investment globally, up 18% from 2023.

Investors continue to pour billions of dollars into AI because the technology is seeing rapid adoption by businesses. According to the report, 78% of global enterprises confirmed they have deployed AI systems in their workflows in 2024, up from 55% one year earlier. This is being driven by what Stanford HAI says is a “growing body of research” that shows AI is boosting productivity in key industries while helping to narrow skills gaps across the workforce.

The report also noted the impact of AI on science, noting that Nobel Prizes were awarded to researchers that utilized deep learning systems to advance physics, and to a second team that applied AI to protein folding.

Increased societal impact

AI increasingly permeates our lives in areas ranging from healthcare to transportation. In the U.S., for instance, the Federal Drug Administration has now approved more than 950 AI-powered medical devices, up from just six in 2015 and 221 in 2023. AI-powered self-driving cars, such as Waymo’s autonomous taxis, are also a thing, providing 150,000 rides per week.

However, the U.S. public is still somewhat opposed to having too much AI in its lives. According to one study, just 39% of Americans said they see AI as more beneficial than harmful. That compares with places such as China, Thailand and Indonesia, where more than 75% of people say they welcome the advantages of AI.

Globally, there remains a significant gap in terms of education. While two-thirds of the world’s nations now offer, or plan to offer computer science education to K-12 students, access remains limited in areas like Africa because of a lack of basic infrastructure such as electricity. Moreover, there are still challenges in terms of teaching basic AI skills In the U.S., 81% of teachers say they believe AI should be a foundational element of computer science education, but less than half say they’re able to teach it.

Governments lead the way in responsible AI

The report highlights a worrying rise in the number of AI-related security incidents over the last year, and notes that despite these, standardized RAI evaluations are rare even among the biggest AI developers, such as OpenAI, Google LLC and Meta Platforms Inc.

On the other hand, the emergence of new benchmarks such as AIR-Bench, HELM Safety and FACTS is a promising development as Stanford HAI believes they can be useful tools for the industry to assess the safety and accuracy of their models.

There is, however, still an alarming gap in the business world in terms of those that recognize AI’s risks and those who take action to try to mitigate those dangers. That’s in contrast to most governments, which are showing “increased urgency” with regard to responsible AI. For instance, last year global organizations such as the European Union, African Union and the United Nations all published frameworks focused on AI transparency, trustworthiness and other key principles of responsible AI.

Looking at the U.S., federal agencies issued 59 AI-based regulations in 2024, more than twice as many as they did in 2023. On a global scale, AI-based legislative mentions increased by an average of 21% across 75 countries.

Image: SiliconANGLE/Meta AI