Artificial Intelligence (AI) systems are now just as good as or can even surpass humans in certain tasks, such as English language comprehension, visual reasoning, and image classification.

Stanford University recently released its latest edition of its AI Index Report, an annual publication that has been tracking the progress of AI in various domains since 2017. This year, the report spans 400 pages and contains some interesting takeaways. One of these is the fact that while AI is beating humans on basic tasks it is still lagging on more complicated tasks, such as competition-level mathematics.

The report also highlighted that in 2023, the industry and academia developed 66 notable machine learning models overall, along with 21 models that resulted from industry-academia collaborations. Sixty-one of these models originated from the US, according to the report, while the European Union and China were responsible for 21 and 15 models, respectively. The report also cited rising costs for training the models but also skyrocketing investment in generative AI.

With the rapid advancement of AI, the report claims that many existing benchmarks that are used for evaluating responsible AI have become obsolete. AI Index editor-in-chief and social scientist Nestor Maslej shared to science journal Nature his observation that benchmarks that once served the community for five to ten years now often become obsolete in just a few years. Per the report, there is also a lack of standardized benchmarks, thereby making it difficult to systematically assess the risks and limits of leading AI models.

Worryingly, the AI Index report also revealed that the number of reported misuse of AI (e.g. wrongful arrests because of negligent use of facial recognition, AI-generated nude images of celebrities, and road accidents caused by erring autonomous cars) are rising every year. Thankfully, the report notes a sharp increase in government regulations over AI.

As for what the public thinks, the report also cited a survey from Ipsos in which respondents recognized the dramatic effect AI would bring to their lives in the next couple of years, with more than half of respondents expressing concern and nervousness instead of excitement.

You can read the full report at Stanford’s website.

Leave a comment

Your email address will not be published. Required fields are marked *