The Next Great Leap in AI Is Behind Schedule and Crazy Expensive

OpenAI's ambitious project, GPT-5, code-named Orion, is encountering significant delays and escalating costs, casting uncertainty over its completion. The model, intended to surpass GPT-4, has been in development for over 18 months, with training runs costing up to half a billion dollars, yet it has not achieved the anticipated advancements.

The Wall Street Journal

A major challenge is the scarcity of high-quality data necessary for training such advanced models. OpenAI has resorted to creating data from scratch and utilizing synthetic data, but these measures have not yet produced the desired results. This situation reflects a broader industry concern that AI progress may be plateauing due to data limitations.

The Wall Street Journal

In response, OpenAI has introduced a new AI model, o3, which emphasizes reasoning to enhance output accuracy. Unlike previous models, o3 deconstructs problems into manageable parts and self-corrects, reducing AI "hallucinations." However, this approach significantly increases processing time and costs, with complex tasks potentially costing up to $20 each. Microsoft, a major partner of OpenAI, is closely monitoring these developments, especially considering the substantial financial implications.

Barron's

The challenges faced by OpenAI highlight the increasing costs and complexities associated with developing frontier AI models. A study from earlier this year indicated that the expenses for training such models have been doubling approximately every nine months since 2016, with projections suggesting that the most extensive training runs could exceed a billion dollars by 2027.

ArXiv

These developments underscore the formidable obstacles in advancing AI technology, including data scarcity, escalating costs, and the need for innovative approaches to achieve significant progress.