Tesla CEO Elon Musk has announced that artificial intelligence (AI) systems have exhausted all available human data, including books, videos, and internet content, for training purposes. Musk made this statement during a livestream conversation with Stagwell chairman Mark Penn, which was streamed on X (formerly Twitter).
“All human data has been used up. We’ve literally run out of the entire internet, all books ever written, and all interesting videos,” Musk stated, adding that this milestone was reached in 2024.
Shift to Synthetic Data
According to Musk, the lack of human-generated data has forced AI developers to turn to synthetic data—information created by AI itself.
“AI is advancing on the hardware front and on the software front. It’s now moving to synthetic data because we’ve exhausted the cumulative sum of human knowledge for AI training. The only way to supplement that is with synthetic data, which AI creates,” Musk explained.
Musk detailed how synthetic data enables AI to self-learn.
“AI will write an essay or come up with a thesis, grade itself, and then go through this process of self-learning with synthetic data,” he said.
Challenges of Synthetic Data
However, Musk acknowledged significant challenges in using synthetic data, particularly in determining its accuracy.
“This is always challenging because how do you know the answer is hallucinated or real? It’s difficult to find the ground truth,” he said.
Some researchers have raised additional concerns about synthetic data. They warn that overreliance on it could lead to “model collapse,” where AI systems become less creative and more biased, eventually compromising their functionality.
Also Read:
- Senate Grills Finance Minister, Others Over Subsidy Savings and Debt Servicing
- Telecom Tariff Hike Likely as Tinubu Administration Approves Increase
- Tesla Pi Phone Set to Revolutionize Smartphone Market
Growing Industry Adoption of Synthetic Data
Despite the challenges, synthetic data is already widely used by tech giants such as Microsoft, Meta, OpenAI, and Anthropic to train their AI models.
Microsoft’s Phi-4 model, released earlier this week, was trained on a mix of synthetic and real-world data. Similarly, Meta fine-tuned its latest Llama models using AI-generated data, and Anthropic developed its Claude 3.5 Sonnet system with partial reliance on synthetic inputs.
Gartner estimates that in 2024, 60% of data used for AI and analytics projects will be synthetically generated.
Industry Perspectives on Peak Data
Musk’s remarks align with earlier predictions by AI experts. Ilya Sutskever, former chief scientist at OpenAI, described this phenomenon as “peak data” in December 2023. Sutskever predicted that the scarcity of human-generated data would transform how AI models are developed.
Musk’s company, xAI, is among the players exploring innovative approaches to maintain AI advancements in this new era.
“The future of AI will be shaped by how effectively we navigate this shift to synthetic data while addressing its inherent challenges,” Musk emphasized.