Elon Musk agreed with other experts that little real-world data remains for training AI models, TechCrunch reports.
“Right now we have exhausted practically the entirety of human knowledge for training AI. That happened last year,” he said during a conversation with Stagwell chairman Mark Penn.
In December, OpenAI co-founder and founder of AI startup Safe Superintelligence Ilya Sutskever said the industry had reached the peak of data use. In his view, AI agents, synthetic information and accelerated computation are the next stage in artificial intelligence’s evolution, which will lead to the emergence of superintelligence.
Musk considers synthetic data — information generated by artificial intelligence — the way forward.
“The only way to augment [real-world information] is synthetic data, when AI creates [training information]. With such material [AI] will sort of evaluate itself and go through this self-learning process,” the entrepreneur added.
In 2024, AI startup Anthropic used synthetic data to train one of its flagship models Claude 3.5 Sonnet. Meta refined its Llama 3.1 models with AI-generated material. OpenAI also applies synthetic information to train o1 — a “reasoning” artificial intelligence.
Amid a shortage of high-quality data, AI startups have begun seeking new ways to scale.