For Yann LeCun, one of the godfathers of AI, the method of modeling the real world developed by OpenAI for Sora is doomed to failure. Explanations.
At OpenAI, the first videos generated by Sora's AI are so impressive that they inspire both wonder and concern. One of the popes of artificial intelligence, the Frenchman Yann LeCun, not only knows what will distinguish true from false in the future, but has also just expressed sharp criticism of the technology used by OpenAI. Meta's head of AI thinks OpenAI's approach is bad. First, it questions the ambition of ChatGPTChatGPT's publisher to be able to create digital twins of the real world from its algorithms. On X (TwitterTwitter) he explains that the method of generating pixels from latent variables is doomed to failure because it is inefficient. OpenAI's models try to infer too many details that are irrelevant, he says. This use of energy to achieve this is commendable for generating videos from text, but when it comes to modeling the world it is not effective at all.
Too many details destroy the model
For LeCun, who has been working on machine learning and deep learning for 30 years, the reason the generative approach worked well with ChatGPT is because the text has a defined number of symbols. To simulate the world, on the other hand, we fall into a much broader and more complex area. While Meta's AI research remains under the radar due to OpenAI's fame and the flood of generative tools, Facebook's parent company Facebook is also working on its own AI model that can create videos.
The method developed by Yann LeCun and his team called Video Joint Embedding Predictive Architecture (V-JEPA) is in fact quite different. The algorithm doesn't try to derive pixels, but rather gets to the essence by removing anything unpredictable. This system would improve training by a factor of 1.5 to 6. Only time will tell whether criticism of Yann LeCun and his model will guide the decisions of competing companies.