Defying the Algorithm: A Rebel's Call for True AI Intelligence

In a packed auditorium at the National University of Singapore, Professor Yann LeCun,Vice President and Chief AI Scientist at Meta and a pioneer in deep learning, delivered a provocative lecture on the future of AI. Speaking at the NUS20 Distinguished Speaker Series, LeCun challenged the AI community to move beyond the hype of Large Language Models (LLMs) and embrace a new paradigm for achieving human-like intelligence. His vision, centered on Advanced Machine Intelligence (AMI), emphasizes learning world models, reasoning, and planning, with a call for open-source collaboration to ensure AI serves humanity’s diverse needs.

The Limits of Large Language Models

LeCun began by acknowledging AI’s transformative impact, from revolutionizing healthcare to optimizing urban systems. However, he quickly turned to the limitations of LLMs, which dominate today’s AI landscape. Trained to predict the next token in a sequence, LLMs excel at tasks like generating text or passing exams but falter in understanding the physical world, reasoning, or inventing novel solutions. “An elephant or a four-year-old child is way smarter than any large language model,” LeCun quipped, noting that a child processes as much sensory data in four years as the largest LLMs do in text (10^14 bytes).

He criticized the industry’s “religion of scaling,” which assumes that larger models with more data will achieve human-level intelligence. This approach, he argued, is flawed due to autoregressive prediction’s exponential error accumulation, making LLMs unreliable for complex tasks. LeCun dismissed claims that LLMs will soon reach PhD-level capabilities, calling them a repetition of overly optimistic predictions from AI’s 70-year history.

A New Paradigm: Joint Embedding Predictive Architectures

To overcome these limitations, LeCun proposed a shift to Joint Embedding Predictive Architectures (JEPAs), which learn abstract representations of the world rather than reconstructing raw inputs. Unlike generative models, which struggle to predict video details (resulting in blurry outputs), JEPAs focus on high-level features, akin to how humans predict an object’s fall without specifying its exact trajectory. “If I drop this object, you know it’ll fall, but not which way,” he explained, drawing parallels to scientific models that simplify complex systems (e.g., predicting Jupiter’s orbit with six numbers).

JEPAs use self-supervised learning to train encoders that predict representations of full inputs from corrupted or partial ones. This approach, LeCun argued, is more efficient and robust, as demonstrated by Meta’s open-source DINO (for images) and V-JEPA (for video). These models outperform supervised learning in tasks like object classification and robotic planning, even detecting physically impossible events in videos, suggesting a rudimentary form of common sense.

Building Intelligent Systems with World Models

At the heart of LeCun’s vision is the concept of world models—mental representations of the physical world learned through observation and interaction, much like infants learn intuitive physics (e.g., object permanence, gravity). Future AI systems, he argued, must process high-bandwidth sensory inputs (video, touch) and perform inference through optimization, using energy-based models to assess input-output compatibility. This is more powerful than the fixed-layer propagation of LLMs, enabling reasoning and planning.

LeCun outlined an architecture where a perception module estimates the world’s state, a memory module retains past knowledge, and a world model predicts future states based on proposed actions. By optimizing action sequences to meet objectives (e.g., reaching a target state), such systems can plan hierarchically, breaking complex goals (e.g., traveling to Paris) into manageable sub-goals (e.g., getting to the airport). “Hierarchical planning is a major unsolved problem,” he noted, urging researchers to tackle it.

Recommendations for AI Research

LeCun’s lecture was a call-to-action for the AI community, with bold recommendations:

• Abandon Generative Models: For understanding the world, JEPAs are superior to generative models, which waste resources on unpredictable details.
• Shift to Energy-Based Models: Probabilistic models are less effective in abstract representation spaces, where energy-based frameworks excel.
• Minimize Reinforcement Learning: Inefficient for real-world tasks, it should be a minor component compared to self-supervised learning, which LeCun likened to the “bulk of the cake” in human intelligence.
• Focus on Unsolved Challenges: Hierarchical planning, scaling world models, handling uncertainty, and associative memory are critical for AMI.
• Embrace Open Source: Open-source platforms are essential for academic contributions, cultural diversity, and preventing dominance by a few tech giants.

Societal Implications and Global Collaboration

LeCun addressed fears about AI’s impact on jobs, expressing optimism based on economists’ predictions of a modest GDP growth increase (7%) rather than mass unemployment. He advised students to study subjects with “long shelf life” (e.g., quantum mechanics) to adapt to technological change and manage AI systems effectively.

A strong advocate for open-source AI, LeCun argued that humanity cannot rely on a handful of AI assistants from the US or China. With 6,000 languages and countless cultures, AI must be inclusive, preserving traditional knowledge and practices. He envisioned Singapore as a potential hub for collaborative AI development in Asia, leveraging its expertise and regional data to train foundation models that speak all languages and understand all cultures.

A Contrarian Vision for AI’s Future

LeCun’s lecture was both a critique of current AI trends and a blueprint for a smarter, more inclusive future. By dismissing LLMs as a “distraction” from human-level AI, he challenged researchers to focus on fundamental problems like reasoning and world model learning. His JEPA framework, grounded in neuroscience and physics, offers a promising path forward, with early successes in DINO and V-JEPA.

As he concluded, LeCun emphasized the urgency of open-source collaboration to ensure AI serves humanity’s diverse needs. “The most incomprehensible thing about the world is that it’s comprehensible,” he quoted Einstein, suggesting that AI, like science, must find the right abstractions to unlock its potential. For researchers, students, and policymakers, his message was clear: the next AI revolution requires bold ideas, global cooperation, and a commitment to understanding the world as humans do.

For more details, watch the lecture recording on NUScast (https://www.youtube.com/live/m3H2q6MXAzs)or read LeCun’s paper, “A Path Towards Advanced Machine Intelligence,” on Open Review.