> ## Content Index > Fetch the complete content index at: https://spyglass.org/llms.txt > Use this file to discover other available public pages before exploring further. # Meta's 'MoE' Mistake - URL: https://spyglass.org/metas-moe-ai-mistake/ - Published: 2025-07-30T09:54:45.000Z - Updated: 2025-09-06T08:52:28.000Z - Description: More details on how Llama's open source AI backfired... - Author: M.G. Siegler - Tags: llama, meta, meta ai, ai, openai, anthropic, deepseek, tech, mark zuckerberg, #nodate [Meta’s AI spending spree is Wall Street’s focus in second-quarter earningsMeta investors will be scrutinizing CEO Mark Zuckerberg’s AI hiring blitz during the company’s second-quarter earnings on Wednesday.![](https://storage.ghost.io/c/af/ca/afcaa655-46e2-45b8-889a-2881de5cce69/content/images/icon/favicon-31.ico)CNBCJonathan Vanian![](https://storage.ghost.io/c/af/ca/afcaa655-46e2-45b8-889a-2881de5cce69/content/images/thumbnail/108043098-1727989387640-gettyimages-2173579499-META_CONNECT-1.jpeg)](https://www.cnbc.com/2025/07/29/meta-ai-q2-earnings.html?ref=spyglass.org) A month ago, I wrote a post entitled ["Meta's Open Source AI Mistake"](https://spyglass.org/metas-open-source-ai-mistake/) outlining how the company found itself in the position of [needing to pay](https://spyglass.org/meta-godfather-offer/) ([or at least offer](https://spyglass.org/meta-openai-the-future-of-ai/)) individuals [hundreds of millions of dollars](https://spyglass.org/ai-signing-bonuses/) to come help to [reboot their AI efforts](https://spyglass.org/meta-scale-ai-reset/). At a high level, their strategy for Llama not only wasn't working in terms of getting Meta to the cutting edge of AI, but the "open source" (read: open weight) ideals they were trying to adhere to actually may have *backfired*. This new reporting on the matter goes more granular on those issues: > Llama 4′s struggles can be traced back to January, when the sudden rise and ensuing popularity of the open-source R1 AI model by [DeepSeek](https://www.cnbc.com/2025/02/19/china-deepseek-origin-story.html?ref=spyglass.org) caught Meta off guard, leading to a reevaluation of Llama’s underlying architecture, the people said. > > DeepSeek’s R1 is a so-called mixture-of-experts AI model, or MoE. R1 is similar to OpenAI’s [o1 family of models](https://www.cnbc.com/2025/04/16/openai-releases-most-advanced-ai-model-yet-o3-o4-mini-reasoning-images.html?ref=spyglass.org) that can be trained to excel at multistep tasks like solving math equations or writing code. > > By contrast, Llama’s models — before their latest release - were dense AI models, which are generally simpler for most AI developers to fine-tune and incorporate into their own apps, the people said. To add insult to this injury, DeepSeek R1 was largely distilled from Llama! Because it was open source! Essentially, DeepSeek used Meta's foundation to [showcase a better way](https://spyglass.org/ai-deepseek-panic/) to build a better model (MoE), [perhaps augmenting it](https://www.theguardian.com/technology/2025/jan/29/openai-chatgpt-deepseek-china-us-ai-models?ref=spyglass.org) with (decidedly not open source) work from OpenAI and others. This was an "oh shit" moment for Meta internally, and so they seemingly scrambled to build Llama 4 in such a manner: > Suddenly, Meta executives thought they had a clearer picture into how to create their own efficient and possibly cheaper MoE models, potentially leapfrogging rivals like OpenAI, people familiar with the matter said. > > Still, some staff members in Meta’s GenAI unit pushed for Llama 4 to remain a dense AI model, which though generally less efficient, is still powerful, and Meta originally planned on that architecture acting as the backbone supporting improved voice recognition capabilities, the people said. > > Ultimately, Meta went with the MoE approach, due in part to DeepSeek’s innovations and the promise of pulling ahead of OpenAI, the people said. Meta [released](https://www.cnbc.com/2025/04/05/meta-debuts-new-llama-4-models-but-most-powerful-ai-model-is-still-to-come.html?ref=spyglass.org) two small versions in April and said a “Behemoth” version would come at a later date. > > But the new MoE architecture disappointed some developers, who were simply hoping Llama 4 would be a souped-up version of Llama 3, people familiar with the matter said. Llama 4 also failed to deliver a significant leap over competing open-source models from China, the people said. To put it in terms that longtime Apple watchers may appreciate: the people wanted [a better Apple II](https://en.wikipedia.org/wiki/Apple%5FII?ref=spyglass.org#Apple%5FIIe), not [the Lisa](https://en.wikipedia.org/wiki/Apple%5FLisa?ref=spyglass.org). Add to this that the "dense" version of Llama was proving to be extremely expensive for Meta to maintain, especially when others were just going to mooch off the work, rather than contribute back to make Meta itself stronger. Zuckerberg passed the hat around to try to get some help on that financial burden, but got no takers because – [why buy the Llama when you get the model for free](https://spyglass.org/llama-milk/)? > Executives at Meta as well as the Superintelligence Labs’ high-profile hires are now questioning the company’s current open-source AI strategy, and have considered skipping the release of Behemoth in favor of developing a more powerful proprietary AI model, the people said. While "spokespeople" continue to downplay this notion, it seems pretty clear that [Meta will go down this path](https://spyglass.org/open-source-ai-was-the-path-forward/) with [the new group](https://spyglass.org/zucks-eleven/). I'm guessing that they'll open source *some* of the models eventually – just as OpenAI itself is now [gearing up to do](https://spiral.spyglass.org/p/big-ai-and-big-streaming?ref=spyglass.org) – but the main work is likely to be behind closed doors, just as it is at OpenAI, Anthropic, and elsewhere. The real question for Meta: can [their newly formed](https://spyglass.org/zucks-eleven/) [band of pirates](https://spyglass.org/meta-scale-ai-reset/) ship the Mac? [Meta’s Open Source AI MistakeThe writing isn’t just on the wall for Llama, it’s on the new paychecks…![](https://storage.ghost.io/c/af/ca/afcaa655-46e2-45b8-889a-2881de5cce69/content/images/icon/Spyglass-Rings-----Multi-58.png)SpyglassM.G. Siegler![](https://storage.ghost.io/c/af/ca/afcaa655-46e2-45b8-889a-2881de5cce69/content/images/thumbnail/mgs22_a_sad_llama_roaming_alone_--ar_43_--v_7_40cbf839-3706-4f29-b2b0-663940a36285_0-3-5.png)](https://spyglass.org/metas-open-source-ai-mistake/) [Meta’s $10B+ AI ResetZuckerberg recruits a band of pirates to shake up and wake up their AI efforts – including a new don’t-call-it-a-deal for Scale AI talent…![](https://storage.ghost.io/c/af/ca/afcaa655-46e2-45b8-889a-2881de5cce69/content/images/icon/Spyglass-Rings-----Multi-59.png)SpyglassM.G. Siegler![](https://storage.ghost.io/c/af/ca/afcaa655-46e2-45b8-889a-2881de5cce69/content/images/thumbnail/mgs22_a_llama_getting_resuscitated_--ar_169_--v_7_abc01d06-74ab-4e1f-ae5f-fa4a530f9a94_0-7.png)](https://spyglass.org/meta-scale-ai-reset/) [Open Source AI was the Path Forward...until it wasn’t for Meta![](https://storage.ghost.io/c/af/ca/afcaa655-46e2-45b8-889a-2881de5cce69/content/images/icon/Spyglass-Rings-----Multi-57.png)SpyglassM.G. Siegler![](https://storage.ghost.io/c/af/ca/afcaa655-46e2-45b8-889a-2881de5cce69/content/images/thumbnail/mgs22_a_llama_at_a_fork_in_the_road_cartoon_--v_7_c9b104a1-864e-4273-ae51-d927024a17b3_0-2-3.png)](https://spyglass.org/open-source-ai-was-the-path-forward/)