DeepSeek V3 Released: A 671B MoE Model That Shook the Industry on Cost

DeepSeek released the V3 large model on December 26, 2024: an MoE architecture with 671B total parameters and 37B activated per token, trained on 14.8 trillion high-quality tokens, reaching the strongest open-model level of its time across benchmarks.

What shook the industry most was its training efficiency — the technical report disclosed a full training cost of roughly $5.57 million in GPU compute, a fraction of the hundreds of millions spent on comparable closed models.

How $5.57M Was Possible

The technical report sketches the answer: sparse MoE activation (37B of 671B per token) cuts per-token compute to a sliver of dense models, stacked with FP8 mixed-precision training, multi-token prediction and layer upon layer of engineering optimization. The real lesson is not one magic trick but systems engineering: under export controls constraining top-end chips, squeezing every unit of compute became a comparative advantage.

The Caveat: $5.57M Is Not the Whole Bill

To be clear, the figure covers only the GPU rental for the final training run — not prior experiments, data construction, team or infrastructure — so comparing it directly against closed vendors' all-in spending is unfair. Yet even adjusting for scope, the order-of-magnitude gap is real, and real enough to ignite the global cost-paradigm debate.

V3 Was the Foundation; R1 Was the Detonation

On the timeline, V3 (Dec 2024) laid the base; R1 (Jan 2025) lit reasoning on top of it with reinforcement learning — and only then came Nvidia's historic $589 billion single-day wipeout (see our coverage). Markets shrugged at V3's launch, but practitioners read the signal in the technical report: the inflection point is usually planted one release before the explosion.

Open and Commercial-Ready: The Underrated Other Half

Just as consequential as the cost figure was V3's choice to release open weights with commercial use allowed — meaning any team could deploy, fine-tune or distill the strongest open base of its day on their own infrastructure. For cloud vendors it instantly became the price anchor for inference APIs; for small teams it dropped the bar for 'running a frontier model' from negotiating a business deal to downloading weights; for the open-source cohort that followed, it set the default motion of 'ship weights plus a thorough technical report.' The Llama line opened that door, but V3 raised the bar to frontier level. The later gpt-oss release (see our coverage) reads as a response to that watermark: once the open frontier was held by Chinese models, the closed camp had to re-enter the arena.

Our Take

Weights and the technical report are fully open; V3 proved careful engineering can radically lower the frontier bar. The deeper impact is methodological: it turned 'not enough compute' from an excuse into an engineering problem. Qwen and Kimi's MoE lines carried the relay (see our coverage), making the Chinese open-source cost advantage a systemic capability — and the methodology all traces back to V3's technical report.