llm

Kimi K3: Is It Really Open Source? And What Would It Take to Recreate It?

Luigi

• 5 min read
Share
Kimi K3: Is It Really Open Source? And What Would It Take to Recreate It?

1. What Is Kimi K3?

Kimi K3 is the latest flagship model from Moonshot AI (Beijing), released July 16, 2026. It is a Mixture-of-Experts model with 2.8 trillion total parameters and about 32 billion active per token (16 out of 896 experts). It supports 1 million tokens of context and handles text, images, and video natively.

On launch, K3 ranked #3 on the Artificial Analysis leaderboard behind Claude Fable 5 (Anthropic) and GPT 5.6 Sol (OpenAI), but #1 on Arena.ai for front-end coding.

2. Key Specs

Spec Value
Total params 2.8T
Active params ~32B
Architecture MoE
Experts 896 total, 16 active per token
Context 1M tokens
Multimodal Text + Images + Video (native)
Attention Kimi Delta Attention (KDA) + Attention Residuals (AttnRes)
MoE framework Stable LatentMoE
Activation Sigmoid Tanh Unit (SiTU)
Quantization MXFP4 weights, MXFP8 activations (QAT from SFT onward)
Thinking Always on (max effort by default)
Input price $3.00/MTok (cache miss), $0.30/MTok (cache hit)
Output price $15.00/MTok

3. Is It Actually Open Source?

Status as of July 17, 2026

Element Public? Details
Weights Pending release by July 27, 2026 Announced by Moonshot
Technical report Not yet published "Coming soon"
API Yes OpenAI-compatible
Consumer apps Yes Kimi.com, Kimi Work, Kimi Code
Architecture code No KDA, AttnRes not released
Training code No No RL scripts
Training data No Neither pretraining nor post-training

The Pattern

Moonshot follows a consistent playbook:

  • K2 (July 2025): Weights on HuggingFace with Modified MIT License. MAI training code released.
  • K2.5 (Jan 2026): Same terms. Community accused them of not being truly open source.
  • K3 (July 2026): Weights promised, but no training code or datasets.

Verdict: K3 is open-weight, not open source by OSI standards. You will be able to download and use the weights freely, but the entire training system stays proprietary. This is the same business model DeepSeek and other Chinese labs use: release weights for adoption, keep training details secret for competitive advantage.

4. Moonshot Timeline

Oct 2023: Kimi chatbot (128K context)
Mar 2024: Kimi 2M character context
Jan 2025: Kimi k1.5 (RL scaling, matched o1)
Apr 2025: Kimi-VL (16B MoE, open source)
Jun 2025: Kimi-Dev (72B coding), Kimi-Researcher
Jul 2025: Kimi K2 (1T params, 32B active, open-weight)
Sep 2025: K2-Instruct-0905 (256K context)
Oct 2025: Kimi Linear (48B, KDA preview)
Jan 2026: Kimi K2.5 (multimodal, Agent Swarm)
Apr 2026: Kimi K2.6 (1000+ parallel agents)
Jun 2026: Kimi K2.7 Code
Jul 16, 2026: Kimi K3 (2.8T params)
Jul 27, 2026: K3 weights release (promised)

5. What Would It Take to Recreate K3?

Assume you have the hardware (tens of thousands of H200/B200 GPUs with supernode interconnects of 64+ accelerators).

5.1 Architecture (Not Documented)

K3's architectural innovations are not publicly detailed:

  • Kimi Delta Attention (KDA): A hybrid linear attention variant. Previewed in Kimi Linear (Oct 2025), but precise details are not public.
  • Attention Residuals (AttnRes): Selectively recovering representations across network depth instead of accumulating them uniformly. Implementation unknown.
  • Stable LatentMoE: Routing framework with 896 experts (16 active). Includes Quantile Balancing that eliminates balancing hyperparameters.
  • Per-Head Muon: An extension of MuonClip that optimizes attention heads independently.
  • Sigmoid Tanh Unit (SiTU): New activation function, not documented.
  • Gated MLA: Evolved version of Multi-Head Latent Attention.

Estimated effort: 6-12 months of reverse engineering just to reconstruct the architecture.

5.2 Pretraining Data

  • K3 was trained on a corpus of unknown size (estimated 20T+ tokens vs 15.5T for K2)
  • Proprietary rephrasing techniques
  • Training curriculum (LR scheduling, data proportions for multilingual/code/math, long-context phases) is entirely not public

5.3 Post-Training Data and Pipeline

  • Agentic data synthesis: Unknown scale, likely much larger than K2 (3000+ real tools, 20,000+ synthetic)
  • Preference data for initializing the critic model: internal only
  • Trajectory generation pipeline: not public

5.4 RL Algorithms and Hyperparameters

K3's technical report has not been published yet. Based on K2 and Kimi-Researcher, the unknowns include:

Parameter Known? Detail
RL algorithm No REINFORCE? PPO? GRPO?
Reward model No Not specified for K3
Gamma-decay factor No Unknown thresholds
Temperature decay schedule No Not published
Token budget limits No Not specified
Negative sample control ratio No Not published
Turn-level partial rollout No Precise mechanism unknown
Context management strategy No Needed for 1M tokens

5.5 Training Infrastructure

  • QAT with MXFP4/MXFP8: proprietary implementation
  • Supernode config with 64+ accelerators: not documented
  • Fully balanced expert-parallel training: code not released
  • Asynchronous rollout system for agent RL: not public
  • Sandbox Kubernetes + MCP: configuration not documented

5.6 Estimated Cost

  • Pretraining only: ~$15-25M (estimate based on 2.8T params with 20T+ tokens)
  • R&D reverse engineering: $5-10M
  • Post-training RL: $3-8M for experiments
  • Infrastructure: $2-5M for cluster setup
  • Total: $25-50M+, with 12-18 months of work from a team of 50+ researchers

6. Why K3 Matters

The US-China Gap Is Shrinking

K3 shows Chinese labs can produce frontier models months earlier than US analysts expected. Bank of America:

"Despite persistent hardware/compute capacity constraints in China, K3 demonstrates that pre-training scaling, paired with architectural innovation, can still deliver step-change gains."

Price Pressure

Model Output price (per MTok)
Claude Fable 5 $50.00
Kimi K3 $15.00
GLM-5.2 (z.ai) $4.40
DeepSeek V4 $0.87

K3 costs one-third of Fable 5 for comparable performance.

Distillation Allegations

Anthropic accused Moonshot (along with z.ai, MiniMax, Alibaba, DeepSeek) of "illicit distillation attacks" -- using US model outputs to train their own. The US Congress is considering legislation to stop this.

Regulatory Context

  • July 2026: US temporarily blocked Mythos/Fable 5 exports
  • June 2026: Executive order on GPT-5.6 Sol licenses
  • K3 launched on the eve of the World AI Conference 2026 in Shanghai

7. Final Verdict

Question Answer
Is K3 open source? No, it is open-weight (weights yes, training code no)
Can you download and use it? Yes, from July 27, 2026
Can you modify it? Yes (Modified MIT License)
Can you recreate it from scratch? No, not without proprietary knowledge
Is it reproducible with $50M? Maybe, with 12-18 months of reverse engineering
Does it compete with top US models? Yes, #1 in front-end coding, #3 overall

Bottom line: Kimi K3 is a remarkable engineering achievement, but reproducing it is a pipe dream for anyone outside Moonshot AI. The weights will be open, but the science behind them stays mostly secret. This is the new industry standard: models keep getting more powerful, and they keep getting harder to reproduce.

Sources