What Happened
Meta released Llama 2, the successor to LLaMA, with a crucial difference: it came with a permissive commercial license allowing free use for most applications. The release included pre-trained and fine-tuned (chat) models at 7B, 13B, and 70B parameter scales, along with a detailed research paper describing the training methodology.
Why It Matters
Llama 2 was a strategic milestone for open-source AI. By granting a commercial license, Meta enabled startups, researchers, and enterprises to build products on top of capable foundation models without paying API fees or relying on closed providers. This:
- Accelerated open-source AI adoption in production environments
- Pressured closed-model providers on pricing and access
- Established Meta as the leader of the open-weight AI movement
- Created a foundation for thousands of fine-tuned models and applications
Technical Details
- Architecture: Transformer decoder with grouped-query attention (GQA) for the 70B model
- Sizes: 7B, 13B, 70B parameters
- Training data: 2 trillion tokens from publicly available sources (40% more than LLaMA)
- Context window: 4,096 tokens
- Alignment: Llama 2 Chat models were fine-tuned using supervised fine-tuning (SFT) followed by RLHF with over 1 million human annotations
- Performance: Llama 2 70B was competitive with GPT-3.5 on many benchmarks while being freely available