O'Reilly said the first physical copies would appear around Dec 22 but my copies arrive on Jan 7 :(

6. AI Incident Database

For those interested in seeing how AI can go wrong, this contains over 3000 reports of AI harms: incidentdatabase.ai

5. Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models (Lu et al., 2023)

A cool study on LLM planners, how they use tools, and their failure modes. An interesting finding is that different LLMs have different tool preferences: arxiv.org/abs/2304.09842

4. Efficiently Scaling Transformer Inference (Pope et al., 2022)

An amazing paper about inference optimization for transformers. It provides a guideline to optimize for different aspects, e.g. lowest possible latency, highest possible throughput, or longest context length: arxiv.org/abs/2211.05102

3. Llama 3 paper

The section on post-training data is a gold mine! It details different techniques they used to generate 2.7M examples for instruction finetuning. It also covers synthetic data verification! arxiv.org/abs/2407.21783

2. OpenAI’s best practices for finetuning

While this guide focuses on GPT-3, many techniques are applicable to finetuning in general. It explains how finetuning works, how to prepare training data, how to pick hyperparameters, and common finetuning mistakes: docs.google.com/document/d/1...

The highlights:

1. Anthropic’s Prompt Engineering Interactive Tutorial

The Google Sheets-based interactive exercises make it easy to experiment with different prompts. docs.google.com/spreadsheets...