LLMs from Scratch is the official code repository for the book Build a Large Language Model (From Scratch) by Sebastian Raschka. It is for Python programmers with some grounding in deep learning who want to understand how large language models work by building one, rather than only calling an API.
The code follows the book chapter by chapter: working with text data, attention mechanisms, implementing a GPT model, pretraining, classification and instruction finetuning, with LoRA covered in an appendix. The main code is designed to run on an ordinary laptop. The repository also links to a companion video course and to a sequel on reasoning models.
Categories
Similar apps
Open-source library of community AI prompts, formerly Awesome ChatGPT Prompts
Open-source AI agent, agent skill and RAG app examples to run and adapt
A free roadmap to large language models, with hands-on Colab notebooks