Build A Large Language Model From Scratch
Download Build A Large Language Model From Scratch PDF/ePub or read online books in Mobi eBooks. Click Download or Read Online button to get Build A Large Language Model From Scratch book now. This website allows unlimited access to, at the time of writing, more than 1.5 million titles, including hundreds of thousands of titles in various foreign languages.
Build a Large Language Model (From Scratch)
Author: Sebastian Raschka
language: en
Publisher: Simon and Schuster
Release Date: 2024-10-29
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up! In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You’ll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks. Build a Large Language Model (from Scratch) teaches you how to: • Plan and code all the parts of an LLM • Prepare a dataset suitable for LLM training • Fine-tune LLMs for text classification and with your own data • Use human feedback to ensure your LLM follows instructions • Load pretrained weights into an LLM Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you’ll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant. About the technology Physicist Richard P. Feynman reportedly said, “I don’t understand anything I can’t build.” Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning. About the book Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself! What's inside • Plan and code an LLM comparable to GPT-2 • Load pretrained weights • Construct a complete training pipeline • Fine-tune your LLM for text classification • Develop LLMs that follow human instructions About the reader Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs. About the author Sebastian Raschka, PhD, is an LLM Research Engineer with over a decade of experience in artificial intelligence. His work spans industry and academia, including implementing LLM solutions as a senior engineer at Lightning AI and teaching as a statistics professor at the University of Wisconsin–Madison. Sebastian collaborates with Fortune 500 companies on AI solutions and serves on the Open Source Board at University of Wisconsin–Madison. He specializes in LLMs and the development of high-performance AI systems, with a deep focus on practical, code-driven implementations. He is the author of the bestselling books Machine Learning with PyTorch and Scikit-Learn, and Machine Learning Q and AI. The technical editor on this book was David Caswell. Table of Contents 1 Understanding large language models 2 Working with text data 3 Coding attention mechanisms 4 Implementing a GPT model from scratch to generate text 5 Pretraining on unlabeled data 6 Fine-tuning for classification 7 Fine-tuning to follow instructions A Introduction to PyTorch B References and further reading C Exercise solutions D Adding bells and whistles to the training loop E Parameter-efficient fine-tuning with LoRA
Test Yourself On Build a Large Language Model (From Scratch)
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up! Sebastian Raschka’s bestselling book Build a Large Language Model (From Scratch) is the best way to learn how Large Language Models function. It uses Python and the PyTorch deep learning library. It’s a unique way to learn this subject, which some believe is the only way to truly learn: you build a model yourself. Even with the clear explanations, diagrams, and code in the book, learning a complex subject is still hard. This Test Yourself guide intends to make it a little easier. The structure mirrors the structure of Build a Large Language Model (From Scratch), focusing on key concepts from each chapter. You can test yourself with multiple-choice quizzes, questions on code and key concepts, and questions with longer answers that push you to think critically. The answers to all questions are provided. Depending on what you know at any point, this Test Yourself guide can help you in different ways. It will solidify your knowledge if used after reading a chapter. But it will also benefit you if you digest it before reading. By testing yourself on the main concepts and their relationships you are primed to navigate a chapter more easily and be ready for its messages. We recommend using it before and after reading, as well as later when you have started forgetting. Repeated learning solidifies our knowledge and integrates it with related knowledge already in our long-term memory. What's inside • Questions on code and key concepts • Critical thinking exercises requiring longer answers • Answers for all questions About the reader For readers of Build a Large Language Model (From Scratch) who want to enhance their learning with exercises and self-assessment tools. About the author Curated from Build a Large Language Model (From Scratch)