Building Llms With Pytorch

Building LLMs with PyTorch

DESCRIPTION PyTorch has become the go-to framework for building cutting-edge large language models (LLMs), enabling developers to harness the power of deep learning for natural language processing. This book serves as your practical guide to navigating the intricacies of PyTorch, empowering you to create your own LLMs from the ground up. You will begin by mastering PyTorch fundamentals, including tensors, autograd, and model creation, before diving into core neural network concepts like gradients, loss functions, and backpropagation. Progressing through regression and image classification with convolutional neural networks, you will then explore advanced image processing through object detection and segmentation. The book seamlessly transitions into NLP, covering RNNs, LSTMs, and attention mechanisms, culminating in the construction of Transformer-based LLMs, including a practical mini-GPT project. You will also get a strong understanding of generative models like VAEs and GANs. By the end of this book, you will possess the technical proficiency to build, train, and deploy sophisticated LLMs using PyTorch, equipping you to contribute to the rapidly evolving landscape of AI. WHAT YOU WILL LEARN ● Build and train PyTorch models for linear and logistic regression. ● Configure PyTorch environments and utilize GPU acceleration with CUDA. ● Construct CNNs for image classification and apply transfer learning techniques. ● Master PyTorch tensors, autograd, and build fundamental neural networks. ● Utilize SSD and YOLO for object detection and perform image segmentation. ● Develop RNNs and LSTMs for sequence modeling and text generation. ● Implement attention mechanisms and build Transformer-based language models. ● Create generative models using VAEs and GANs for diverse applications. ● Build and deploy your own mini-GPT language model, applying the acquired skills. WHO THIS BOOK IS FOR Software engineers, AI researchers, architects seeking AI insights, and professionals in finance, medical, engineering, and mathematics will find this book a comprehensive starting point, regardless of prior deep learning expertise. TABLE OF CONTENTS 1. Introduction to Deep Learning 2. Nuts and Bolts of AI with PyTorch 3. Introduction to Convolution Neural Network 4. Model Building with Custom Layers and PyTorch 2.0 5. Advances in Computer Vision: Transfer Learning and Object Detection 6. Advanced Object Detection and Segmentation 7. Mastering Object Detection with Detectron2 8. Introduction to RNNs and LSTMs 9. Understanding Text Processing and Generation in Machine Learning 10. Transformers Unleashed 11. Introduction to GANs: Building Blocks of Generative Models 12. Conditional GANs, Latent Spaces, and Diffusion Models 13. PyTorch 2.0: New Features, Efficient CUDA Usage, and Accelerated Model Training 14. Building Large Language Models from Scratch

BUILDING LLMS WITH PYTORCH AND TENSORFLOW

Build a Large Language Model (From Scratch)

Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up! In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You’ll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks. Build a Large Language Model (from Scratch) teaches you how to: • Plan and code all the parts of an LLM • Prepare a dataset suitable for LLM training • Fine-tune LLMs for text classification and with your own data • Use human feedback to ensure your LLM follows instructions • Load pretrained weights into an LLM Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you’ll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant. About the technology Physicist Richard P. Feynman reportedly said, “I don’t understand anything I can’t build.” Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning. About the book Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself! What's inside • Plan and code an LLM comparable to GPT-2 • Load pretrained weights • Construct a complete training pipeline • Fine-tune your LLM for text classification • Develop LLMs that follow human instructions About the reader Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs. About the author Sebastian Raschka, PhD, is an LLM Research Engineer with over a decade of experience in artificial intelligence. His work spans industry and academia, including implementing LLM solutions as a senior engineer at Lightning AI and teaching as a statistics professor at the University of Wisconsin–Madison. Sebastian collaborates with Fortune 500 companies on AI solutions and serves on the Open Source Board at University of Wisconsin–Madison. He specializes in LLMs and the development of high-performance AI systems, with a deep focus on practical, code-driven implementations. He is the author of the bestselling books Machine Learning with PyTorch and Scikit-Learn, and Machine Learning Q and AI. The technical editor on this book was David Caswell. Table of Contents 1 Understanding large language models 2 Working with text data 3 Coding attention mechanisms 4 Implementing a GPT model from scratch to generate text 5 Pretraining on unlabeled data 6 Fine-tuning for classification 7 Fine-tuning to follow instructions A Introduction to PyTorch B References and further reading C Exercise solutions D Adding bells and whistles to the training loop E Parameter-efficient fine-tuning with LoRA