LLMOps: Model Quantization and Inference with ONNX Runtime

Running large language models efficiently requires optimizing hardware resources, reducing memory overhead, and speeding up token generation. This comprehensive text-based course guides you through foundational LLMOps practices to compress and run models smoothly using ONNX Generative Runtime. You will build a thorough understanding of model architecture export, precision reduction, and modern inference pipelines through detailed written walkthroughs and clear code examples. What you'll learn: - Understand foundational LLMOps terminology, execution graphs, and model architectures - Master core quantization concepts, including weight-only and dynamic quantization formats - Convert modern open-source language models to ONNX representations - Configure ONNX Generative Runtime execution providers for CPU and GPU acceleration - Evaluate trade-offs between model memory footprint, inference latency, and text generation quality - Implement structured Python inference scripts for streaming and batch text completion The course begins with fundamental machine learning deployment concepts and mathematical foundations of quantization before progressing to hands-on configuration and optimization workflows. This course is structured for software developers, AI practitioners, and beginners interested in machine learning operations. Basic familiarity with Python is helpful, but no previous model optimization experience is required. Start reading today to build efficient, scalable deployment pipelines for language models.