AI Inference with Ollama, llama.cpp, and vLLM

Gk Marballi

ISBN 10: 1105842738 / ISBN 13: 9781105842733

Editore: Lulu.Com, 2026

Riguardo questo articolo

nach der Bestellung gedruckt Neuware - Printed after ordering - The era of cloud-dependent AI is over. Today's developers can run state-of-the-art language models on their own hardware-from laptops to GPU clusters-without ever sending data to a third party. But the gap between downloading a model and deploying it efficiently is filled with questions about quantization, memory bandwidth, batching strategies, and tool selection. This book is your guide through that gap, showing you how to build scalable, cost-effective inference systems using the three pillars of open-source AI: Ollama, llama.cpp, and vLLM.AI Inference with Ollama, llama.cpp, and vLLM takes you from running your first local model in minutes to optimizing production deployments serving thousands of requests per second. You'll learn when to use each tool, how to navigate the memory wall that bottlenecks LLM performance, and how to choose the right hardware and quantization strategy for your use case. Whether you're building RAG systems, deploying chatbots, or scaling inference across GPU clusters, this book gives you the practical knowledge to move from experimentation to production with confidence.About the AuthorGK Marballi has spent 20+ years turning data into competitive advantage for global brands from Priceline to S&P Global and Barnes & Noble. He has led high-impact product and analytics teams, and navigated the front lines of the AI revolution. He is based in New York City and holds an MBA from Harvard Business School.

Codice articolo 9781105842733

Dati bibliografici

• Titolo: AI Inference with Ollama, llama.cpp, and vLLM
• Editore: Lulu.Com
• Data di pubblicazione: 2026
• Lingua: Inglese
• ISBN 10: 1105842738
• ISBN 13: 9781105842733
• Rilegatura: Taschenbuch
• Condizione: Neu
• Dimensioni: 229 L x 152 A x 14 P (millimetri) 229 millimetri di larghezza per 152 millimetri di altezza per 14 millimetri di profondità
• Peso: 361 grammi

Descrizione articolo

Riassunto

The era of cloud-dependent AI is over. Today's developers can run state-of-the-art language models on their own hardware-from laptops to GPU clusters-without ever sending data to a third party. But the gap between downloading a model and deploying it efficiently is filled with questions about quantization, memory bandwidth, batching strategies, and tool selection. This book is your guide through that gap, showing you how to build scalable, cost-effective inference systems using the three pillars of open-source AI: Ollama, llama.cpp, and vLLM. AI Inference with Ollama, llama.cpp, and vLLM takes you from running your first local model in minutes to optimizing production deployments serving thousands of requests per second. You'll learn when to use each tool, how to navigate the memory wall that bottlenecks LLM performance, and how to choose the right hardware and quantization strategy for your use case. Whether you're building RAG systems, deploying chatbots, or scaling inference across GPU clusters, this book gives you the practical knowledge to move from experimentation to production with confidence. About the Author GK Marballi has spent 20+ years turning data into competitive advantage for global brands from Priceline to S&P Global and Barnes & Noble. He has led high-impact product and analytics teams, and navigated the front lines of the AI revolution. He is based in New York City and holds an MBA from Harvard Business School.