Monitoring LLM Inference with Prometheus and Grafana (vLLM, TGI, Llama.cpp)(glukhov.org)

Learn how to monitor LLM inference in production using Prometheus and Grafana. Track p95 latency, tokens/sec, queue duration, and KV cache usage across vLLM, TGI, and llama.cpp. Includes PromQL examples, dashboards, alerts, Docker & Kubernetes setups.