LM Studio

LM StudioSince 2023-04

LM Studio is the premier desktop application for discovering, downloading, and running large language models locally on your personal computer. With a polished visual interface, LM Studio makes running models like Llama 3.3, DeepSeek-R1, Mistral, and Qwen as simple as a single click, keeping all personal conversations, code, and documents 100% private and offline on your hardware. Whether running on Apple Silicon with unified memory or dedicated Nvidia/AMD GPUs, LM Studio automatically configures optimal hardware acceleration (Metal, CUDA, ROCm, Vulkan) for smooth, high-speed local inference.

Beyond conversational chat, LM Studio includes a built-in local developer server that exposes OpenAI-compatible `/v1/models` and `/v1/chat/completions` endpoints on localhost. This enables developers to connect IDE extensions (Cursor, VS Code, Windsurf) and local agent pipelines directly to offline models. It features comprehensive support for GGUF model formats from Hugging Face, multi-model side-by-side benchmarking, GPU layer offloading controls, and system prompt presets.

vLLMSince 2023-06

vLLM is the industry-standard open-source LLM serving and inference engine designed for ultra-high throughput and minimal memory waste. Developed by UC Berkeley researchers, vLLM introduced PagedAttention—a revolutionary memory management algorithm that manages attention key-value (KV) cache like virtual memory in operating systems, virtually eliminating memory fragmentation. Capable of delivering 2x to 4x higher throughput than Hugging Face TGI and standard PyTorch runtimes, vLLM powers production AI inference infrastructure across enterprise cloud clusters and high-volume API providers worldwide.

vLLM features state-of-the-art inference optimizations including continuous request batching, Chunked Prefill, speculative decoding, prefix caching, and native quantization support (AWQ, GPTQ, FP8, INT4, SqueezeLLM). It provides drop-in OpenAI-compatible REST API endpoints, supports multi-GPU distributed tensor parallelism with Ray/NCCL, and serves all major model architectures including DeepSeek-V3, Llama 3.3, Mistral, Qwen 2.5, and Command R+.