DSPy

DSPySince 2026-08

DSPy is an open-source framework created by Stanford University that replaces fragile, manual prompt engineering with algorithmic programming and systematic optimization. Instead of hand-tweaking prompt strings and few-shot examples, DSPy allows developers to express multi-stage AI workflows as modular Python modules with declarative Signatures. DSPy’s teleprompter optimizers automatically synthesize optimal prompt instructions, select high-performing few-shot demonstrations, and fine-tune smaller local language model weights to maximize pipeline accuracy on defined validation metrics.

DSPy introduces a compiler-like mental model for building with Language Models. Developers define Signatures (input/output contracts like "question -> answer" or "context, query -> rationale, sql") and assemble them into Modules like ChainOfThought, ReAct, or MultiHop. When compiled against a small training set, DSPy systematically optimizes prompt variations and demonstration examples, often boosting pipeline performance by 20% to 40% while making pipelines resilient to underlying model swaps.

vLLMSince 2023-06

vLLM is the industry-standard open-source LLM serving and inference engine designed for ultra-high throughput and minimal memory waste. Developed by UC Berkeley researchers, vLLM introduced PagedAttention—a revolutionary memory management algorithm that manages attention key-value (KV) cache like virtual memory in operating systems, virtually eliminating memory fragmentation. Capable of delivering 2x to 4x higher throughput than Hugging Face TGI and standard PyTorch runtimes, vLLM powers production AI inference infrastructure across enterprise cloud clusters and high-volume API providers worldwide.

vLLM features state-of-the-art inference optimizations including continuous request batching, Chunked Prefill, speculative decoding, prefix caching, and native quantization support (AWQ, GPTQ, FP8, INT4, SqueezeLLM). It provides drop-in OpenAI-compatible REST API endpoints, supports multi-GPU distributed tensor parallelism with Ray/NCCL, and serves all major model architectures including DeepSeek-V3, Llama 3.3, Mistral, Qwen 2.5, and Command R+.