Glossary and references

源码版本v0.25.1

Core terminology

Term	English	Description
Engine args	EngineArgs	All startup parameters; derives VllmConfig. vllm/engine/arg_utils.py
Global config	VllmConfig	Aggregate of model / cache / scheduler / parallel sub-configs. vllm/config/__init__.py
LLM offline class	LLM	Synchronous offline generation entry, LLM.generate. vllm/entrypoints/llm.py
Async LLM	AsyncLLM	Async implementation of EngineClient, server entry. vllm/v1/engine/async_llm.py
LLM engine	LLMEngine	v1 thin shell, add_request/step, delegates to EngineCore. vllm/v1/engine/llm_engine.py
Engine core	EngineCore	The actual core, composes scheduler + worker + kv cache. vllm/v1/engine/core.py
Engine core process	EngineCoreProc	ZMQ background process shell, inherits from EngineCore. vllm/v1/engine/core.py
Engine client	EngineCoreClient	Inproc/MP/AsyncMP client ABC. vllm/v1/engine/core_client.py
In-process client	InprocClient	No process spawned, calls step_fn directly. vllm/v1/engine/core_client.py
Multiprocess client	MPClient	Multiprocess synchronization, communicates over ZMQ. vllm/v1/engine/core_client.py
Scheduler	Scheduler	waiting/running queues, schedule, preemption. vllm/v1/core/sched/scheduler.py
Request queue	RequestQueue	FCFS / Priority strategies. vllm/v1/core/sched/queue.py
Continuous batching	Continuous Batching	Dynamically builds batches every step, prefill and decode interleaved. vllm/v1/core/sched/scheduler.py
Preemption	Preemption	Evicts running requests when memory is tight, frees space for new requests. vllm/v1/core/sched/scheduler.py
Executor	Executor	UniProc/Multiproc/Ray variants. vllm/v1/executor/
Ray distributed executor	RayDistributedExecutor	Schedules workers across a Ray cluster. vllm/v1/executor/ray_distributed_executor.py
Worker	Worker	Distributed environment + GPU execution unit. vllm/v1/worker/gpu_worker.py
Model runner	GPUModelRunner	execute_model, input_batch, cudagraph management. vllm/v1/worker/gpu_model_runner.py
Microbatch	ubatch	Splits requests into microbatches inside a step to fit cudagraph. vllm/v1/worker/gpu_model_runner.py
CUDA graph	cudagraph	Pre-recorded kernel sequence with fixed shape, eliminates launch overhead. vllm/v1/worker/gpu_model_runner.py
Default model loader	DefaultModelLoader	Pulls weights from HF/s3 etc., safetensors / pytorch formats. vllm/model_executor/model_loader/default_loader.py
Column-parallel linear	ColumnParallelLinear	Weights sharded along output dim, all-gather after forward or reduce before RowParallel. vllm/model_executor/layers/linear.py
QKV-parallel linear	QKVParallelLinear	Shards Q/K/V together by head, merges projections for GQA/MQA. vllm/model_executor/layers/linear.py
Row-parallel linear	RowParallelLinear	Weights sharded along input dim, all-reduce after forward. vllm/model_executor/layers/linear.py
KV cache manager	KVCacheManager	Block allocation and lifecycle, across groups. vllm/v1/core/kv_cache_manager.py
KV cache coordinator	KVCacheCoordinator	Hybrid/Unitary/NoPrefixCache variants. vllm/v1/core/kv_cache_coordinator.py
Block pool	BlockPool	PagedAttention block allocation + prefix cache LRU. vllm/v1/core/block_pool.py
Paged attention	PagedAttention	Splits KV cache by block, avoids contiguous memory fragmentation. csrc/attention/
Prefix caching	Prefix Caching	Reuses KV blocks of shared prefixes across requests. vllm/v1/core/kv_cache_manager.py
Attention backend	AttentionBackend	Abstraction layer for FlashAttention/FlashInfer/Triton/MLA. vllm/v1/attention/backend.py
FlashAttention	FlashAttention	Dao Labs flash_attn kernel, auto-selects FA2/3/4. vllm/v1/attention/backends/flash_attn.py
FlashInfer	FlashInfer	Kernels from the FlashInfer project, native GQA/FP8/large-page support. vllm/v1/attention/backends/flashinfer.py
Triton	Triton	Pure-Python Triton kernels, fp32 and edge-quantization fallback. vllm/v1/attention/backends/triton_attn.py
Multi-head latent attention	MLA	Multi-head Latent Attention, DeepSeek family, caches latent + RoPE. vllm/v1/attention/backends/mla/
Sampler	Sampler	forward/sample, logits → token. vllm/v1/sample/sampler.py
Logits processor	LogitsProcessor	Abstraction for logits transforms before sampling, argmax-invariant category. vllm/v1/sample/logits_processor/interface.py
Parallel state	parallel_state	Global singleton of TP/PP/DP communication groups. vllm/distributed/parallel_state.py
Group coordinator	GroupCoordinator	ProcessGroup wrapper, holds CPU/device dual groups. vllm/distributed/parallel_state.py
Tensor parallelism	TP	Tensor Parallelism, shards weights along a dim, all-reduce/all-gather during forward.
Pipeline parallelism	PP	Pipeline Parallelism, partitions model layers across GPUs, microbatch pipelining.

Directory conventions

• Source: vllm-project/vllm, main code under vllm/ (Python), CUDA kernels under csrc/.
• v0.25 has switched to the v1 architecture: vllm/v1/ is the main arena, vllm/engine/ is mostly aliases.
• Entry points: vllm/entrypoints/ (llm.py offline, openai/ server, cli/).

Official references

• vLLM official docs
• vLLM design docs
• GitHub repository (tag v0.25.1, Apache-2.0 license)

Source version

This site pins the source version to v0.25.1; line numbers are accurate against that tag. All SrcLink point to github.com/vllm-project/vllm/blob/v0.25.1/<path>#Lxx-Lyy. To bump the version, edit VLLM_TAG in docs/.vitepress/site.ts and run npm run check:refs.