vLLM

All

22 repositories

vllm
Public
A high-throughput and memory-efficient inference and serving engine for LLMs
amd cuda inference pytorch transformer llama gpt rocm model-serving tpu
Python
•
Apache License 2.0
•9.9k•57k•1.8k•1.1k•Updated Sep 6, 2025Sep 6, 2025
semantic-router
Public
Intelligent Mixture-of-Models Router for Efficient LLM Inference
python rust golang pii-detection mixture-of-models huggingface-transformers bert-classification vllm huggingface-candle semantic-router
Python
•
Apache License 2.0
•64•759•27•2•Updated Sep 6, 2025Sep 6, 2025
vllm-spyre
Public
Community maintained hardware plugin for vLLM on Spyre
Python
•
Apache License 2.0
•21•32•6•16•Updated Sep 6, 2025Sep 6, 2025
speculators
Public
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
Python
•
Apache License 2.0
•7•40•14•18•Updated Sep 6, 2025Sep 6, 2025
aibrix
Public
Cost-efficient and pluggable Infrastructure components for GenAI inference
Go
•
Apache License 2.0
•451•4.2k•221•20•Updated Sep 6, 2025Sep 6, 2025
vllm-project.github.io
Public
HTML
•24•17•0•3•Updated Sep 6, 2025Sep 6, 2025
production-stack
Public
vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
Python
•
Apache License 2.0
•278•1.8k•75•49•Updated Sep 6, 2025Sep 6, 2025
recipes
Public
Common recipes to run vLLM
Apache License 2.0
•35•123•3•2•Updated Sep 5, 2025Sep 5, 2025
ci-infra
Public
This repo hosts code for vLLM CI & Performance Benchmark infrastructure.
HCL
•36•19•0•10•Updated Sep 5, 2025Sep 5, 2025
llm-compressor
Public
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
sparsity compression quantization
Python
•
Apache License 2.0
•221•1.9k•52•43•Updated Sep 5, 2025Sep 5, 2025
guidellm
Public
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Python
•
Apache License 2.0
•75•558•62•26•Updated Sep 5, 2025Sep 5, 2025
vllm-gaudi
Public
Community maintained hardware plugin for vLLM on Intel Gaudi
Python
•30•8•1•34•Updated Sep 5, 2025Sep 5, 2025
vllm-xpu-kernels
Public
The vLLM XPU kernels for Intel GPU
Python
•
Apache License 2.0
•11•6•0•8•Updated Sep 5, 2025Sep 5, 2025
vllm-ascend
Public
Community maintained hardware plugin for vLLM on Ascend
inference transformer model-serving mlops ascend llm llmops llm-serving vllm
Python
•
Apache License 2.0
•410•1.1k•399•154•Updated Sep 5, 2025Sep 5, 2025
flash-attention
Public
Fast and memory-efficient exact attention
Python
•
BSD 3-Clause "New" or "Revised" License
•2k•92•0•13•Updated Aug 28, 2025Aug 28, 2025
FlashMLA
Public
C++
•
MIT License
•897•5•0•2•Updated Aug 18, 2025Aug 18, 2025
vllm-openvino
Public
Python
•
Apache License 2.0
•7•22•2•0•Updated Aug 18, 2025Aug 18, 2025
rfcs
Public
0•1•0•0•Updated Jun 3, 2025Jun 3, 2025
vllm-project.github.io-static
Public archive
HTML
•
MIT License
•7•8•0•1•Updated Feb 7, 2025Feb 7, 2025
media-kit
Public
vLLM Logo Assets
1•3•0•0•Updated Dec 12, 2024Dec 12, 2024
vllm-nccl
Public archive
Manages vllm-nccl dependency
Python
•
Apache License 2.0
•3•17•2•0•Updated Jun 3, 2024Jun 3, 2024
dashboard
Public
vLLM performance dashboard
Python
•
Apache License 2.0
•7•34•0•0•Updated Apr 26, 2024Apr 26, 2024