vLLM

Other Free

High-throughput inference engine for LLM serving.

About vLLM

vLLM optimizes LLM serving performance and memory for production deployment.