Loading listing…
serving-llms-vllm
Security scan passedServes LLMs with high throughput using vLLM's PagedAttention and continuous batching.
Part of AI Research Skills (Orchestra)Pack · 98 skills
by Orchestra-Research· AI & agentsUnclaimed · Yours? Claim it
What serving-llms-vllm does
About
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
Install serving-llms-vllm
More from AI & agents
Skills people install alongside serving-llms-vllm.
