GPT-4 to Self-Hosted Llama 4 Migration Guide
Migrate from GPT-4o (now retired) or GPT-5.1 to self-hosted Llama 4 with near-zero code changes, but plan carefully for hardware, EU licensing, and realistic context window limits.
Articles Tagged "VLLM"
NVIDIA Dynamo Snapshot Slashes Kubernetes AI Cold Starts
NVIDIA's Dynamo Snapshot uses CRIU and cuda-checkpoint to freeze and restore GPU inference containers in seconds, cutting Kubernetes cold-start times by up to 21x for large models.
Best Open-Source LLM Inference Servers 2026
A benchmark-driven comparison of the top open-source LLM inference servers - vLLM, SGLang, TGI, llama.cpp, TensorRT-LLM, LMDeploy, and more.
llm-d Joins CNCF - Kubernetes Gets a Native LLM Inference Stack
IBM Research, Red Hat, and Google Cloud donated llm-d to the CNCF at KubeCon EU, giving Kubernetes a production-grade distributed LLM inference framework built on vLLM.
IBM Granite 4.0 1B Speech Tops OpenASR Leaderboard
IBM's new 1B-parameter speech model claims the top spot on the Open ASR Leaderboard while running on consumer hardware, beating Whisper Large V3 by 25% on word error rate.
vLLM 0.17 Ships FlashAttention 4 and Live MoE Scaling
vLLM v0.17.0 adds FlashAttention 4, elastic expert parallelism for live MoE rescaling, full Qwen3.5 support, and a performance-mode flag, all in 699 commits from 272 contributors.
Qwen 3.5 FP8 Weights Drop - How to Actually Deploy a 397B Model on 8 GPUs
Alibaba releases official FP8-quantized weights for the Qwen 3.5 flagship and 27B dense model, cutting memory requirements roughly in half and enabling deployment on 8x H100 GPUs with native vLLM and SGLang support.
