Hailo-10H - Edge AI With On-Device LLMs
Complete specs, benchmarks, and analysis of the Hailo-10H - a 2.5W edge AI accelerator with 40 TOPS INT4, on-module LPDDR4, and the ability to run LLMs and VLMs on a Raspberry Pi at 10 tokens per second.
Articles Tagged "Inference"
NVIDIA Rubin CPX - Inference GPU With GDDR7
Full specs, benchmarks, and analysis of the NVIDIA Rubin CPX - a purpose-built inference GPU with 128GB GDDR7, 30 PFLOPS NVFP4, and 3x faster attention versus Blackwell, targeting million-token context workloads.
Mac Studio Clusters Now Run Trillion-Parameter Models for $40K
macOS RDMA over Thunderbolt 5 has turned four Mac Studios into a 1.5TB unified memory cluster that runs Kimi K2 at 25 tokens per second - a setup that would cost $780K with NVIDIA H100s.
Intel Crescent Island - Inference-Only AI GPU
Intel Crescent Island specs and analysis - an Xe3P inference GPU with 160GB LPDDR5X, air cooling, and a cost-optimized approach to AI serving.
NVIDIA Rubin R200 - Next-Gen AI Superchip
Complete specs, benchmarks, and analysis of the NVIDIA Rubin R200 GPU - the post-Blackwell flagship with 288GB HBM4, 22 TB/s bandwidth, and 50 PFLOPS FP4.
Qualcomm AI200 - Rack-Scale Inference ASIC
Qualcomm AI200 specs and analysis - a Hexagon-based inference accelerator with 768GB LPDDR per card, rack-scale design, and a focus on inference TCO.
SambaNova SN50 RDU - Agentic Inference Chip
Complete specs and analysis of SambaNova's SN50 RDU - a TSMC 3nm dataflow chip with 3.2 PFLOPS FP8, three-tier memory, and claimed 5x speed over NVIDIA B200.
Apple M4 Max - Unified Memory for Large Models
Full specs and benchmarks for the Apple M4 Max SoC - up to 128GB unified memory at 546 GB/s, 3nm process, and why it has become the quiet favorite for running 70B+ models locally.
Cambricon MLU590 - China's Inference Accelerator
Full specs and analysis of the Cambricon MLU590 - 192GB HBM2e, ~2,400 GB/s bandwidth, TSMC 7nm, and what it means for AI inference outside the NVIDIA ecosystem.
Google TPU v7 Ironwood
Google TPU v7 Ironwood specs, architecture, and performance estimates. Google's next-gen inference-optimized TPU with massive memory per chip, announced at Cloud Next 2025.
Groq LPU - Deterministic Inference at Scale
Groq's Language Processing Unit (LPU) is a purpose-built inference ASIC that trades HBM for 230MB of on-chip SRAM, delivering deterministic latency and record-breaking tokens-per-second for LLM serving.
Intel Gaudi 3 - Challenging NVIDIA on Price
Intel Gaudi 3 is a TSMC 5nm AI accelerator with 128GB HBM2e and 1,835 TFLOPS FP8 performance, positioned as a cost-effective alternative to NVIDIA H100 for training and inference workloads.
