Nemotron-Cascade 2: 30B Open MoE, One GPU, Beats 120B
NVIDIA's new Nemotron-Cascade-2-30B-A3B activates just 3B parameters per token, runs on a single RTX 4090, and outscores NVIDIA's own 120B model on coding and math benchmarks.
Articles Tagged "Local AI"
MacBook Neo: Apple's iPhone Chip Lands in a $599 Mac
Apple's cheapest Mac ever packs the A18 Pro iPhone chip with a 16-core Neural Engine - but its 60 GB/s memory bandwidth puts a hard ceiling on what local models you can actually run.
Apple's M5 Pro and Max Make 70B Models Portable
Apple launches M5 Pro and M5 Max MacBook Pros with Neural Accelerators in every GPU core, 128GB unified memory, and 614GB/s bandwidth - enough to run Llama 70B on a laptop.
Someone Reverse-Engineered Apple's Neural Engine and Trained a Model on It
A developer cracked Apple's undocumented ANE private APIs, measured its real throughput at 19 TFLOPS FP16 (not the marketed 38 TOPS), and trained a 109M-parameter transformer on hardware Apple designed exclusively for inference.
Mac Studio Clusters Now Run Trillion-Parameter Models for $40K
macOS RDMA over Thunderbolt 5 has turned four Mac Studios into a 1.5TB unified memory cluster that runs Kimi K2 at 25 tokens per second - a setup that would cost $780K with NVIDIA H100s.
Apple M4 Max - Unified Memory for Large Models
Full specs and benchmarks for the Apple M4 Max SoC - up to 128GB unified memory at 546 GB/s, 3nm process, and why it has become the quiet favorite for running 70B+ models locally.
NVIDIA RTX 3090 - The Budget 24GB Value King
Full specs and benchmarks for the NVIDIA GeForce RTX 3090 - 24GB GDDR6X at 936 GB/s, Ampere architecture, and why used 3090s remain the best value option for local AI inference in 2026.
NVIDIA RTX 4090 - The Home Lab AI Standard
Full specs and benchmarks for the NVIDIA GeForce RTX 4090 - 24GB GDDR6X, 1,008 GB/s bandwidth, Ada Lovelace architecture, and why it remains the default home lab GPU for local AI inference.
NVIDIA RTX 5090 - Blackwell for the Home Lab
Full specs and benchmarks for the NVIDIA GeForce RTX 5090 - 32GB GDDR7, 1,792 GB/s bandwidth, Blackwell architecture, and what it means for local AI inference.
AORUS RTX 5090 AI BOX vs NVIDIA DGX Spark for Local AI
Two very different approaches to desktop AI hardware - a 32 GB eGPU with 1,792 GB/s bandwidth versus a 128 GB unified memory mini PC with full CUDA. Which one should you buy?
AORUS RTX 5090 AI BOX Review: Desktop GPU Power via Thunderbolt 5
A review of the Gigabyte AORUS RTX 5090 AI BOX - a liquid-cooled eGPU packing a full desktop RTX 5090 with 32 GB GDDR7, connecting to any laptop over Thunderbolt 5 for $2,999.
DGX Spark Review: NVIDIA's $4,699 Desktop AI Box
A hands-on review of the NVIDIA DGX Spark - a 128 GB Grace Blackwell mini PC that promises 1 petaflop of AI performance on your desk for $4,699.
