Engineering
Why Is PyTorch Compile So Fast: Kernel Fusion
PyTorch’s compiler achieves significant speedups through kernel fusion, a technique that combines dependent operations into single, efficient kernels. By fusing…
Engineering
Understanding PyTorch’s Test Infrastructure
PyTorch’s test infrastructure dynamically generates tests across devices and data types at import time, which is why CI test failures…
Engineering
Adding a GPU Without Building One
VKAE is an inference acceleration technology that optimizes GPU performance through software-level kernel improvements rather than hardware changes. The system…
Engineering
DiffusionGemma: 4x Faster Text Generation
Google released DiffusionGemma, a 26B mixture-of-experts model that generates entire blocks of text simultaneously rather than processing tokens sequentially like…
Engineering
Core dump epidemiology: fixing an 18-year-old bug
OpenAI engineers used population-level analysis of core dumps rather than case-by-case inspection to separate a mysterious recurring crash in its…
Engineering
Serving DeepSeek-V4 on GB300 with SGLang: 5x Higher Throughput at the Same Interactivity Since Day-0
The SGLang team and NVIDIA achieved a 5x improvement in DeepSeek-V4 throughput on NVIDIA GB300 GPUs since the model’s Day-0…
Engineering
TokenSpeed-Kernel: Portable APIs and High-Performance Kernels for Multi-Silicon LLM Inference
TokenSpeed-Kernel is an open-source subsystem that simplifies LLM inference backend complexity through a layered API and registry system separating high-level…
Engineering
Miles: A PyTorch-Native Stack for Large-Scale LLM RL Post-Training
Miles is an open-source framework from RadixArk built to address the distributed-systems challenges of large-scale reinforcement learning post-training for LLMs.…
Engineering
Bringing PyTorch Monarch to AMD GPUs: Single-Controller Distributed Training on ROCm
AMD and Meta ported PyTorch Monarch, a distributed training framework, to AMD Instinct GPUs via ROCm, enabling fault-tolerant distributed training…
Engineering
Introducing Cross-Repository CI Relay: Scalable CI for PyTorch’s Out-of-Tree Backends
PyTorch introduced Cross-Repository CI Relay (CRCR), a system that automatically triggers continuous integration in downstream repositories whenever pull requests or…
Engineering
Securing the future of AI agents
Google DeepMind introduced an AI Control Roadmap, a security framework for managing increasingly capable AI agents deployed internally by treating…
Engineering
Does Your LLM Know When It’s About to Be Wrong?
This article introduces a metacognition benchmark and leaderboard designed to measure whether large language models can recognize their own errors…