Yuhao Ge

I am an LLM serving and performance engineer with cross-accelerator expertise (TPU, Trainium, GPU). My work spans framework, compiler, and kernel optimization for large-scale serving systems.

At Google, I built vLLM-TorchTPU from the ground up and grew the team to over 50 people. I work closely with key customers and collaborate with Inferact and RadixArk on production adoption and optimization. My work spans initial design and bring-up, Pallas kernels, compiler/runtime co-design, and distributed serving features such as parallelism, quantization, and scheduling, together with model bring-up and performance hill climbing for production and SemiAnalysis AgentX submissions. I also co-develop TorchTPU and SGLang-TorchTPU as part of this open-source serving stack.

Previously, at AWS Annapurna Labs, I was a founding engineer of NKI, co-creating its domain-specific language and compiler for AWS Trainium chips. My work also included compiler optimizations, FlashAttention kernels, and autotuning infrastructure.

I earned an M.S. in Computer Science and a B.S. in Computer Engineering from UIUC, graduating with Highest Honors and Bronze Tablet recognition. Working with Prof. Charith Mendis, I co-authored SPLAT, a GPU code-generation framework for sparse attention published at OOPSLA 2025.

Earlier, I worked on ML-driven FPGA accelerator optimization at the UCLA VAST Lab with Prof. Jason Cong, and on real-time avatar animation systems at TikTok.