ML Compiler Engineer

ML Compiler Engineers build the bridge between the neural network a researcher writes in Python and the stream of instructions that actually executes on silicon. The job exists because the gap between what ML frameworks express (high-level tensor operations on abstract shapes) and what hardware executes efficiently (tightly scheduled, memory-layout-aware, vectorized instruction sequences) is enormous — and closing it requires the kind of compiler engineering that performance can't live without.

On any given day, the work might involve writing a new MLIR lowering pass that fuses a layer normalization into the preceding matrix multiply to avoid a round-trip through global memory, debugging why a model that runs correctly at fp32 produces NaN values after fp16 quantization is applied during a fusion pass, or profiling a transformer inference path on a new GPU generation and tracing the performance gap to a suboptimal tile size in the attention kernel.

The production stakes are significant. A 10% improvement in inference throughput for a model serving hundreds of millions of requests per day translates directly to infrastructure cost and latency SLAs. Compiler engineers in these environments get precise feedback on the value of their work — FLOP/s utilization, memory bandwidth efficiency, and tokens-per-second numbers don't lie.

The hardware landscape is what makes the role both demanding and interesting. NVIDIA's GPU architecture evolves every 18–24 months, AMD is competing seriously in the data center, Google continues developing TPU generations, and a wave of custom AI accelerators from Amazon (Trainium/Inferentia), Microsoft (Maia), Apple (Neural Engine), Meta (MTIA), and dozens of startups are all demanding compiler backends. Each new target has a distinct ISA, memory hierarchy, and performance bottleneck profile. An ML Compiler Engineer who can bring up a new hardware backend — write the instruction selection, memory planning, and autotuning infrastructure from scratch — is extraordinarily valuable.

Collaboration is deeper than in many software roles. Compiler engineers work with hardware architects before a chip tapes out, giving feedback on ISA features that will or won't be compilable efficiently. They work with ML researchers to understand which operator patterns are performance-critical. They work with framework engineers to ensure that torch.compile or JAX's JIT machinery feeds the compiler the right representation. The role requires genuine breadth: you need enough computer architecture to understand why a memory access pattern is cache-hostile, enough ML to understand why a model author wrote the operation the way they did, and enough compiler theory to express the transformation that fixes both problems.

Education:

• Bachelor's degree in computer science, computer engineering, or electrical engineering (minimum for most roles)
• Master's or PhD in compilers, computer architecture, or programming languages is common, particularly at research-oriented teams at Google Brain, Meta FAIR, and academic-adjacent labs
• PhD is not required at most production compiler teams — strong systems engineering experience frequently substitutes

Core compiler knowledge:

• IR design: SSA form, dataflow analysis, def-use chains, dominance trees
• Loop optimization: tiling, loop interchange, unrolling, vectorization, polyhedral models
• Register allocation, instruction scheduling, and code generation for SIMD/VLIW architectures
• MLIR: dialect design, conversion passes, pattern rewriting, progressive lowering
• LLVM: writing custom passes, using the pass manager, targeting custom backends via TableGen

ML framework familiarity:

• PyTorch internals: torch.compile, TorchDynamo graph capture, AOTAutograd, TorchInductor
• JAX tracing model, XLA HLO representation, StableHLO dialect
• ONNX graph format and opset evolution for inference deployment
• TVM / Apache TVM: Relax IR, TIR, MetaSchedule autotuning

Hardware and systems:

• GPU architecture: CUDA programming model, warp execution, shared memory, tensor core utilization
• Profiling tools: NVIDIA Nsight Compute, AMD ROCProf, Google TPU profiler, vendor memory bandwidth analysis tools
• Understanding of DMA engines, scratchpad memory, and prefetch behavior on custom accelerators
• Experience with at least one non-GPU target (TPU, NPU, FPGA-backed accelerator) is a strong differentiator

Languages:

• C++ (required; most compiler infrastructure is C++17/20)
• Python (required for framework integration, test harnesses, and autotuning)
• CUDA C or HIP for kernel-level verification work
• Familiarity with TableGen, MLIR's op definition DSL, and build systems (Bazel, CMake)

What strong candidates show in interviews:

• The ability to walk through a specific optimization they wrote: what the before/after IR looked like, what the correctness constraint was, and what the measured speedup was
• Familiarity with at least one real production compiler stack well enough to have an opinion about its design tradeoffs
• Comfort reading hardware performance counter data and mapping it to compiler decisions

The ML Compiler Engineer role is one of the fastest-growing specializations in software engineering, driven by a confluence of hardware diversification, model scale, and production deployment pressure that shows no sign of reversing.

Hardware proliferation is the primary driver. For most of software history, compilers targeted a small number of dominant ISAs — x86, ARM, MIPS. ML inference and training workloads are now running on NVIDIA GPUs, AMD GPUs, Google TPUs, Apple Neural Engines, Amazon Trainium, Microsoft Maia, Meta MTIA, and a dozen well-funded custom ASIC startups. Each target requires a backend. Each backend requires engineers who can write it, tune it, and maintain it across hardware generations. The number of distinct hardware targets requiring compiler support has roughly tripled in five years, and the pipeline of new silicon continues.

Model architecture changes faster than hardware. The transformer architecture that dominated 2020–2023 has spawned variants — MoE models, state space models (Mamba, RWKV), diffusion transformers, multi-modal architectures with heterogeneous compute graphs — each with distinct compilation challenges. Dynamic shapes, variable sequence lengths, speculative decoding, and disaggregated serving topologies have all introduced compiler problems that didn't exist at scale in prior generations. The workload keeps changing, which means the compiler work keeps changing.

Demand significantly outpaces supply. A compiler engineer with MLIR expertise, GPU profiling skills, and production ML framework knowledge is a genuinely rare combination. Most university programs produce software engineers with either compiler theory or ML systems experience — few with both at production depth. The gap between how many such engineers the industry needs and how many exist is large, and compensation reflects it.

Near-term headcount growth is concentrated at hyperscalers (Google, Meta, Microsoft, Amazon, Apple), leading AI labs (OpenAI, Anthropic, xAI, Mistral), and AI accelerator companies (NVIDIA, AMD, Intel Habana, Groq, Tenstorrent, Cerebras, Axonn, d-Matrix). Defense AI programs (DARPA, DoD, national labs) are also actively hiring for classified and unclassified ML compiler work.

The five-year outlook is strong. The proliferation of edge AI deployment — inference on phones, cars, embedded devices — extends the demand beyond data center hardware into the broader embedded and mobile space. An ML Compiler Engineer who maintains currency with MLIR, learns to work on two or three hardware targets, and develops genuine production debugging skills is not facing displacement pressure from AI automation — they are in the middle of the infrastructure that makes AI automation possible.

Dear Hiring Manager,

I'm applying for the ML Compiler Engineer position at [Company]. I've spent the past four years building and tuning compiler infrastructure for ML inference workloads, most recently at [Company] where I own the TorchInductor backend for our custom accelerator.

The specific work I'm most proud of is a loop tiling and fusion pass I wrote for attention kernels on our in-house NPU. The hardware has a 512KB scratchpad and no L2 cache — naive attention implementations spent 60% of cycles on DRAM traffic. I built a multi-level tiling strategy in MLIR that tiles the Q/K/V matrices to fit the scratchpad, fuses the softmax normalization into the output accumulation loop, and lowers to our DMA engine's async prefetch API. End-to-end, that pass improved throughput on a 7B parameter model's attention layers by 2.4x. The harder part was getting it to handle dynamic sequence lengths correctly — we use speculative decoding in production, and the draft model generates variable-length batches that the original pass assumed were statically shaped.

Before that work I spent two years on the TVM team at [Previous Company], writing MetaSchedule tuning rules for conv2d and depthwise convolution on ARM Mali GPUs. That experience gave me a strong foundation in autotuning search space design, which I've applied to tile-size selection on the NPU backend.

I've been watching [Company]'s public MLIR work closely — particularly the work on the Linalg dialect and the structured op abstraction. I have opinions about where the fusion heuristics could be stronger, and I'd welcome the chance to talk through them.

[Your Name]