ML Compiler Engineer Job Description

An ML Compiler Engineer writes the software that turns a model defined in PyTorch, JAX or TensorFlow into instructions a particular chip can run quickly. The input is a graph of tensor operations: matrix multiplies, attention, normalizations, elementwise math. The output is a set of kernels and a schedule tuned for one target, whether that is an NVIDIA or AMD GPU, a Google TPU, an AWS Trainium chip or a startup's custom accelerator. Everything in between is the compiler, and this engineer owns some slice of it.

The lowering pipeline

The center of the job is lowering. A framework hands over a graph, and the compiler moves it down through a stack of intermediate representations, each one closer to the hardware. MLIR, the multi-level IR framework maintained inside the LLVM project, underpins several ML compiler stacks. The MLIR project's own list of users names OpenXLA, whose XLA, StableHLO and Shardy projects use MLIR to represent and transform models from frameworks such as JAX, TensorFlow and PyTorch; IREE, an end-to-end MLIR-based compiler and runtime for machine-learning models; and Hexagon-MLIR, an open-source compiler stack for Qualcomm Hexagon NPUs that lowers Triton kernels and PyTorch models through Hexagon-specific dialects.

What a working week looks like

The tasks are concrete. One day an engineer writes a pass that folds a bias add and activation into the preceding matrix multiply, so the intermediate tensor never leaves on-chip memory. Another day goes to tile-size selection for an attention kernel, choosing block shapes that fit shared memory or a scratchpad and keep the matrix units busy. Another goes to a numerical bug: a model that matches its reference in fp32 drifts after a pass reorders a reduction in bf16, and the fix means bisecting IR dumps until the offending rewrite turns up.

Kernel languages and autotuners

Kernel languages sit next to the compiler proper. Triton describes itself as a language and compiler for parallel programming, with a Python-based environment for writing custom compute kernels for GPUs. NVIDIA describes cuTile as a parallel programming model for NVIDIA GPUs and a Python-based DSL, and its cuTile Python package generates kernels based on Tile IR. An ML compiler engineer may build the code generator behind a language like these, write the autotuner that searches its configuration space, or connect it to a framework so model authors never call it directly.

Who they work with

The role is collaborative by design. Compiler engineers talk to hardware architects about what an instruction set exposes and what each feature costs, to framework teams about graph capture and dynamic shapes, and to model teams about which operators matter for the next release. On a chip company's team the compiler is how customers' models reach the new silicon at all. At a cloud provider or AI lab, the same skills are judged on throughput, memory footprint and serving cost.

Good results show up on a benchmark dashboard, but they rest on correctness. A fusion that speeds up a kernel and silently changes its output is a regression, so reference implementations, tolerance checks and regression suites are as much a part of the job as the optimizations.

Education. As a rough guide, the path starts with a bachelor's degree in computer engineering, computer science or electrical engineering, with coursework in compilers, computer architecture and operating systems. A master's or PhD focused on compilers, programming languages or architecture is a further option, particularly for research-leaning teams, while production teams weigh shipped work. For the broader occupation, BLS says software developers typically need a bachelor's degree in computer and information technology or a related field, and that some employers prefer to hire developers who have a master's degree.

Experience. As a rough guide, people move into ML compiler roles after two to five years of compiler, GPU programming or systems performance work, and senior or staff roles expect ownership of a full backend or a major subsystem such as autotuning or graph capture. One entry route is a general compiler job, for example on an LLVM backend, followed by a move to an ML team.

Compiler fundamentals. Expect to be tested on:

• SSA form, dataflow analysis, dominance and the pass pipeline model
• Loop transformations: tiling, interchange, unrolling, software pipelining and vectorization
• Instruction selection, register allocation and scheduling, including for VLIW or SIMD targets
• MLIR concepts: dialects, operation definitions in ODS and TableGen, pattern rewriting, conversion passes and progressive lowering

ML systems knowledge. Compiler work is easier when the engineer understands the workloads:

• How attention, convolution and mixture-of-experts layers map onto matrix units and memory
• Precision formats such as bf16, fp8 and integer quantization, and where each loses accuracy
• Graph capture in torch.compile (TorchDynamo, AOTAutograd, TorchInductor), JAX tracing and StableHLO
• The difference between training and inference compilation, including dynamic shapes and batching

Hardware. GPU execution model, shared memory and caches, tensor cores or equivalent matrix engines, DMA engines and scratchpads on custom accelerators, and enough interconnect knowledge to reason about multi-chip sharding.

Languages and tools. C++ is the working language of LLVM and MLIR, and Python drives framework integration, test harnesses and autotuning scripts. CUDA, HIP or a vendor kernel language helps for writing reference kernels. Build systems such as CMake and Bazel, plus profilers like Nsight Compute and Nsight Systems, round out the kit.

Credentials. Evidence of skill is the code itself: merged patches to LLVM, MLIR, Triton or OpenXLA, a published benchmark study, or a backend brought up on real hardware.

Interview signals. Strong candidates can walk through one optimization end to end: the IR before and after, the legality conditions, how correctness was tested and what the profiler showed. They can also explain a design tradeoff in a compiler they have used, such as when to fuse and when fusion hurts.

BLS does not publish counts or projections for ML compiler engineers. The closest official signal is the broader group of software developers, quality assurance analysts and testers. In its Occupational Outlook Handbook, BLS counts 1,905,400 jobs in that group in 2025 and projects employment to grow 10% from 2025 to 2035, which it labels much faster than average, an employment change of 185,400. That is direction for a very large workforce, not a forecast for this specialty.

More hardware, more backends

The more useful signal is the number of chips that need a compiler. Each new accelerator needs a path from PyTorch and JAX to its instruction set, and that path is built and maintained by compiler engineers. Recent examples are concrete. On Oct. 20, 2026, AWS launched the public beta of TorchNeuron, an open source native PyTorch backend for its Trainium chips that keeps compatibility with eager execution, torch.compile and standard PyTorch APIs, and integrates with the Neuron Kernel Interface (NKI) for custom operators. At PyTorch Conference North America 2026, held Oct. 20 to 21 in San Jose, the program listed a Meta session on Helion, PyTorch's high-level kernel-authoring DSL, gaining two new compiler backends, CuteDSL for recent NVIDIA GPUs and Pallas for TPUs, so one kernel source can target very different hardware.

Compiler teams sit at several kinds of employer: chip designers that must ship a software stack with their silicon, cloud providers with in-house chips, AI labs that tune training and serving on several hardware generations, and the open-source framework projects that tie it together. Each group has its own mix of research and production work.

Where AI tools fit

The same conference program described its kernel engineering track as spanning the DSLs, autotuners, and hand- and agent-written kernels that squeeze more performance out of every GPU cycle, so machine-written kernels sat on the program next to hand-written ones. Where generated code enters a stack, someone still has to build the verification, autotuning and integration layers that decide whether it is correct and fast enough to ship, and that is compiler work.

How to stay employable

The durable skills are the fundamentals: IR design, dependence analysis, memory hierarchy, numerical behavior of low-precision formats and disciplined performance measurement. On top of that, depth in at least one production stack (TorchInductor and Triton, XLA, IREE or a vendor compiler) and experience bringing a model up on more than one kind of hardware make a candidate portable across employers. Watching the LLVM and PyTorch communities, through their developer meetings, forums and conference talks, is a practical way to see which abstractions teams are adopting.

Where the work leads

Senior compiler engineers can grow into roles that set compiler architecture across a product line, move toward hardware architecture, where knowing what a compiler can exploit shapes the next chip, or take on framework and runtime leadership. Each path draws on the same mix of IR design, performance analysis and cross-team negotiation.

Dear Hiring Manager,

I am applying for the ML Compiler Engineer role on your accelerator software team. For the past four years I have worked on the compiler for an edge inference chip at Northfield Systems, where I own the MLIR lowering path from our Linalg-level representation down to the vector unit and DMA engine.

The project I am proudest of started as a customer complaint. A speech model ran correctly but slowly on our part, and the profile showed the vector unit idle while the DMA engine shuttled the same activation tiles in and out of the scratchpad. I rewrote our tiling pass to choose tile shapes from a cost model of scratchpad capacity and DMA latency instead of fixed defaults, then fused the following normalization so the tile stayed resident. The model hit the customer's latency target, and the pass now handles every convolution and matrix-multiply layer we ship.

The harder lesson came a month later, when the same fusion broke numerical parity on a quantized vision model. Tracking it down meant bisecting pass by pass and comparing every intermediate tensor against a reference run. The fix was small, but I came out of it convinced that test infrastructure is part of the compiler, not an afterthought, and I built the per-pass parity harness our team now runs on every change.

Before Northfield I spent two years on a GPU driver team writing CUDA reference kernels and learning to read hardware counters, which still shapes how I debug.

Your team's public work on retargeting a shared IR across several chip generations is exactly the problem I want to spend the next few years on. I would welcome the chance to talk through how my backend experience could help with your next bring-up.

Sincerely, Jordan Reyes