Inference & Performance Engineer
Engineers in this role optimize how AI models train and run in production, focusing on the full stack from GPU kernels and inference runtimes to distributed training systems and cluster orchestration. They bridge research breakthroughs and production reality, writing CUDA/Triton kernels, tuning serving frameworks like vLLM, profiling end-to-end inference pipelines, and solving the performance and efficiency challenges that emerge when deploying transformer models at scale. They typically sit in infrastructure, platform, or systems teams at AI labs and companies, working closely with researchers and product teams to ensure models meet latency, throughput, and cost targets in real deployments.
Measured across 182 of 183 open postings.
This role is advertised at 3 levels, so a single figure for the role would describe none of them. Experience and pay are the midpoints for each level on its own.
| Level | Share | Median years | Median pay |
|---|---|---|---|
| Mid | 35%(63) | 3 | $308k |
| Senior | 23%(41) | 5 | — |
| Staff / Principal | 37%(67) | 7 | $300k |
A dash means too few postings stated it to report a midpoint. Most companies do not publish a salary band, so pay is indicative rather than a market rate. 2 levels with fewer than 10 open postings are not shown.
“You have strong taste and real curiosity about model and agent behavior. You try new models the day they come out and have opinions about where each one shines.”
“4+ years of hands-on work with LLMs in inference, GPU performance, evals, or post-training and RL, in production or research”
“Experience using AI tools (e.g., GitHub Copilot, ChatGPT, Claude) to write, debug, and optimize code”
“decide what 'better' means for a model and how to prove it — then build the infrastructure that makes those proofs reliable”
Requirements are a share of every open posting, so a role missing from this list is one where almost nobody asks. Work mode is different: many postings never say, so that figure counts only the ones that do. A posting stops being advertised when it is filled, cancelled or reorganised, so read the last figure as how long these stay on the market, not as time to hire.
Skills
What companies are looking for in this role.
Systems performance optimization
Distributed systems architecture
Monitoring and observability
Backend and API engineering
ML systems development
Large-scale model training
Cloud infrastructure operations
Hardware and embedded systems
Incident response and reliability
Developer platform engineering
Inference performance optimization
ML platform engineering
AI and GPU infrastructure operations
LLM deployment support
AI evaluation design
Technology
The tools and technologies that define this role.
Open Jobs
183 open Inference & Performance Engineer jobs across 49 companies.
Other Engineering roles
General-purpose software engineering roles focused on building and maintaining software systems. Covers generalist SWE positions that don't clearly fall into frontend, backend, fullstack, or other specialized tracks.
Engineers focused on server-side systems, APIs, services, and data processing pipelines. Includes roles explicitly labeled as backend or server-side development.
Engineers specializing in user-facing interfaces, web applications, and client-side development. Includes UI/UX engineering and web development roles.
Engineers working across the entire application stack, handling both frontend and backend responsibilities.
Engineers building and maintaining internal platforms, cloud infrastructure, compute systems, and developer tooling.