The Inference team builds and maintains critical systems that serve AI models to millions of users globally. The role involves managing the entire stack, from intelligent request routing to fleet-wide orchestration across diverse hardware accelerators. Responsibilities include maximizing compute efficiency to support customer growth and providing high-performance infrastructure for research teams. The work requires addressing complex, distributed systems challenges across multiple cloud platforms. Engineers contribute to projects such as designing routing algorithms, building deployment pipelines, and managing multi-region infrastructure.

Candidates must possess significant software engineering experience, with a specific focus on distributed systems. The role requires a minimum of a bachelor's degree or an equivalent combination of education, training, and experience in a relevant field. Applicants should demonstrate a results-oriented mindset and the ability to work flexibly within a team environment. While not strictly required, familiarity with LLM inference optimization, batching strategies, and cloud infrastructure is encouraged.