Want to professionalize your AI skills, pivot to an AI role and increase your salary?
Master AI Engineering with the most practical and comprehensive LLM Development certifications at Towards AI Academy.

Google

Staff Software Engineer, GPU Performance

Google

Published 16 May 2026
New York, NY, USA& Other locations
207K - 300K USD Annual
Full Time
27500 - 275000 people

Share this job

Role Highlights

Languages used

CUDA
Triton

Key skills

Machine Learning
Computer Science
Software Design
Data Structures
Technical Leadership
LLMs
Testing
Architecture
Deployment
PhD
Optimization
Infrastructure
Cloud
Reliability
Operations
Research

Tools, Libraries and Frameworks

Vertex AI
GCP
GPUs
XLA
Core ML

Description

This role involves identifying and maintaining benchmarks for large language model training and serving to uncover performance opportunities. The engineer will engage with internal teams to resolve complex machine learning model performance issues. Responsibilities include running architecture-level simulations on hardware designs and performing roofline analysis to guide partner teams. The position requires analyzing efficiency metrics to identify bottlenecks and implementing solutions at a fleet-wide scale. Additionally, the role involves running performance benchmarks on hardware using various internal and external tools.

Required Qualifications and Skills

Candidates must possess a Bachelor's degree or equivalent practical experience. The role requires eight years of experience in software development, including five years in testing and launching products and three years in software design and architecture. Applicants need expertise in GPU architectures, memory hierarchies, performance engineering, and low-level GPU programming. A Master's degree or PhD in a technical field is preferred, along with experience in technical leadership and cross-functional project management.

Disclaimer

Disclaimer: Job and company description information and some of the data fields may have been generated via GPT-4 summarisation and could contain inaccuracies. The full external job listing link should always be relied on for authoritative information.

About the company

Google

Size

275773

Website

goo.gle

HQ

Mountain View, US

Public/Private

Public Company

Description

At gTech's Users and Products team, innovations are focused on enhancing user engagement and solving complex customer needs through technical expertise and a deep understanding of Google's and Alphabet's broad product environments. The team acts as a bridge between Google’s users and its product teams, ensuring that user insights are integrated into product offerings which support numerous annual product launches. The newly formed Machine Learning Data Operations (MLDO) team within gUP Operations plays a critical role in delivering and tuning machine learning and GenAI data operations across Google’s product suite, leveraging extensive global vendor networks. Google's overarching goal is to create impactful products and services, with gTech playing a strategic role in bringing these solutions to fruition through technological and operational expertise.

Share

Share this job

Related jobs

Tech Lead
Computer Science
API
Product Development
Mountain View, CA, USA
Full Time
AI
NLP
Reinforcement Learning
Information Retrieval
Mountain View, CA, USA
Full Time
Prompt Engineering
API
Product Development
Program Management
Addison, IL, USA
Full Time