Want to professionalize your AI skills, pivot to an AI role and increase your salary?
Master AI Engineering with the most practical and comprehensive LLM Development certifications at Towards AI Academy.

Capco

Data Engineer (Databrick + Pyspark)

Capco

Published 16 May 2026
Pune, India
Full Time

Share this job

Role Highlights

Languages used

S3
SQL

Key skills

Data Engineer
Data Governance
Data Lakes
CICD
Data Processing
Data Quality
DataSets
ETL
ELT
Transformation
Serverless
Reliability
Optimization
Cloud
Streaming

Tools, Libraries and Frameworks

DataBricks
Airflow
EMR
Kafka
Apache Spark
PySpark

Description

The role involves designing, developing, and maintaining scalable ETL and ELT data pipelines. The individual will work extensively with large-scale data processing and manage complex workflows. Responsibilities include developing and optimizing data solutions on cloud-based platforms. The position requires writing efficient SQL queries for data transformation and analysis. Additionally, the role entails managing Spark workloads and ensuring the overall quality, reliability, and performance of data pipelines.

Required Qualifications and Skills

The role requires extensive hands-on experience with PySpark, Apache Spark internals, and Databricks, specifically regarding Jobs and Delta Lake. Candidates must demonstrate proficiency in building ETL and ELT pipelines at scale and utilizing Apache Airflow for workflow orchestration. Strong SQL skills and experience with Data Warehouse systems are necessary. Furthermore, the position requires practical experience with cloud data lakes and managing Spark workloads on platforms such as EMR Serverless.

Disclaimer

Disclaimer: Job and company description information and some of the data fields may have been generated via GPT-4 summarisation and could contain inaccuracies. The full external job listing link should always be relied on for authoritative information.

About the company

Capco

Size

6474

Founded

HQ

London, GB

Public/Private

Privately Held

Description

Capco is a leading management consulting firm that focuses on transforming the competitive landscape in financial services through strategy, digital transformation, business consulting, data, and analytics, alongside cyber, cloud, technology architecture, and engineering solutions. They serve a wide range of clients from global banks to emerging FinTechs, offering a broad spectrum of services from business design to technology implementation. As a firm, Capco is characterized by its entrepreneurial spirit, growth mindset, minimal bureaucracy, and its commitment to creating an innovative, collaborative, diverse, and inclusive work environment. The firm takes pride in reshaping financial services to create lasting value for its clients and employees, indicating a strong reliance on technological advancements including Artificial Intelligence to drive transformation.

Share

Share this job

Related jobs

Data Engineer
Data Architect
Integrations
Gaming
Frankfurt am Main, Germany
Full Time
Apache Spark
Data Engineer
Data Governance
Data Lakes
Pune, India
Full Time
Data Management
Financial Services
Product Owner
Change Management
London, UK
Full Time