The role involves designing, building, and maintaining efficient, scalable ETL and ELT pipelines across batch, file-based, and streaming architectures. The individual will ensure data quality, reliability, and timeliness by following established engineering practices. Responsibilities include modeling and storing large datasets using modern file and table formats. The position requires deploying pipelines on cloud infrastructure and collaborating with data analysts and scientists to deliver datasets that support their work. Additionally, the role involves working with security and governance teams to ensure data compliance and discoverability, while contributing to the development of a Generative AI framework.

Candidates are required to have at least four years of experience building ETL pipelines for data lakes or warehouses. Technical expertise in Python, Spark, SQL, and Airflow is necessary for the role. Applicants must possess knowledge of data warehousing concepts, analytical data modeling, and pipeline reliability practices. Familiarity with big data file types and table formats such as Hive, Iceberg, Parquet, Avro, and JSON is required, along with a background in building data platforms on cloud infrastructure.