Data Engineering Tech Lead (Databricks/Python)

Roles & Responsibilities

• Lead and mentor a high-performing data engineering team, providing hands-on technical guidance across Databricks (DLT), PySpark, Python, SQL, and Azure Data Factory.

• Architect, build, and optimize scalable data pipelines and platforms to support high-volume trading and analytics workloads.

• Drive performance tuning, cost optimization, and best-practice engineering standards within Databricks and cloud data environments.

• Own end-to-end technical delivery-ensuring reliability, quality, and timely execution of data projects in a fast-paced trading context.

• Act as the technical SME for modern data engineering, contributing to data strategy, solution design, and platform enhancements.

• Collaborate closely with trading, product, and platform teams to translate business requirements into robust data solutions.

• Promote engineering excellence through code reviews, automation, DevOps practices, and continuous improvement initiatives.

• Foster a culture of knowledge sharing through community activities, training, and thought leadership within the data engineering domain.

• Ensure smooth execution of change management, incident management, and problem resolution processes.

Mandatory Skills

• Cloud Platforms: AWS/Azure/SAP - Master

• ELT Pipelines:Master

• Data Modelling:Master

• Data Integration & Ingestion:Master

• Data Manipulation & Processing:Skilled

• DevOps & Version Control: GitHub, GitHub Actions, Azure DevOps - Skilled

• Data Engineering Tools: Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest - Skilled

Optional Skills

• Experience in project management or leading a Scrum team

• Experience working with BPC/Planning

• Exposure to external technical partner ecosystems

• Experience managing small teams

• Familiarity with MKDocs documentation