Lead Python Data Engineer - Leading Technology AI Brand

• Senior Engineer/Architect
• Leading Technology AI Brand
• SaaS - Platform based Technology Services
• London/City
• £70-100k salary + equity package

Our client a global technology leader is currently looking for a Senior/Lead Data Engineer to work with the dev team to guide the provision of Software Development for an exciting new AI product.

Key Responsibilities:-

• Architect and build scalable data pipelines and infrastructure
• Design and maintain data ingestion, transformation, and storage architectures for operational and AI workloads.
• Develop and manage batch and Real Time data pipelines.
• Build and optimize systems for vector search, retrieval, and ML data pipelines.
• Ensure data reliability, security, and governance across the platform.
• Collaborate with AI and Back End engineering teams to support training, inference, and product features.
• Implement monitoring, observability, and data quality frameworks.

Core Experience:-

• 7+ years of experience in data engineering or Back End engineering roles.
• Strong experience designing and building data pipelines and distributed data systems.
• Experience working with relational databases (PostgreSQL preferred, but MySQL or similar is acceptable).
• Experience with NoSQL databases.
• Experience with vector databases used in modern AI systems.
• Strong programming experience in Python.

Frameworks/Infrastructure:-

• Apache Spark
• Apache Airflow
• Kafka
• Elasticsearch/OpenSearch