Senior Data Engineer (DV Cleared)

Posted 3 hours 16 minutes ago by IO Associates

Permanent
Not Specified
Other
Gloucestershire, United Kingdom
Job Description

Senior Data Engineer (DV Cleared)

Fully On-Site | Permanent

Note: Candidates must hold active DV (Developed Vetting) security clearance. This is a fully on-site role.

The Role

We're seeking a detail-oriented, analytical Senior Data Engineer to join a high-performing multi-disciplinary data team. You'll be essential to building and maintaining robust data pipelines and bespoke applications, ensuring data-driven projects deliver against demanding customer timelines and requirements.

Core Responsibilities

Pipeline Engineering:

  • Design, manage, and optimise data ingestion pipelines and orchestration workflows

  • Develop and maintain end-to-end bespoke data projects from requirements to production

  • Monitor system performance and ensure high uptime across Linux-based infrastructure

  • Collaborate closely with data scientists, analysts, product owners, and engineering teams

Technical Delivery:

  • Implement scalable data processing solutions handling complex, high-volume datasets

  • Drive continuous improvement in pipeline reliability, performance, and observability

  • Support cross-functional delivery of mission-critical intelligence platforms

What We're Looking For

Core Technical Expertise:

  • Strong Python proficiency for data engineering and pipeline development

  • Linux systems administration and performance optimisation experience

  • Hands-on with Python orchestration frameworks (Dagster, Apache Airflow)

Cloud & Infrastructure:

  • Serverless AWS experience (Lambda, Step Functions, Athena, Glue)

  • Docker containerisation - building/maintaining production-grade containers

  • Kubernetes familiarity for container orchestration and scaling

Security & Compliance:

  • Knowledge of security, encryption, and compliance standards for secure networks

  • Experience delivering data platforms in high-security environments