Technical Lead - Security Cleared - Outside IR35

We're looking for a hands-on Tech Lead to lead the design and delivery of a complex, secure, and scalable AWS data platform within a public services client. This is a senior leadership role requiring not just strong technical capability, but the confidence and experience to lead delivery across a highly regulated programme with complex internal and external stakeholders.

You'll act as the technical authority for the platform, guiding a small multi-disciplinary team while working closely with client architects, security leads, product owners, and delivery leadership to ensure solutions are robust, compliant, and delivered incrementally to production.

This role suits someone who is technically deep, delivery-focused, and comfortable owning decisions in challenging environments.

Key Responsibilities

Technical Leadership & Delivery

• Provide clear technical leadership across data engineering, analytics engineering, and DevOps.

• Own end-to-end technical design and delivery of the AWS data platform.

• Translate complex, high-level requirements into pragmatic architectures, delivery plans, and well-scoped work packages.

• Take accountability for technical decisions, trade-offs, and delivery outcomes.

• Lead technical risk management in a complex, regulated delivery environment.

Platform & Data Architecture

• Design and govern data ingestion, transformation, storage, and access patterns using:

  • Amazon S3 (data lake)

  • AWS Glue (ETL, PySpark, Crawlers, orchestration)

  • Amazon Redshift (data warehousing and analytics)

• Drive data modelling best practices (star/snowflake schemas, performance optimisation).

• Optimise Redshift performance (distribution/sort keys, WLM, concurrency scaling, Spectrum where appropriate).

• Design reliable, cost-optimised ETL/ELT pipelines using Glue, Lambda, Step Functions, and event-driven patterns.

Stakeholder Management & Communication

• Act as a senior technical interface with client stakeholders, architects, and delivery leadership.

• Clearly articulate technical decisions, risks, and dependencies to both technical and non-technical audiences.

• Build trust and credibility in a challenging stakeholder landscape.

• Provide regular technical reporting and progress updates.

Required Skills & Experience

• Proven experience as a Tech Lead or Lead Data Engineer delivering AWS-based data platforms.

• Strong hands-on experience with:

  • Amazon S3 (data lake design, partitioning, life cycle management)

  • AWS Glue (PySpark jobs, crawlers, catalog, orchestration)

  • Amazon Redshift (performance tuning, WLM, Spectrum)

• Excellent Python (PySpark) and advanced SQL skills.

• Strong experience with CI/CD and Infrastructure as Code (Terraform or CloudFormation).

• Deep understanding of security and governance in regulated or public sector environments.

• Demonstrable experience leading delivery in complex programmes with multiple stakeholders.

• Strong communication skills with the ability to influence and lead at senior levels.

• Comfortable working in Agile delivery environments.