SC Cleared Senior Data Engineer

SC Cleared Senior Data Engineer - Occasional Travel - Up to £540/day Inside IR35

We are seeking an experienced Senior Data Engineer with active SC clearance to support a Central Government department on a mission-critical data and analytics programme. This role suits someone with strong Microsoft Fabric expertise and experience delivering secure, scalable data solutions in complex public sector environments.

The Role
You'll design, build, and operate production-grade data pipelines that underpin reporting, MI, and analytics. Working closely with product, data, and platform teams, you'll help modernise Legacy data estates and embed strong data engineering standards.

Key Responsibilities:

• Build and operate scalable data pipelines using Microsoft Fabric (OneLake/Delta Lake, Data Factory, Synapse Data Engineering).

• Develop batch processing solutions using PySpark, Spark SQL, Python, and SQL, with a focus on performance, resilience, and data quality.

• Support reporting and MI use cases, including transformations and data models feeding Power BI.

• Own CI/CD and version control practices; review code and uphold engineering standards.

• Mentor engineers and provide technical input into design and architecture.

• Work in Agile teams, collaborating via Jira and Confluence.

• Embed security and compliance by design, in line with government data handling policies.

About You

Essential Experience:

• Proven experience as a Senior Data Engineer or equivalent.

• Active SC clearance (BPSS minimum).

• Strong hands-on experience with Microsoft Fabric and Azure data services.

• Advanced skills in PySpark, Spark SQL, Python, and SQL.

• Experience delivering data engineering solutions in government or similarly regulated environments.

• CI/CD, DevOps, and IaC experience (eg Terraform).

• Strong Git/GitLab experience, including code reviews and branching strategies.

• Experience integrating data via APIs and services.

• Clear communication skills and experience working in Agile delivery teams.

Nice to Have:

• Data warehousing and dimensional modelling (eg dbt).

• Basic Power BI knowledge.

• Relevant Microsoft or Azure certifications.

Reasonable Adjustments:

Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built, and we welcome applications from people of all backgrounds and perspectives. Our success is driven by our people, united by the spirit of partnership to deliver the best resourcing solutions for our clients.

If you need any help or adjustments during the recruitment process for any reason, please let us know when you apply or talk to the recruiters directly so we can support you.