Senior Data Engineer

About Scrumconnect
Scrumconnect is a leading force in technology consultancy, proudly contributing to over 20% of the UK's most significant citizen-facing public services. Our award-winning team has made a substantial impact, delivering more than 64 services in the past two years alone. This work has not only reached over 50 million citizens but also achieved considerable savings for the taxpayer, amounting to over £25 million.

At Scrumconnect, we foster a community of talented consultants who thrive on collaboration, sharing knowledge, and continuous learning to address and solve complex challenges. Our mission is to combine advanced software engineering, human-focused design, and data-driven insights to deliver unparalleled service to our clients.

Role Overview

We are seeking an experienced Senior Data Engineer to develop, maintain, and optimise data products and a Strategic Data Platform. You will be part of multi-functional Agile delivery teams, ensuring operational stability, ongoing support, and enhancement of scalable data solutions. The role requires strong technical expertise in Azure data tools, SQL, Python, and modern engineering practices, with a focus on migration, data governance, and advanced analytics capabilities.

Key Responsibilities

• Design, build, and maintain data solutions using Azure Data Factory and Azure Synapse.

• Manage the end-to-end data development life cycle within Agile delivery teams.

• Integrate and automate workflows into Azure DevOps pipelines.

• Create and maintain dimension data models and semantic models for Power BI integration.

• Develop advanced dashboards, visualisations, and reporting solutions using Power BI.

• Implement data governance, quality checks, and profiling to ensure accuracy and compliance.

• Lead the migration of Legacy data capabilities to modern Azure-based platforms.

• Apply dbt with SQL databases for data transformation, modelling, and workflow optimisation.

• Collaborate with stakeholders to translate business requirements into robust technical solutions.

• Coach and mentor team members, fostering best practices in data engineering.

Essential Skills & Experience

• Active SC Clearance (mandatory at application stage).

• Proven expertise in:

• Azure Data Factory & Azure Synapse

• Azure DevOps & Microsoft Azure ecosystem

• Power BI (including semantic models)

• Python (incl. PySpark) and advanced SQL

• dbt with SQL DBs (data transformation & modelling)

• Dimension data modelling

• Terraform for infrastructure-as-code deployments

• Strong experience with both structured and unstructured data.

• Delivery track record in Agile environments.

• Business analysis skills to capture and translate service needs into technical solutions.

• Proven success in Legacy migration projects within complex organisations.

• Excellent communication and collaboration skills across technical and non-technical teams.

Desirable

• Azure certifications (Data Engineer, Data Scientist, or related).
• Knowledge of GDPR compliance, data security, and governance best practices.