Lead Data Engineer/BI Developer

Posted 9 minutes 7 seconds ago by scrumconnect ltd

Contract
Not Specified
I.T. & Communications Jobs
Not Specified, United Kingdom
Job Description

Overview

A public-sector data team is seeking an experienced Lead Data Engineer/BI Developer to support the development and enhancement of analytics products across several programmes. The role focuses on Power BI dashboard development, Databricks (Delta Lake) data modelling, and migration of existing SQL-based products to a modern data platform.

You will work within an Agile, multi-disciplinary team delivering data products that meet government accessibility, design and technical standards.

Key Responsibilities

1. Dashboard & Data Product Development

  • Build and enhance multiple dashboards using Power BI, working from refined user stories.

  • Develop and maintain semantic data models to support reporting needs.

  • Participate in user-centred design and accessibility reviews.

  • Peer review dashboards for quality, consistency and performance.

2. Databricks Migration

  • Migrate existing BI products from SQL sources into Databricks Delta Lake (Bronze/Silver/Gold layers)

  • Validate data source mappings and document gaps.

  • Re-engineer data transformations and ensure outputs match existing products.

3. R Shiny Dashboards (Desirable)

  • Support the enhancement of R Shiny dashboards with new indicators and updated datasets.

4. Agile Delivery

  • Participate in fortnightly sprints, sprint planning, reviews, and retros.

  • Collaborate closely with Delivery Managers, Product Owners and Analysts.

  • Manage and communicate progress, risks, blockers, and dependencies.

5. Documentation & Knowledge Transfer

  • Produce clear, accessible documentation of data processes, architecture, and modelling decisions.

  • Support handover into BAU and wider delivery teams.

Essential Skills & Experience

    • Strong experience developing Power BI dashboards, including DAX, data modelling and semantic layer design.

    • Experience delivering data pipelines and models in Databricks (PySpark/Spark SQL).

    • Excellent SQL skills across relational and cloud environments.

    • Familiarity with Delta Lake architectures.

    • Experience working with large, complex datasets.

    • Strong Agile delivery experience (Scrum/Kanban).

    • Ability to work independently, communicate effectively and interact with cross-functional teams.

Desirable Skills

      • R and R Shiny dashboard development.

      • Public-sector data standards (accessibility, WCAG 2.1, GDS).

      • Knowledge of education, social care, or attendance datasets.

      • Experience with government data platforms or governed datasets.

Compliance Requirements

  • BPSS and DBS clearance (or willingness to obtain).

  • Use of organisation-approved devices only (no personal accounts or tools).

  • Mandatory data protection and security training.