Data Engineer for Shelf Analytics MŁ

Posted 1 day 5 hours ago by Luxoft

£80,000 - £100,000 Annual
Permanent
Full Time
Other
Not Specified, United Kingdom
Job Description
Project description

We are looking for an experienced Data Engineer to join the Shelf Analytics project - a data-driven application designed to analyze how P&G products are positioned on store shelves. The primary objective of the solution is to improve product visibility, optimize in-store execution, and ultimately increase sales by combining shelf layout data with sales insights. As a Data Engineer, you will play a key role in building, maintaining, and enhancing scalable data pipelines and analytics workflows that power shelf-level insights. You will work closely with analytics and business stakeholders to ensure high-quality, reliable, and performant data solutions.

Responsibilities
  • Design, develop, and maintain data pipelines and workflows using Databricks and PySpark
  • Read, understand, and extend existing codebases; independently develop new components for Databricks workflows
  • Implement object-oriented Python solutions (classes, inheritance, reusable modules)
  • Develop and maintain unit tests to ensure code quality and reliability
  • Work with Spark SQL and SQL Server Management Studio to create and optimize complex queries
  • Create and manage Databricks workflows, clusters, databases, and tables
  • Handle data storage and access management in Azure Data Lake Storage (ADLS), including ACL permissions
  • Collaborate using GitHub, following CI/CD best practices and working with GitHub Actions
  • Support continuous improvement of data engineering standards, performance, and scalability
Skills - must have
  • Strong programming skills in Python and PySpark
  • Hands-on experience with Databricks (workflows, clusters, tables, databases)
  • Solid knowledge of SQL and experience with Spark SQL and SQL Server Management Studio
  • Experience with pandas, dbx, and unit testing frameworks
  • Practical experience working with Azure Storage (ADLS) and access control (ACLs)
  • Proficiency with GitHub, including CI/CD pipelines and GitHub Actions
  • Ability to work independently, analyze existing solutions, and propose improvements
Nice to have

Experience with retail, CPG, or shelf analytics-related solutions

Familiarity with large-scale data processing and analytics platforms

Strong communication skills and a proactive, problem-solving mindset