Data Engineer for Shelf Analytics MŁ

Project description

We are looking for an experienced Data Engineer to join the Shelf Analytics project - a data-driven application designed to analyze how P&G products are positioned on store shelves. The primary objective of the solution is to improve product visibility, optimize in-store execution, and ultimately increase sales by combining shelf layout data with sales insights. As a Data Engineer, you will play a key role in building, maintaining, and enhancing scalable data pipelines and analytics workflows that power shelf-level insights. You will work closely with analytics and business stakeholders to ensure high-quality, reliable, and performant data solutions.

Responsibilities

• Design, develop, and maintain data pipelines and workflows using Databricks and PySpark
• Read, understand, and extend existing codebases; independently develop new components for Databricks workflows
• Implement object-oriented Python solutions (classes, inheritance, reusable modules)
• Develop and maintain unit tests to ensure code quality and reliability
• Work with Spark SQL and SQL Server Management Studio to create and optimize complex queries
• Create and manage Databricks workflows, clusters, databases, and tables
• Handle data storage and access management in Azure Data Lake Storage (ADLS), including ACL permissions
• Collaborate using GitHub, following CI/CD best practices and working with GitHub Actions
• Support continuous improvement of data engineering standards, performance, and scalability

Skills - must have

• Strong programming skills in Python and PySpark
• Hands-on experience with Databricks (workflows, clusters, tables, databases)
• Solid knowledge of SQL and experience with Spark SQL and SQL Server Management Studio
• Experience with pandas, dbx, and unit testing frameworks
• Practical experience working with Azure Storage (ADLS) and access control (ACLs)
• Proficiency with GitHub, including CI/CD pipelines and GitHub Actions
• Ability to work independently, analyze existing solutions, and propose improvements

Nice to have

Experience with retail, CPG, or shelf analytics-related solutions

Familiarity with large-scale data processing and analytics platforms

Strong communication skills and a proactive, problem-solving mindset