Senior Data Engineer

Job Title: Senior Data Engineer

Work Arrangement: Hybrid (1/2 days Onsite)

Location: Leopardstown, Dublin

12 month Rolling Contract

*Only applicants with 6+ years experience will be considered.*

Job Description

Join our growing SME Data Team as a Data Engineer II, where you will work alongside a talented team to build scalable data pipelines and contribute to platform modernisation efforts. Your role will involve supporting business-critical data solutions across on-prem Hadoop and Databricks Cloud environments. This is a hands-on engineering role that offers opportunities for collaboration, learning, and growth in a technically rich environment.

Responsibilities

• Build, maintain, and optimise batch and streaming data pipelines using Spark (PySpark/Scala).
• Contribute to the migration and modernisation of on-prem Hadoop workloads to Databricks Cloud.
• Develop and support ETL/ELT processes leveraging Hive, Impala, HDFS, and Airflow.
• Implement data ingestion, transformation, and quality checks using tools like Delta Lake, SQL Warehouses, and DLT.
• Collaborate with Senior engineers and leads to design data solutions that are scalable and maintainable.
• Support troubleshooting, performance tuning, and documentation of data workflows and systems.

Essential Skills

• Strong coding skills in Python or Scala with a solid understanding of distributed data processing.
• Hands-on experience with data engineering in Spark and Hadoop ecosystems (HDFS, Hive, Impala).
• Working knowledge of Airflow or similar orchestration tools.
• Exposure to cloud platforms, preferably Databricks on AWS, including S3, Delta Tables, and DLT.
• Familiarity with SQL performance tuning and large-scale data processing techniques.

Additional Skills & Qualifications

• Exposure to Databricks SQL Warehouses and Unity Catalog.
• experience in data quality, validation, or testing frameworks.
• Understanding of data governance, lineage, or security practices in enterprise environments.

Why Work Here?

We provide a dynamic and inclusive work environment that encourages growth and learning. Our commitment to building a sustainable world that unlocks priceless possibilities for all drives our culture. Join us to be part of a global technology company in the payments industry that is at the forefront of innovation, ensuring a safe, simple, smart, and accessible digital economy.

Work Environment

The role is based within a collaborative and supportive team environment, utilising modern technologies such as Spark, Hadoop, and Databricks Cloud. You will have the opportunity to work in a flexible setting that supports a balanced work-life dynamic.

Dublin, Ireland

Trading as TEKsystems. Allegis Group Limited. Level 1, The Chase, Carmanhall Road, Sandyford, Dublin 18, Ireland. No. 909257. TEKsystems is a company within the Allegis Group network of companies (collectively referred to as "Allegis Group"). Aerotek, Aston Carter, EASi, Talentis Solutions, TEKsystems, Stamford Consultants and The Stamford Group are Allegis Group brands. If you apply, your personal data will be processed as described in the Allegis Group Online Privacy Notice available at our website.

To access our Online Privacy Notice, which explains what information we may collect, use, share, and store about you, and describes your rights and choices about this, please go our website.

We are part of a global network of companies and as a result, the personal data you provide will be shared within Allegis Group and transferred and processed outside the UK, Switzerland and European Economic Area subject to the protections described in the Allegis Group Online Privacy Notice. We store personal data in the UK, EEA, Switzerland and the USA. If you would like to exercise your privacy rights, please visit the "Contacting Us" section of our Online Privacy Notice on our website for details on how to contact us. To protect your privacy and security, we may take steps to verify your identity, such as a password and user ID if there is an account associated with your request, or identifying information such as your address or date of birth, before proceeding with your request. commitments under the UK Data Protection Act, EU-U.S. Privacy Shield or the Swiss-U.S. Privacy Shield.