Senior Data Engineer

Salary -  $125,000 (Chicago)

About Retail Insight

At Retail Insight, we're transforming the way some of the world's biggest retailers and CPG brands use data, analytics and technology to make smarter decisions.

Our people are at the heart of what we do. We bring together talented people with different experiences, perspectives and expertise to solve complex problems, challenge the status quo and make a real impact for our clients.

We're growing, and we're looking for a Senior Data Engineer to join our Engineering team.

About the role

We're looking for an experienced Senior Data Engineer to build and support the reliable, scalable data systems that power our solutions for US retail clients.

As our US client portfolio grows, you'll work with complex data environments, helping translate business requirements into practical technical solutions. Working closely with our Chicago-based Lead Data Engineer, alongside data scientists, analysts and product teams, you'll build data pipelines and products that help our clients make better decisions.

This is a hands-on role combining strong technical skills with collaboration and clear communication. You'll contribute to the evolution of our data platform, improve reliability and performance, and help maintain high engineering standards.

You'll also work with colleagues and client stakeholders to understand their needs and support successful delivery. 

What you'll be doing

• Building and maintaining scalable data pipelines, developing and optimising batch and streaming workflows for reliability and performance.
• Contributing to the design and implementation of data architecture.
• Implementing data quality checks, monitoring and observability across pipelines.
• Applying security, access control and governance standards established by our platform team.
• Collaborating with data scientists and analysts to deliver data products that support their work.
• Working with the Lead Data Engineer to translate business requirements into technical solutions.
• Participating in code reviews, setting a high bar for engineering quality and contributing to team standards.
• Helping identify opportunities to optimise cloud costs and platform performance.
• Contributing to platform documentation, tooling choices and continuous improvement.

What we’re looking for

• Solid, demonstrable experience in data engineering, including building and maintaining production-grade Databricks pipelines.
• Experience with Databricks Unity Catalog and Databricks Asset Bundles.
• Strong hands-on knowledge of Apache Spark, Delta Lake and Databricks Workflows.
• Proficiency in Python and SQL, with a good understanding of data modelling, ETL/ELT patterns and distributed data processing.
• Experience with Azure data platforms, including Azure Data Factory, ADLS Gen2 and Azure DevOps, or equivalent services on AWS or GCP.
• Experience implementing data quality checks and pipeline monitoring.
• Strong communication skills and the ability to work effectively with technical and non-technical colleagues.
• A collaborative approach, ownership of quality outcomes and a drive to learn and improve.
• A degree in computer science, data engineering or a related field, or equivalent practical experience.
• Willingness to travel as required.

Bonus points if you also have

• Familiarity with infrastructure-as-code tools such as Terraform or Bicep.
• Familiarity with dbt or similar data transformation frameworks.
• Exposure to real-time streaming technologies such as Kafka or Event Hubs.
• Exposure to retail or consumer packaged goods (CPG) data, such as point of sale, inventory or supply chain.
• Experience working directly with clients.

Some of our extras...

We believe in flexibility, growth, and taking care of our people, both in and out of the office.

Hybrid & Flexible Working: Work from anywhere benefits, plus summer hours and extra days off for life’s big moments (including your birthday and wedding).

Generous Time Off: 26 days PTO (accrued with service), paid volunteer days, and full support for your wellbeing, including mental health resources.

Career & Learning: Access to a top-tier learning platform, career progression plans, and support to create your ideal home workspace.

Health & Financial Benefits: Comprehensive medical, dental, and vision coverage, plus a 6% matched retirement plan.

Personal Perks: Annual phone reimbursement, onsite gym access, and customizable benefits via our online platform.

Plus much more!

Our Interview Process

Our interview process may vary slightly depending on the role. We’ll always let you know what to expect at each stage and give you plenty of opportunities to ask questions along the way.

People Team Introduction and Cultural Drivers Interview

An initial conversation with a member of our People Team to learn more about you, your experience and what you’re looking for in your next role. We’ll also introduce you to Retail Insight and explain the opportunity and interview process. During the call, we’ll ask interview questions based on our cultural drivers to understand how your values, behaviours and ways of working align with ours.

Role and Technical Interview

This stage explores your experience and the skills most relevant to the role, as well as how you approach your work. Depending on the position, it may include a practical task, presentation or technical assessment. We’ll always explain what’s involved in advance.

Senior Leadership Coffee Chat

The final stage is an informal coffee chat with a member of our senior leadership team. It’s an opportunity to learn more about our business and culture, ask any remaining questions and make sure Retail Insight feels like the right fit for you.

Be your authentic self - Retail Insight is committed to promoting equal opportunities in employment. All employees and any job applicants will receive equal treatment. We actively seek to create an environment where everyone feels respected, supported, and encouraged to contribute their best work.