About the job

NielsenIQ Advanced Analytics team develops and promotes a portfolio of global products. Leveraging NielsenIQ’s unparalleled data and global platforms, the Advanced Analytics team creates solutions to enable our clients to make better business decisions every day.  We partner with Product Owners and Core Technology Teams to build scalable, always on, analytic products using relevant modeling/ML techniques.

We’re looking for a passionate and talented Sr Data Engineer to join our growing team. In this role, you’ll have the chance to be part of a global analytic team of data scientists and data engineers that are focused on designing and implementing analytic products in a Microsoft Azure environment using Python, Spark and cloud based technologies.

Responsibilities 

• Design, develop and optimize data pipelines to support analytics products - using AzureML, PySpark and other Python libraries.

• Work with data scientists to optimise and scale statistical analysis and model implementations.

• Collaborate with product owners, data scientists, and operations personnel to clarify requirements and objectives.

• Create and implement QC testing routines and production monitoring processes.

• Provide level 2 support for analytics products.

• Investigate and implement new software components and techniques as needed.

• Become familiar with NielsenIQ data assets

• Coach and mentor junior data engineers on the team

About you

You’ve worked with large datasets, complex schemas, and have an emphasis on automation. Ideally you are someone who has a strong interest in implementing data models and data pipelines to build scalable products.

Qualifications:

• Degree in computer science or related discipline and 5+ years of experience in big data processing.  

• Working knowledge of Python, SQL, and ETL

• Solid knowledge of SQL databases and development experience, SQL performance monitoring and optimization techniques

• Familiarity with the Hadoop ecosystem

• A minimum of a year of professional Apache Spark data processing application design and development experience, PySpark preferred, but Java/Scala are also acceptable.

• Experience using Pandas, NumPy and Spark ML is desirable

• Experience using collaborative development tools such as Git 

• Experience working in a unix/linux environment

• Experience in working on distributed or cloud computing platforms such as Microsoft Azure

• Experience with Airflow or equivalent orchestration tool is desirable

• Experience implementing machine learning (statistical modelling) models

• Working using agile methodology (Scrum) is desired

• Familiarity with Azure Storage (ADLS Gen2), Azure Databricks, MLFlow, Presto (Trino) and Snowflake is a plus

• Certifications are preferred: Data Engineering on Microsoft Azure, Databricks Certified Associate Developer

About NielsenIQ 

NielsenIQ is a global measurement and data analytics company that provides the most complete and trusted view available of consumers and markets worldwide. We provide consumer packaged goods manufacturers/fast-moving consumer goods and retailers with accurate, actionable information and insights and a complete picture of the complex and changing marketplace that companies need to innovate and grow. Our approach marries proprietary NielsenIQ data with other data sources to help clients around the world understand what’s happening now, what’s happening next, and how to best act on this knowledge.  We like to be in the middle of the action. That’s why you can find us at work in over 90 countries, covering more than 90% of the world’s population. For more information, visit www.niq.com.

NielsenIQ is committed to hiring and retaining a diverse workforce. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability status, age, marital status, protected veteran status or any other protected class.