Data Engineer

  • Full-time

Company Description

Consumer division of financial services

Job Description

·         4 <years <10 of relevant experience with data engineering using Spark

·         Design and Develop methods to connect & collect data from different data sources using Spark, Kafka, Hive

·         Design and Develop methods to filter/cleanse the data

·         Design and Develop SQL , Hive queries, APIs to extract data from the store

·         Work closely with data Scientists to ensure the source data is aggregated and cleansed

·         Work with product managers to understand the business objectives

·         Work with cloud and data architects to define robust architecture in cloud setup pipelines and work flows

·         Work with DevOps to build automated data pipelines

·         Business Domain knowledge: Finance & banking systems, Fraud, Payments    

·         Working with Business on storyboarding and requirement elicitation • Can do delivery presentation, getting Business approval/sign-off

·         Experience using Agile methodologies and Scrum • Work with other developers, designers, and test engineers to improve their skills and help deliver business outcomes

·         Ability to apply both technical and business knowledge while participating in the full lifecycle development process

·         Ability to interpret business requirements and logically think through to developing technical solutions

·         The candidate should have performed client facing roles and possess excellent communication skills

·         Analyze and understand  data sources & APIs

 

Required Technical Skills:

·         Big Data-Hadoop, Hive, Apache Spark

·         Python

Qualifications

·         Big Data-Hadoop, Hive, Apache Spark

·         Python

Additional Information

All your information will be kept confidential according to EEO guidelines.

By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply