Data Engineer
- Full-time
Company Description
Consumer division of financial services
Job Description
· 4 <years <10 of relevant experience with data engineering using Spark
· Design and Develop methods to connect & collect data from different data sources using Spark, Kafka, Hive
· Design and Develop methods to filter/cleanse the data
· Design and Develop SQL , Hive queries, APIs to extract data from the store
· Work closely with data Scientists to ensure the source data is aggregated and cleansed
· Work with product managers to understand the business objectives
· Work with cloud and data architects to define robust architecture in cloud setup pipelines and work flows
· Work with DevOps to build automated data pipelines
· Business Domain knowledge: Finance & banking systems, Fraud, Payments
· Working with Business on storyboarding and requirement elicitation • Can do delivery presentation, getting Business approval/sign-off
· Experience using Agile methodologies and Scrum • Work with other developers, designers, and test engineers to improve their skills and help deliver business outcomes
· Ability to apply both technical and business knowledge while participating in the full lifecycle development process
· Ability to interpret business requirements and logically think through to developing technical solutions
· The candidate should have performed client facing roles and possess excellent communication skills
· Analyze and understand data sources & APIs
Required Technical Skills:
· Big Data-Hadoop, Hive, Apache Spark
· Python
Qualifications
· Big Data-Hadoop, Hive, Apache Spark
· Python
Additional Information
All your information will be kept confidential according to EEO guidelines.
By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply