Jr. Data Engineer
- Foster City, CA, USA
VISA Commercial Payments is seeking talented Big Data Engineer. Commercial Payments team develops Data pipeline which supports Cross Border payments rail for VISA globally through advanced algorithms running on big data technologies. The ideal candidate thrives in a fast-paced environment and relishes working with petabytes of extremely complex and dynamic data. In this role you will be part of a team of high caliber data engineers to build data pipelines using big data technologies such as Apache Spark, Hive/Hadoop, and distributed query engines. You should be passionate about working with big data and have the aptitude to incorporate new technologies and evaluate them critically. You should have excellent business and communication skills and be able to work with business owners to develop and define key business questions and build solutions. You are a self-starter, comfortable with ambiguity, and working in a fast-paced and ever-changing environment. Ideally, you are also experienced with at least one of the programming languages such as Java, C++, Scala, etc.
• Interface with PMs, business customers, and software developers to understand requirements and implement solutions
• Design, develop, and operate highly-scalable, high-performance, low-cost, and accurate data pipelines in distributed data processing platforms
• Recognize and adopt best practices in data processing, reporting, and analysis: data integrity, test design, analysis, validation, and documentation
• Keep up to date with big data technologies, evaluate and make decisions around the use of new or existing software products to design the data architecture
• Bachelor’s degree in Computer Science, Electrical Engineering, Information Systems, Mathematics, or a related field
• Ability to work and communicate effectively with developers and Business users
• Strong organizational and multitasking skills with ability to balance competing priorities
• Experience building large-scale applications and services with big data technologies
• 2-3 year of experience in designing and developing analytical systems
• 2-3 years of experience in designing and developing data processing pipelines using distributed computing technologies such as Spark, and Pig, Kafka streaming
• Expertise in Data storing mechanisms and No-SQL interface
• Experience with scripting languages such as Python, Perl, etc.
• Proficiency with Linux/Unix systems
• Experience with full software development life cycle, including coding standards, code reviews, source control management, CI/CD , Automation testing
• Experience with big data technologies such as Hadoop, Hive, Hbase, Pig, Spark, etc.
• Experience with programming languages such as Java, C++, Scala, etc.
• Experience with open source big data processing systems and infrastructure such as Spark, Hive, Kafka, H-Base, etc.
What we are not looking for:
Fresh Undergraduates or having experience less than 2 Years
Have not used Spark-Scala in production deployment, troubleshooting
Candidate having only HIVE / SQL experience
Candidate will be working in VISA Global Commercial Payments area, team will be building a new payment rail leveraging Blockchain technology and distributed ledger, project details can be found here --