Senior Data Engineer- Platform as a Service

  • Palo Alto, CA, USA
  • Full-time

Company Description

Visa is a world leader in digital payments, facilitating more than 215 billion payments transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable and secure payments network, enabling individuals, businesses and economies to thrive.

When you join Visa, you join a culture of purpose and belonging – where your growth is priority, your identity is embraced, and the work you do matters. We believe that economies that include everyone everywhere, uplift everyone everywhere. Your work will have a direct impact on billions of people around the world – helping unlock financial access to enable the future of money movement.

Join Visa: A Network Working for Everyone.

Job Description

Visa is seeking an Senior Data Engineer in the Data and AI Platform department to build and manage Visa’s Open-Source Hadoop in the Data and AI Platform.

This individual will act as a trusted technical expert and strategic contributor to the Data and AI Platform department. The successful candidate should be comfortable navigating challenging projects in big data and Data and AI Platform areas and be comfortable in a fast-paced environment with significant changes, developing/implementing strategies and processes that enable the organization to innovate and grow fast. You will have the opportunity to lead, participate, guide, and mentor other people in the team on architecture and design in a hands-on manner. This position reports to the head of the Large-Scale Shared Platform team in Palo Alto. The ideal candidate will have very strong development skill while exposure Open-Source Hadoop and big data / large-scale software platforms like Hadoop, Presto, Hive and HDFS are required.

Essential Functions

  • Provide Platform-as-a-Service offerings that are easy to consume, scalable, secure, and reliable using open source based solutions for Big Data technologies
  • Work at the intersection of infrastructure and software engineering, and design and deploy cloud infrastructure build on top of open-source components, including Hadoop, Hive, Spark, HBase, Kafka streaming and other Open Source Big Data technologies.
  • Collaborate with various teams to build and maintain the most innovative, reliable, secure, and cost-effective distributed solutions. 
  • Bring together cross-functional teams and build automation to deliver programmable and vendor-neutral infrastructure that transforms virtualization to new compute paradigm. Design the Cloud, and Container strategy around PaaS Solutions.
  • Facilitate knowledge transfer to the Engineering and Operations team during and after deployment.
  • Work with geographically distributed teams on technical challenges and process improvements.
  • Responsible for the design and implementation of agile-innovative infrastructure solutions that take advantage of technology advances that allow cost reduction, standardization, and commoditization.
  • Drive the adoption of open standard toolsets to reduce complexity and support operational goals for increasing automation across the enterprise.
  • Champion the adoption of open infrastructure solutions that are fit for purpose while keeping technology relevant.


Basic Qualifications:
•2+ years of related work experience with a Bachelor's Degree or at least 4 years of related work experience with an Advanced Degree (e.g. Masters/MBA/JD/MD) in Computer Science or related field

Preferred Qualifications:

  • Experience in building developer productivity tools to improve the overall developer productivity
  • Expertise in design and management of complex data structures and data processes
  • Expertise in managing and operating distributed big data systems, including but not limited to Hadoop eco system - Hive, HDFS, Spark, Kafka and etc.
  • Deep knowledge and hands on experience cloud computing technologies.
  • Expertise on design/coding using any one of the programming languages: Python, Golang, Java, C/C++. Can write very solid prototypes or core components quickly.
  • Experience in service architecture and development experience with high performance and scalability
  • Technical background in data with deep understanding of issues in multiple areas such as data acquisition and processing, data management, distributed processing, and high availability is required
  • Outstanding verbal, written, presentation, facilitation, and interaction skills, including ability to effectively communicate architectural issues and concepts to multiple organization levels and executive management

Additional Information

Visa has adopted a COVID-19 vaccination policy to safeguard the health and well-being of our employees and visitors. As a condition of employment, all employees based in the U.S. are required to be fully vaccinated for COVID-19, unless a reasonable accommodation is approved or as otherwise required by law.

Work Hours: Varies upon the needs of the department.

Travel Requirements: This position requires travel 5-10% of the time.

Mental/Physical Requirements: This position will be performed in an office setting.  The position will require the incumbent to sit and stand at a desk, communicate in person and by telephone, frequently operate standard office equipment, such as telephones and computers.

Visa is an EEO Employer.  Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status.  Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Visa will consider for employment qualified applicants with criminal histories in a manner consistent with applicable local law, including the requirements of Article 49 of the San Francisco Police Code.

Privacy Policy