Sr. Data Engineer - Python

  • Full-time
  • Job Family Group: Engineering and Technology

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

About the Team:

At Data Science Platform (DScP), we are revolutionizing the field of Data Engineering and Data Science. We are a dynamic and innovative team, driven by a passion for Self-service Data Platform,  Data Mesh and  AI/ML technology and a commitment to delivering exceptional platforms and services to our business partners internally at Visa. Join us in shaping the future of Generative AI for Data Solutions and making a significant impact on the enterprise. 

As a member of the team, you will have the opportunity to directly contribute and deliver solutions that are intuitive, user centric, and accessible for Visa’s Data Mesh and beyond.

Essential Functions: 

  • Hands-on Programming 
  • Build analytics tools that utilize the data automation pipeline to provide actionable insights.  
  • Proficient in building, scheduling, and managing DAGs in Apache Airflow.
  • Monitor data processing tasks in the data pipelines monitoring tool - Airflow.
  • Quality control of data assets to ensure the data loaded across different stages in the data pipeline are reconciled.
  • Strong analytical and debugging skills on data issues and clearly communicating clarifications and recommendations to stakeholders.
  • Work on emerging technologies, building highly scalable, secure, reliable and fault tolerant distributed systems. 
  • Participate in design and code review sessions as appropriate. 
  • Provide third-level support to business users. 
  • Present technical solutions, capabilities, considerations, and features in business terms. Effectively communicate status, issues, and risks in a precise and timely manner. 
  • Independently understands data ecosystem, data, security, data privacy, and retention requirements needed to support business and product features.
  • Interprets pipeline designs with minimal guidance.
  • Creates and implements code and applies coding patterns, guidelines, styles, and best practices; and adheres to all security requirements.
  • Follows general guidelines to conduct unit testing to confirm functional capability of code with minimal oversight and participates in user acceptance testing in collaboration with customer.
  • Independently merges data into distributed systems, products, or tools for further processing, and helps peers as needed.

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

Qualifications

Basic Qualifications
• 2 or more years of work experience with a Bachelor’s Degree or an Advanced Degree (e.g. Masters, MBA, JD, MD, or PhD)

Preferred Qualifications
• 3 or more years of work experience with a Bachelor’s Degree or more than 2 years of work experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) in Computer Science, Information Technology, Engineering, Computer Engineering.
• 3+ years' experience in PySpark, Spark, Scala, Unix, Python programming/scripting experience (must haves).
• 2+ years' experience in one or more of the following: Kafka, Spark streaming, Airflow, Control-M, Presto.
• 2+ years' experience in one or more of the following: SQL, NOSQL Database.
• Experience in Hadoop, HDFC, Hive Tuning, Bucketing, Partitioning.
• Java experience is a plus.
• Expert level usage with Jenkins, GitHub / Bitbucket
• Experience on DataLakes, Data Mesh and datahub implementation desired

Additional Information

Work Hours: Varies upon the needs of the department.

Travel Requirements: This position requires travel 5-10% of the time.

Mental/Physical Requirements: This position will be performed in an office setting.  The position will require the incumbent to sit and stand at a desk, communicate in person and by telephone, frequently operate standard office equipment, such as telephones and computers.

Visa is an EEO Employer.  Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status.  Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Visa will consider for employment qualified applicants with criminal histories in a manner consistent with applicable local law, including the requirements of Article 49 of the San Francisco Police Code.

U.S. APPLICANTS ONLY: The estimated salary range for a new hire into this position is 125,500.00 to 177,250.00 USD per year, which may include potential sales incentive payments (if applicable). Salary may vary depending on job-related factors which may include knowledge, skills, experience, and location. In addition, this position may be eligible for bonus and equity. Visa has a comprehensive benefits package for which this position may be eligible that includes Medical, Dental, Vision, 401 (k), FSA/HSA, Life Insurance, Paid Time Off, and Wellness Program.

By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply

Privacy Notice