Lead Data Engineer (Director Level) - Hadoop/Big Data Solution Engineering
- Bengaluru, Karnataka, India
As the world's leader in digital payments technology, Visa's mission is to connect the world through the most creative, reliable and secure payment network - enabling individuals, businesses, and economies to thrive. Our advanced global processing network, VisaNet, provides secure and reliable payments around the world, and is capable of handling more than 65,000 transaction messages a second. The company's dedication to innovation drives the rapid growth of connected commerce on any device, and fuels the dream of a cashless future for everyone, everywhere. As the world moves from analog to digital, Visa is applying our brand, products, people, network and scale to reshape the future of commerce.
At Visa, your individuality fits right in. Working here gives you an opportunity to impact the world, invest in your career growth, and be part of an inclusive and diverse workplace. We are a global team of disruptors, trailblazers, innovators and risk-takers who are helping drive economic growth in even the most remote parts of the world, creatively moving the industry forward, and doing meaningful work that brings financial literacy and digital commerce to millions of unbanked and underserved consumers.
You'rean Individual. We'rethe team for you. Together, let's transform the way the world pays.
- Lead and participate in the design and implementation of our Analytics infrastructure to align with the vision of Visa Data Platform Infrastructure
- Design and implement technical solutions to support the ecosystem of Analytics Infrastructure.
- Responsible for the implementation of ongoing Hadoop infrastructure including monitoring, tuning and troubleshooting.
- Design, implement and maintain security, data capacity, node forecasting and planning.
- Provide hardware architectural guidance, plan and estimate cluster capacity, and create roadmaps for the Hadoop cluster deployment.
- Actively participate in all phases of software development lifecycle: analysis, technical design, planning, development, testing/CICD, release, post production/escalation support
- Work on a highly scalable, available and resilient multi-tenant Platforms that can host large scale and highly critical applications.
- Innovate and Re-pivot Problems with Models and Higher Dimension Concepts to Support Current and Potential Use Case(s)
- Develop large scale multi-tenant software components on the Platform in an Agile based methodology to provide self-service capabilities
- Collaborating with partners from business and technology organizations, develop key deliverables for Data Platform Strategy - Scalability, optimization, operations, availability, roadmap.
- Build partnerships with other technology teams to drive efficiency in their use of platforms and services that PaaS provide
- Expertise in applying the appropriate software engineering patterns to build robust and scalable systems.
- Ability to act as a Tech lead, guide and mentor junior members in the scrum team. Stays current with new and evolving technologies.
- Take ownership of Platform/PaaS architecture and drive it to the next level of effectiveness to support current and long term requirements
- Builds strong commitment within the team to support the appropriate team priorities
- Bachelor's degree in Computer Science, or related technical discipline. With 12+ years of software development experience in building PaaS or Application frameworks.
- Good to be as contributor/committer in one of the Bigdata technologies - Spark, Hive, Kafka, Kubernetes, Presto, Yarn, Hadoop/HDFS
- Proficiency in engineering practices and writing high quality code, with expertise in either one of Java or Python or Go.
- Experience in Bigdata Technologies (Hadoop/Spark/Hive/Presto/HBase...), streaming platforms (Kafka), containers and kubernetes.
- Experience in building REST services and API’s following best practices of service abstractions, Micro-services and Orchestration frameworks.
- Experience building generic services and instrumentation of libraries for wide use across an organization / company.
- Experience in Agile methodology and CICD - tool Integration, automation, configuration management in GIT
- Excellent verbal and written communication and presentation skills, analytical and problem solving skills
- Appreciation of diverse LOBs(Line of business) and their platform needs combined with deep technical expertise are required for the role.