Principal Software Development Engineer in Test – Big Data

  • Full-time
  • Department: QA

Company Description

PubMatic (Nasdaq: PUBM) is an independent technology company maximizing customer value by delivering digital advertising’s supply chain of the future. PubMatic’s sell-side platform empowers the world’s leading digital content creators across the open internet to control access to their inventory and increase monetization by enabling marketers to drive return on investment and reach addressable audiences across ad formats and devices. Since 2006, our infrastructure-driven approach has allowed for the efficient processing and utilization of data in real time. By delivering scalable and flexible programmatic innovation, we improve outcomes for our customers while championing a vibrant and transparent digital advertising supply chain.

Job Description

PubMatic is one of the leaders in tech stack when it comes to Big data infrastructure and data processing. We at PubMatic process more than 150 Bn ad impressions a day which contribute to around 100 tera bytes of uncompressed data. To maintain and process this raw data we have our own data centres across the globe and do in house ingestion, ETL and aggregation using a mammoth Hadoop infrastructure on top of thousands of BareMetal nodes built from scratch. Having most things built in house, PubMatic is a very early adaptor of new technologies that come along in the big data space.

We are looking for individuals with knowledge and experience of working with distributed environments for becoming a part of our big data test engineering group. The individual will also be responsible for automating various big data flows, performance engineering and fine tuning the big data pipeline and making sure the quality of the above mentioned giant business critical infrastructure is intact. Individual will also get an opportunity to work with the high speed GPU machines used for rapid computing of large data sets.


  • Should have minimum 4 years of experience on working BigData technologies
  • Good Programming skills.
  • Hands on Experience in Automating Backend Applications (e.g. db, REST API's)
  • Hands on experience with Automating any backend applications (e.g db , server side).
  • Knowledge of relational databases and SQL
  • Good debugging skills.
  • Strong working experience working in Linux/Unix environment.
  • Strong understanding of testing methodologies.
  • Hands on experience in working on Big Data technologies like Hadoop, Spark
  • Hand on experience in working with ETL Testing
  • Hands on experience in QA Automation Framework development & Design & Strong hold on data structures.
  • Preferred language Python/Shell Scripting
  • Strong Understanding of OS and performance benchmarking
  • Quick learner and good team member with positive attitude.
  • Good verbal and written communication skills.

Duties and Responsibilities:

  • Testing big data ingestion and aggregation flows using spark shell and related queries
  • Developing automation framework using programming languages such as python and automate the big data workflows such as ingestion, aggregation, ETL processing etc
  • Debugging and troubleshooting issues within the big data ecosystem
  • Set up the Big data platform and Hadoop ecosystem for testing
  • Define test strategy and write test plan for the data platform enhancements and new features/services built on it.
  • Define the operating procedures, service monitors and alerts and work with the NOC team to get them implemented.
  • Responsible for system & performance testing of the data platform and applications
  • Solve problems and establish plans and provide technical consultation in the design, development and test effort of complex engineering projects
  • Review product specifications and write test cases, develop test plans for assigned areas.
  • Identifies issues and technical interdependencies and suggest possible solutions.
  • Recreate complex customer and production reported issues to determine root cause and verify the fix.


Primary (Mandatory) Skills:

  • QA with good hands on experience with Unix/Linux
  • QA experience in Networking and/or Big Data domain.
  • Automation experience into Python.
  • QA Methodologies understanding.

 Secondary Skills (Good to have):

  • Experience in Big data platform & data analytics testing is an advantage.
  • The Senior Software Engineer will have the end to end ownership of feature starting from testing, automation (if applicable), deployment and helping with monitoring of the feature(s)


Additional Information

Return to Office: PubMatic employees throughout the global have returned to our offices via a hybrid work schedule (3 days “in office” and 2 days “working remotely”) that is intended to maximize collaboration, innovation, and productivity among teams and across functions. All PubMatic employees in the US and India are required to be fully vaccinated to return to our offices. Covid-19 boosters are not required at this point in time.

Benefits: Our benefits package includes the best of what leading organizations provide, such as stock options, paternity/maternity leave, healthcare insurance, broadband reimbursement. As well, when we’re back in the office, we all benefit from a kitchen loaded with healthy snacks and drinks and catered lunches and much more!

Diversity and Inclusion: PubMatic is proud to be an equal opportunity employer; we don’t just value diversity, we promote and celebrate it. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.