Distributed Systems Engineer

  • San Francisco, CA
  • Full-time

Company Description

Optimizely is the world's leader in customer experience optimization, allowing businesses to dramatically drive up the value of their digital products, commerce and campaigns through its best in class experimentation software platform. By replacing digital guesswork with evidence-based results, Optimizely enables product and marketing professionals to accelerate innovation, lower the risk of new features, and drive up the return on investment from digital by up to 10X. Over 26 of the Fortune 100 companies choose Optimizely to power their global digital experiences. Optimizely’s impressive customer list includes eBay, FOX, IBM, The New York Times and many more global enterprises.

Job Description

The Challenge

At Optimizely, the Data Platform is the core foundation driving all our Experimentation and Personalization products that directly impact the customer experience. The Platform is leveraged across all product lines from spanning Analytics, Targeting, Recommendations and large-scale Ingestion. This platform, based primarily on the Hadoop Ecosystem on AWS, processes billions of events a day and we’ve built many sophisticated data pipelines that can power a variety of analytical queries at web scale.

We are looking for a strong Distributed Systems Engineer to build cutting-edge capabilities on the Data Platform for our rapidly growing enterprise customer base.

Why is this exciting for you?

This is a unique opportunity for you to shape the future of Optimizely’s Data Platform.

  • You’ll truly enjoy the fun challenges in scaling and operating business-critical services that are built on cutting-edge technologies such as Apache Kafka, Samza and HBase at web scale.

  • You will build highly scalable systems and services that ingest, deduplicate, count, aggregate, store, and archive these events as well as serve real-time analytics insights.

  • You’ll be challenged to engineer solutions that need to work at enterprise grade for our customers that send us their user activity events at web scale (6B+ and multiple TB/day).

  • Examples of projects you’ll work on: a true Lambda architecture (http://bit.ly/2AiEzlB) for large-scale, real-time querying, real-time server-side enrichment of events for top-notch client (browser) performance with Optimizely, identity inference, new analytical capabilities, etc.


Your background

  • You have experience building data-centric applications in a web-scale environment with scale, operability, and performance in mind.

  • You’ve worked on systems leveraging Amazon Web Services (AWS) services.  Hands-on development and deployment experience with Open Data platforms like Apache Hadoop, HBase or Spark would be a big plus.

  • You have strong development experience with Unix/Linux systems and the Java Virtual Machine (JVM) environments with solid troubleshooting and problem-solving skills.

  • Strong interpersonal communication skills and ability to work well in a diverse, team-focused environment with other engineers, Product Managers, Site Reliability Operations etc.

Additional Information


Team’s accomplishments and contributions:

  • Our Streaming Analytics architecture in highscalability, one of the top sites for Distributed Systems architectures: http://bit.ly/2BweFLi

  • Our presentation at LinkedIn’s Stream Processing Meetup on our use of Samza for Sessionization: http://bit.ly/2Ahj6fk

  • Our presentation in Apache Big Data Conference ‘17 in Miami on the use of a Lambda architecture with Streaming: http://bit.ly/2AiEzlB

  • Presentation at HBase Conference in 2014 on migrating to HBase from MongoDB: http://bit.ly/2Bmybcl

  • Many blogs on our engineering Blog site: Scalable data pipelines (http://bit.ly/2hWVrGe), From Batching to Streaming with Apache Samza Part 1 (http://bit.ly/2AfdUZl), Oozie monitoring (http://bit.ly/2hXMlJ9), Druid segment file anatomy (http://bit.ly/2iZLgkT), Druid slicing and dicing (http://bit.ly/2k9B6BL) and Approximate Counting (http://bit.ly/2hXMlJ9).

  • Committed to Apache Samza for our streaming needs: http://bit.ly/2hZ6uyo

  • Our Engineering blog: http://bit.ly/2zBmbDn



  • Commuter and transportation benefits
  • Catered in-office lunch and dinner on weekdays
  • Full medical insurance with very low co-pay and deductible. HMO, PPO, and HSA options available
  • Full dental coverage including orthodontics
  • Full vision coverage including contacts
  • Dependents 100% covered for medical, dental, and vision
  • Wellness Grant
  • Unlimited vacation policy and seventeen weeks of paid parental leave
  • 401k benefit
  • Working with a great team and having a huge impact!


At Optimizely, we embody inclusion and embrace diversity.  Optimizely is an equal opportunity employer and makes employment decisions on the basis of merit.  Optimizely prohibits discrimination based on race, color, religion, sex, sexual identity, gender identity, marital status, veteran status, nationality, citizenship, age, disability, medical condition, pregnancy, or any other unlawful consideration. All your information will be kept confidential according to EEO guidelines.

Privacy Policy