Distributed Systems Engineer

  • San Francisco, CA
  • Full-time

Company Description

Optimizely is the world’s leading experience optimization platform, providing website and mobile A/B testing and personalization for the world’s leading brands. The platform’s ease of use and speed of deployment empowers organizations to conceive of and run experiments that help them make better data-inspired decisions. Optimizely meets the diverse needs of thousands of customers worldwide looking to deliver connected experiences to their audiences across channels. To date, those customers have created and delivered more than 700 billion optimized visitor experiences.

Job Description

The Challenge

At Optimizely, the Data Platform is the core foundation driving all our Experimentation and Personalization products that directly impact the customer experience. The Platform is leveraged across all product lines from spanning Analytics, Targeting, Recommendations and large-scale Ingestion. This platform, based primarily on the Hadoop Ecosystem on AWS, processes billions of events a day and we’ve built many sophisticated data pipelines that can power a variety of analytical queries at web scale.

We are looking for a strong Distributed Systems Engineer to build cutting-edge capabilities on the Data Platform for our rapidly growing enterprise customer base.

Why is this exciting for you?

This is a unique opportunity for you to shape the future of Optimizely’s Data Platform.

  • You’ll truly enjoy the fun challenges in scaling and operating business-critical services that are built on cutting-edge technologies such as Apache Kafka, Samza and HBase at web scale.

  • You will build highly scalable systems and services that ingest, deduplicate, count, aggregate, store, and archive these events as well as serve real-time analytics insights.

  • You’ll be challenged to engineer solutions that need to work at enterprise grade for our customers that send us their user activity events at web scale (6B+ and multiple TB/day).

  • Examples of projects you’ll work on: a true Lambda architecture (http://bit.ly/2AiEzlB) for large-scale, real-time querying, real-time server-side enrichment of events for top-notch client (browser) performance with Optimizely, identity inference, new analytical capabilities, etc.


Your background

  • You have experience building data-centric applications in a web-scale environment with scale, operability, and performance in mind.

  • You’ve worked on systems leveraging Amazon Web Services (AWS) services.  Hands-on development and deployment experience with Open Data platforms like Apache Hadoop, HBase or Spark would be a big plus.

  • You have strong development experience with Unix/Linux systems and the Java Virtual Machine (JVM) environments with solid troubleshooting and problem-solving skills.

  • Strong interpersonal communication skills and ability to work well in a diverse, team-focused environment with other engineers, Product Managers, Site Reliability Operations etc.

Additional Information

Team’s accomplishments and contributions:

  • Our Streaming Analytics architecture in highscalability, one of the top sites for Distributed Systems architectures: http://bit.ly/2BweFLi

  • Our presentation at LinkedIn’s Stream Processing Meetup on our use of Samza for Sessionization: http://bit.ly/2Ahj6fk

  • Our presentation in Apache Big Data Conference ‘17 in Miami on the use of a Lambda architecture with Streaming: http://bit.ly/2AiEzlB

  • Presentation at HBase Conference in 2014 on migrating to HBase from MongoDB: http://bit.ly/2Bmybcl

  • Many blogs on our engineering Blog site: Scalable data pipelines (http://bit.ly/2hWVrGe), From Batching to Streaming with Apache Samza Part 1 (http://bit.ly/2AfdUZl), Oozie monitoring (http://bit.ly/2hXMlJ9), Druid segment file anatomy (http://bit.ly/2iZLgkT), Druid slicing and dicing (http://bit.ly/2k9B6BL) and Approximate Counting (http://bit.ly/2hXMlJ9).

  • Committed to Apache Samza for our streaming needs: http://bit.ly/2hZ6uyo

  • Our Engineering blog: http://bit.ly/2zBmbDn


  • Commuter and transportation benefits
  • Catered in-office lunch and dinner on weekdays
  • Full medical insurance with very low co-pay and deductible. HMO, PPO, and HSA options available
  • Full dental coverage including orthodontics
  • Full vision coverage including contacts
  • Dependents 100% covered for medical, dental, and vision
  • Wellness Grant
  • Unlimited vacation policy and seventeen weeks of paid parental leave
  • 401k benefit
  • Working with a great team and having a huge impact!

All your information will be kept confidential according to EEO guidelines.

Privacy Policy