Cortex empowers internal teams to efficiently leverage ML by providing a platform and by unifying, educating, and advancing the state of the art in ML technologies within Twitter.

We win when our customers win by helping our users stay informed, share and discuss what matters; by serving the public conversation. We’re building an AI-first company and every major initiative is increasingly dependent on the successful application of machine learning. Cortex is at the nexus of this evolution. Our team of ML software engineers is constructing one of the strongest machine learning platforms in the world, based on the latest ML industry practices, deep learning, engineering excellence, and powered by Twitter data at scale.

The ML Feature Management (MLFM) team is part of the Cortex Platform group. Together with the other four Cortex Platform teams, we develop tools and infrastructure that standardizes machine learning development at Twitter on a common platform. This allows ML engineers in our customer teams to do their work better and faster and with a more modern stack that follows broad industry trends.

More specifically, our team’s vision is to maximize the velocity of Feature Engineering. To date, we’ve built and heavily invested in Twitter’s Feature Store, to share features across different ML product teams in production and thereby to broaden the impact of feature engineering investments across the company. This has been a wildly successful mission over the last two years. The Feature Store is running in production in many of Twitter’s ML stacks, serving millions of requests per second at super low latencies which has impacted top-line metrics like daily active users. These wins have prompted all remaining ML teams to migrate to the Feature Store. We’re assisting with this migration which we expect to complete in 2020.

Our other investments for 2020 include:

• Core API improvements include new functionality to deliver a more complete solution to our customers.
• Simplifying our architecture with an eye toward expandability and maintainability.
• Systematic performance optimization and increasing production resilience.
• Automating workflows for feature registration and for migration to the Feature Store as much as possible.
• Support new and different data formats for feature data.
• We’re participating in Cortex’s virtual team to explore ML tooling available in the Google Could Platform (GCP). For our team, this means integrating data processing engines such as BigQuery and Dataflow and their application to feature engineering use cases.
• Seed projects to broach new frontiers:
  • Prototyping tooling to accelerate feature experimentation and exploration.
  • Building ML-as-a-service, a system that can handle dataset creation, training and online feature hydration purely from configurations and model specification, with the purpose of taking successful notebook experiments to an A/B test in production quickly.
  • Systematically tracking and analyzing metadata of all stages of tooling for answering questions around feature-model lineage, data privacy protection and easier model performance debugging.
  • Supporting trained embeddings as first class features throughout our tooling ecosystem.

What you will do

We’re looking for a senior engineer to join us in our core Feature Store workstream who can help us simplify the code base while adding new core functionality to tackle complex use cases, but without compromising performance. If thinking in architectural frameworks, like primitives, design patterns, functional paradigms, or the commonality-variability principle is your middle name, then we want to talk to you. If chasing down latency losses, drilling into performance traces and dissecting the JVM memory model is up your alley, give us a call.

The position is available in New York or San Francisco, but we're also open to other locations.

Who you are

Do you identify with the majority of the following traits? Yes? We believe they will make you successful in this role.

• You’re fluent in concepts like design patterns, composition vs inheritance, commonality variability principle, primitives, etc.
• You have a solid grasp on type systems and functional programming, ideally in Scala or Java and a dynamic language like Python.
• You have educated opinions about what good library and API design in distributed systems looks like, both from formal study and from lived experience with past design decisions you made.
• You’re able to articulate such opinions and concepts persuasively both in spoken and written form, with the intent to teach and enroll others, while remaining open to a constructive dialogue. Your thought process intrigues others with its clarity.
• Your take a principled, disciplined approach to software engineering, characterized by high volume and quality, solid testing practices, data-driven decision making, and a low rate of incidents in production.
• You take pride in the developer experience, the ease of use, documentation and the durability of the products you build, together with the team.
• You have a passion for machine learning and high-leverage infrastructure platforms.
• You bring partners together across organizational and functional boundaries.
• You’re motivated by shipping high-impact software to production and you have a track record of doing so.

By nature of the problem domain, we expect you to have experience in:

• 5+ years building and delivering working software through an iterative, agile process.
• 2+ years with large Scala or Java code bases with many contributors
• 3+ years in Sr engineering or architect capacity with a track record of resilient design decisions shipped to production.
• 5+ years of work experience in software engineering in the areas of distributed data processing, in high traffic or large data production environments.
• M.S. or Ph.D. degree in computer science or a related field or equivalent work experience.

We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, ethnicity, color, ancestry, national origin, religion, sex, sexual orientation, gender identity, age, disability, veteran, genetic information, marital status or any other legally protected status.

San Francisco applicants: Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.