Software Engineer -Cortex

  • Full-time

Company Description

Twitter is what’s happening and what people are talking about right now. For us, life's not about a job, it's about purpose. We believe real change starts with conversation. Here, your voice matters. Come as you are and together we'll do what's right (not what's easy) to serve the public conversation.

Who We Are

At Cortex our purpose is to improve Twitter by enabling advanced and ethical AI.

Within Cortex, the ML Platform group was formed to accelerate the impact of ML through tools and infrastructure, in an ethical and responsible way. All of Twitter’s major product initiatives, serving the public conversation, ensuring its health, and amplifying diversified revenue streams integrally rely on machine learning. Our teams seek to provide a unified, forward-looking and fast developer experience to all machine learning engineers and researchers across Twitter.

In this role, we are hiring for 3 different focus areas within Cortex ML Platform:

  • ML Data: Enabling teams at Twitter to quickly experiment with new features, data management and data preparation. Providing tools to help customers understand their datasets, and providing better data engineering capabilities. Product: Feature Store.

  • ML Experimentation: Enabling teams to quickly test and iterate on their ML hypotheses via ML training capabilities, first class notebook solution and an ML pipeline solution. Providing ML infrastructure for offline workloads. Product: ML Notebooks, ML Training jobs, ML Pipelines, Kubeflow

  • ML Serving: Enabling teams at Twitter to serve ML models at high scale in production while ensuring performance, reliability and ease of use. Product: ML Prediction Services, Hosted Services

Some of our current projects include:

  • Augmenting ML Platform capabilities for data preparation, feature extraction, training and analysis with Google Cloud technologies like BigQuery and Dataflow

  • Developing technologies for advanced ML model training, e.g. BERT

  • Developing capabilities to hot swap hundreds of live production models

  • Model and Feature Coverage Analysis and Alerting

  • Developing ML pipeline solutions using Airflow, TFX, and Kubeflow

  • Developing a first class Notebook Environment for Twitter.

  • Distributed training on GPUs

  • Working with product teams to ensure ML algorithms are fair and free of unwanted bias.

The ML Platform group started in 2018 with the goal of creating a unified, standardized ML experience for all ML applications at Twitter. This has been a wildly successful journey with all of our products finding significant adoption across our customers. Our goal now is to increase the velocity of our customer engineers’ iteration and development cycles, by creating a more cohesive, integrated and managed experience. We are aiming for an order-of-magnitude productivity improvement within three years.

We’re a distributed group across New York, Boulder and San Francisco as main locations with several additional members working from other offices or remote locations. We’re paying close attention to hiring and retaining a diverse workforce and are proud of our people-first culture of open collaboration, transparency and psychological safety.

Job Description

 

What you’ll do

We’re hiring several engineers across all ML Platform teams to help create an industry-leading ML Platform. If building better ML tools and 10x productivity increases excite you, give us a call. Have a specialized skill or solved a related problem before? Come talk to us! Interested yet? We’ll decide the final team after a successful interview based on both your background and interests, as well as business needs across our org.

Who you are

Do you identify with the majority of the following traits? Yes? We believe they will make you successful in this role.

  • You’re fluent in concepts like design patterns, composition vs inheritance, commonality variability principle, primitives, etc.

  • You have a solid grasp on type systems and functional programming, ideally in Scala or Java and a dynamic language like Python.

  • You have educated opinions about what good library and API design in distributed systems looks like, both from formal study and from lived experience with past design decisions you made.

  • You’re able to articulate such opinions and concepts persuasively both in spoken and written form, with the intent to teach and enroll others, while remaining open to a constructive dialogue. Your thought process intrigues others with its clarity.

  • You take a principled, disciplined approach to software engineering, characterized by high volume and quality, solid testing practices, data-driven decision making, and a low rate of incidents in production.

  • You take pride in the developer experience, the ease of use, documentation and the durability of the products you build, together with the team.

  • You have a passion for machine learning and high-leverage infrastructure platforms.

  • You bring partners together across organizational and functional boundaries.

You’re motivated by shipping high-impact software to production and you have a track record of doing so.

 

 

Qualifications

By nature of the problem domain, we expect you to have experience in:

  • B.S., M.S. or Ph.D. degree in computer science or a related field or equivalent work experience.

  • 2+ years of experience in two of: Scala, Python, C++, Java

  • 2+ years of building and delivering working software through an iterative, agile process.

  • 2+ years of work experience in software engineering in the areas of distributed data processing, in high traffic or large data production environments.

  • M.S. or Ph.D. degree in computer science or a related field or equivalent work experience.

 

 

 

Additional Information

We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, color, ethnicity, ancestry, national origin, religion, sex, gender, gender identity, gender expression, sexual orientation, age, disability, veteran status, genetic information, marital status or any legally protected status.

San Francisco applicants: Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

All your information will be kept confidential according to EEO guidelines.

Privacy Policy