ML Software Engineer -Cortex

• New York, NY, USA

• Employees can work remotely

• Full-time

Company Description

Who We Are

Twitter is what’s happening and what people are talking about right now. For us, life's not about a job, it's about purpose. We believe real change starts with conversation. Here, your voice matters. Come as you are and together we'll do what's right (not what's easy) to serve the public conversation.

At Cortex our purpose is to improve Twitter by enabling advanced and ethical AI.

Within Cortex, the ML Platform group was formed to accelerate the impact of ML through tools and infrastructure, in an ethical and responsible way. All of Twitter’s major product initiatives, serving the public conversation, ensuring its health, and amplifying diversified revenue streams integrally rely on machine learning. Our teams seek to provide a unified, forward-looking and fast developer experience to all machine learning engineers and researchers across Twitter.

In this role, we are hiring for 3 different focus areas within Cortex ML Platform:

• ML Data: Enabling teams at Twitter to quickly experiment with new features, data management and data preparation. Providing tools to help customers understand their datasets, and providing better data engineering capabilities. Product: Feature Store.

• ML Experimentation: Enabling teams to quickly test and iterate on their ML hypotheses via ML training capabilities, first class notebook solution and an ML pipeline solution. Providing ML infrastructure for offline workloads. Product: ML Notebooks, ML Training jobs, ML Pipelines, Kubeflow

• ML Serving: Enabling teams at Twitter to serve ML models at high scale in production while ensuring performance, reliability and ease of use. Product: ML Prediction Services, Hosted Services

Some of our current projects include:

• Augmenting ML Platform capabilities for data preparation, feature extraction, training and analysis with Google Cloud technologies like BigQuery and Dataflow

• Developing technologies for advanced ML model training, e.g. BERT

• Developing capabilities to hot swap hundreds of live production models

• Model and Feature Coverage Analysis and Alerting

• Developing ML pipeline solutions using Airflow, TFX, and Kubeflow

• Developing a first class Notebook Environment for Twitter.

• Distributed training on GPUs

The ML Platform group started in 2018 with the goal of creating a unified, standardized ML experience for all ML applications at Twitter. This has been a wildly successful journey with all of our products finding significant adoption across our customers. Our goal now is to increase the velocity of our customer engineers’ iteration and development cycles, by creating a more cohesive, integrated and managed experience. We are aiming for an order-of-magnitude productivity improvement within three years.

We’re a distributed group across New York, Boulder and San Francisco as main locations with several additional members working from other offices or remote locations. We’re paying close attention to hiring and retaining a diverse workforce and are proud of our people-first culture of open collaboration, transparency and psychological safety.

What you’ll do

We’re hiring several ML engineers across all ML Platform teams to help create an industry-leading ML Platform. If building better ML tools and 10x productivity increases excite you, give us a call. Have a specialized skill or solved a related problem before? Come talk to us! Interested yet? We’ll decide the final team after a successful interview based on both your background and interests, as well as business needs across the four teams.

Who you are

Do you identify with the majority of the following traits? Yes? We believe they will make you successful in this role.

• A passion for machine learning and developer tools.

• Motivated by delivering impactful products that accelerate our customers' workflows.

• An innovator with listening skills, empathy and a knack for discovering “product-market fit” for seed-stage ideas and delivering strong outcomes.

• You believe in software quality and set examples by writing robust interfaces, considering design principles and applying sound testing practices.

• A systematic approach toward project management and dealing with ambiguity (such as formulating and testing product hypotheses).

• A track record of shipping working software fast and reliably.

You have contributed to or working knowledge in two or more of the following:

• Open-source ML frameworks (e.g. Tensorflow, TFX, PyTorch)

• Cloud technology stacks (e.g. GCP or AWS and their product offerings)

• ML pipelines and their orchestration

• Jupyter notebooks

• Distributed data processing in Hadoop, Spark, BigQuery, or Apache Beam

• Modeling, model architecture or optimization

• Data and feature engineering

• Distributed training and/or GPU-based training and inference

• Experience with distributed systems, their performance optimization and improving their resilience

By nature of the problem domain, we expect you to also have experience in:

• B.S., M.S. or Ph.D. degree in computer science or a related field or equivalent work experience.

• 2+ years of experience in two of: Scala, Python, C++, Java

• 2+ years of building and delivering working software through an iterative, agile process.

• 2+ years of experience with ML problems and tools.

Additional Information

All your information will be kept confidential according to EEO guidelines.

We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, color, ethnicity, ancestry, national origin, religion, sex, gender, gender identity, gender expression, sexual orientation, age, disability, veteran status, genetic information, marital status or any legally protected status.