Senior Machine Learning Platform Engineer
- New York, NY, USA
- Employees can work remotely
Who We Are
Twitter is what’s happening and what people are talking about right now. For us, life's not about a job, it's about purpose. We believe real change starts with conversation. Here, your voice matters. Come as you are and together we'll do what's right (not what's easy) to serve the public conversation.
At Cortex our purpose is to improve Twitter by enabling advanced and ethical AI.
Within Cortex, the ML Platform group was formed to accelerate the impact of ML through tools and infrastructure, in an ethical and responsible way. All of Twitter’s major product initiatives, serving the public conversation, ensuring its health, and amplifying diversified revenue streams integrally rely on machine learning. Our teams seek to provide a unified, forward-looking and fast developer experience to all machine learning engineers and researchers across Twitter.
In this role, we are hiring for 3 different focus areas within Cortex ML Platform:
ML Data: Enabling teams at Twitter to quickly experiment with new features, data management and data preparation. Providing tools to help customers understand their datasets, and providing better data engineering capabilities. Product: Feature Store.
ML Experimentation: Enabling teams to quickly test and iterate on their ML hypotheses via ML training capabilities, first class notebook solution and an ML pipeline solution. Providing ML infrastructure for offline workloads. Product: ML Notebooks, ML Training jobs, ML Pipelines, Kubeflow
ML Serving: Enabling teams at Twitter to serve ML models at high scale in production while ensuring performance, reliability and ease of use. Product: ML Prediction Services, Hosted Services
Some of our current projects include:
Augmenting ML Platform capabilities for data preparation, feature extraction, training and analysis with Google Cloud technologies like BigQuery and Dataflow
Developing technologies for advanced ML model training, e.g. BERT
Developing capabilities to hot swap hundreds of live production models
Model and Feature Coverage Analysis and Alerting
Developing ML pipeline solutions using Airflow, TFX, and Kubeflow
Developing a first class Notebook Environment for Twitter.
Distributed training on GPUs
Working with product teams to ensure ML algorithms are fair and free of unwanted bias.
The ML Platform group started in 2018 with the goal of creating a unified, standardized ML experience for all ML applications at Twitter. This has been a wildly successful journey with all of our products finding significant adoption across our customers. Our goal now is to increase the velocity of our customer engineers’ iteration and development cycles, by creating a more cohesive, integrated and managed experience. We are aiming for an order-of-magnitude productivity improvement within three years.
We’re a distributed group across New York, Boulder and San Francisco as main locations with several additional members working from other offices or remote locations. We’re paying close attention to hiring and retaining a diverse workforce and are proud of our people-first culture of open collaboration, transparency and psychological safety.
What you’ll do
We’re hiring several ML engineers across all ML Platform teams to help create an industry-leading ML Platform. If building better ML tools and 10x productivity increases excite you, give us a call. Have a specialized skill or solved a related problem before? Come talk to us! Interested yet? We’ll decide the final team after a successful interview based on both your background and interests, as well as business needs across the four teams.
Who you are
Do you identify with the majority of the following traits? Yes? We believe they will make you successful in this role.
A passion for machine learning and developer tools.
Motivated by delivering impactful products that accelerate our customers' workflows.
An innovator with listening skills, empathy and a knack for discovering “product-market fit” for seed-stage ideas and delivering strong outcomes.
You believe in software quality and set examples by writing robust interfaces, considering design principles and applying sound testing practices.
A systematic approach toward project management and dealing with ambiguity (such as formulating and testing product hypotheses).
A track record of shipping working software fast and reliably.
You bring partners together across organizational and functional boundaries.
You’re able to articulate a clear vision and enroll the team and partners into it, both in spoken and written form, while remaining open to a constructive dialogue.
You multiply the effect of contributors by inspiring and growing them on and off the team across different levels of seniority, skills and geographical boundaries.
You have contributed to or working knowledge in three or more of the following:
Open-source ML frameworks (e.g. Tensorflow, TFX, PyTorch)
Cloud technology stacks (e.g. GCP or AWS and their product offerings)
ML pipelines and their orchestration
Distributed data processing in Hadoop, Spark, BigQuery, or Apache Beam
Modeling, model architecture or optimization
Data and feature engineering
Distributed training and/or GPU-based training and inference
Experience with distributed run-time systems, their performance optimization and improving their resilience
B.S., M.S. or Ph.D. degree in computer science or a related field or equivalent work experience.
3+ years of experience in two of: Scala, Python, C++, Java
5+ years of building and delivering working software through an iterative, agile process.
3+ years of demonstrated facility with ML problems and tools.
2+ years in Sr engineering capacity with demonstrated leadership skills. (This role does not include people management responsibilities.)
All your information will be kept confidential according to EEO guidelines.
We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, color, ethnicity, ancestry, national origin, religion, sex, gender, gender identity, gender expression, sexual orientation, age, disability, veteran status, genetic information, marital status or any legally protected status.