The Twitter Service organization creates meaningful connections with our customers that promote healthy exchange in the public conversation. We do this by developing customer support solutions at scale in a rapidly changing and global environment. Whether we're optimizing operations, guarding the ability of customers to safely use Twitter through investigation of and  enforcement against those who violate our terms, or driving content and support channel strategies, we're with our customers every step of the way.

The Human Computation team’s mission is to streamline production of high-quality labeled data and context-based insights & measurement, in support of Machine Learning, offline metrics, and ALL other human-in-the-loop initiatives across Twitter. Our work directly impacts the quality and consumption of our most popular products, helping to prevent the abuse and misuse of Twitter, and delighting our ever expanding global user base. 

We’re looking for a dynamic Technical Program Manager who has a proven track record of designing and driving high-impact high-visibility Engineering and Engineering support programs. The right person would have experience driving projects/initiatives for Human Computation/ Data Annotation and working cross-functionally (across Engineering, Product, Research, Legal, Operations) in support of large company objectives. The right person would also have experience working with multiple global staffing vendors, managing scaled contingent business resources, and effectively navigating a challenging, fast-paced, and dynamic daily working environment.

Responsibilities:

• Independently drive global team goals, roadmap, priorities, and initiatives to ensure support for Machine Learning projects and accurate qualitative measurements.

• Drive the development of various technical solutions that are essential to running Human Computation effectively and efficiently, at scale. Manage internal and external tooling backlogs.

• Identify opportunities, define, scope, and get buy-in across cross-functional partners for projects that can leverage human evaluations to improve existing Machine Learning models and research in support of opportunity sizing new features/understanding usage behavior of new launches. 

• Work directly with Engineers, Product Managers, Researchers, and Data Scientists to define, scope, build, and execute human evaluations in support of different areas of the product.

• Interface with cross-functional stakeholders across Operations, Policy, Legal, Procurement, and external vendors to produce quarterly roadmaps in support of top-level company goals.

• Write clear and concise technical documentation and obtain buy-in from leadership across a wide variety of stakeholder teams. 

• Review and provide feedback on technical design documents from EPDR, with respect to Human Computation’s scope. 

• Train, oversee, and own accountability for an international and designated contingent team, large-scale and multilingual liquid workforce, and a crowdsourced workforce.

• Effectively determine contingent staffing needs; implement effective roles, responsibilities, and global team structure.

• Review and/or process sensitive and potentially disturbing content including but not limited to pornographic images, child pornography, vulgar language, hate speech, violent threats, and other graphic images.

Minimum Qualifications

• BA or BS in Cognitive Science, Computer Science/Engineering, or equivalent work experience in a related field. 

• Over 5 years of experience driving large scale Engineering & Engineering-support programs. 

• Over 3 year direct experience with Human Computation, crowdsourcing, or experimental design.

• Over 2 year experience with HTML, XML, CSS, JavaScript, and SQL or specialized database query languages.

• Proven ability to identify and resolve systemic issues in both software and people systems.

• Excellent written and verbal communication skills, organizational skills, and highly developed interpersonal skills.

• Active, passionate Twitter user with a strong interest in social media. Passionate interest in Twitter’s products and mission, and about the role machine learning could play towards advancing them.

Prefered Qualifications

• MA, MS, or PhD Cognitive Science, Computer Science/Engineering, or equivalent work experience in a related field. 

• Over 2 years experience working with large data-sets & exposure to basics of Machine Learning.

• Over 1 year experience with R/Python and other proprietary data extraction and analysis tools. 

• Over 2 years experience designing and implementing research experiments.

• 3-5 years experience managing, directing, and evaluating the work of others.

• Proficiency in one or more non-English languages.

We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, color, ethnicity, ancestry, national origin, religion, sex, gender, gender identity, gender expression, sexual orientation, age, disability, veteran status, genetic information, marital status or any legally protected status. All of your information will be kept confidential according to EEO guidelines.

Notice (Colorado Equal Pay for Equal Work Act)

The expected salary range for this role to be performed in Colorado is USD$128,000.00 - USD$179,000.00. Starting pay for the successful applicant will depend on a variety of job-related factors, which may include education, training, experience, location, business needs, or market demands. This range may be modified in the future.

This job is also eligible for participation in Twitter’s Performance Bonus Plan and Equity Incentive Plan subject to the terms of the applicable plans and policies.

Twitter offers a wide range of benefits to U.S.-based employees, including medical, dental, and vision insurance, 401(k) program with employer match, generous time off for vacation, sick time, and parental leave. Twitter's benefits prioritize employee wellness and progressive support to our diverse workforce.