Data Engineer
- Full-time
Company Description
A best-in-class city that attracts best-in-class talent, Philadelphia is an incredible place to build a career. From our thriving arts scene and rich history to our culture of passion and grit, there are countless reasons to love living and working here. With a workforce of over 30,000 people, and more than 1,000 different job categories, the City of Philadelphia offers boundless opportunities to make an impact.
As an employer, the City of Philadelphia values inclusion, integrity, innovation, empowerment, and hard work above all else. We offer a vibrant work environment, comprehensive health care and benefits, and the experience you need to grow and excel. If you’re interested in working with a passionate team of people who care about the future of Philadelphia, start here.
What We Offer:
Impact - The work you do here matters to millions.
Growth - Philadelphia is growing, why not grow with it?
Diversity & Inclusion - Find a career in a place where everyone belongs
Benefits - We care about your well being
Job Description
OIT provides leadership and expertise on the design, integration, and management of data across city government. Data is key to municipal operations and business intelligence. The City of Philadelphia shares hundreds of datasets with the public and between departments: Open Data - opendataphilly.org, Metadata – metadata.phila.gov. As a result, residents can find information about assets, services, and conditions in their neighborhood. Civic associations can lookup landlords and see if they have violations before supporting their zoning variance. Voters can trust that campaign finance reports and city contracts are posted online for anyone to see.
The City’s applications that support bill paying, filling permits, and all day to day transactions are reliant on OIT’s ability to effectively integrate and ingest data among various systems in multiple formats. This work is done not just for the benefit of government operations, but for the public. Examples of OIT’s commitment to serving the public through the effective and innovative integration of City open data include:
https://Atlas.phila.gov
https://Openmaps.phila.gov
https://streetsmartphl.phila.gov/
www.phila.gov/solarmap
www.phila.gov/stressmap
OIT’s teams apply agile development techniques and are staffed by engaged, diverse, talented, and committed analysts, application developers, and engineers eager to do their best professional work to improve City processes and policies to better serve Philadelphia residents.
Data Engineering at the City is a unique opportunity for meaningful, exciting work and professional development using state-of-the art technologies and software development best practices. The data engineer will develop an intimate understanding of the City’s diverse data and contribute to improving the City’s data engineering infrastructure, pipelines, models, and accessibility for the public.
The data engineer will join an engineering team responsible for developing and maintaining data integrations between various departmental agencies through OIT’s central data warehouse (DataBridge), publishing APIs, and managing the underlying infrastructure. The team also manages the City’s Address Information System (AIS) – a cloud-based, address search API leveraging DataBridge that enables developers to build applications that pull a myriad of the City’s information and serve as subject matter experts for efforts across departments related to improving address, parcel, and other integrated data.
Primary tools used include Python, Bash, SQL, GIS, Docker, Terraform, Apache Airflow, Postgres, AWS, GitHub, Software AG Web Methods and SAS API providers like ArcGIS Online and CARTO.
Essential Functions
Help City departments share data with other departments and with the public by:
Working with business partners of varying technical ability to understand how their data is produced, stored, and updated
Setting up automated extract, transform, and load (ETL) workflows, including standardization and enriching of datasets that pass through DataBridge
Improving the City’s “platform” used to share data (reusable components, scheduling, logging, centralized storage, etc.) so that workflows are easier to write and maintain
Write descriptive and accurate technical documentation of systems or applications
Other duties as assigned
Competencies, Knowledge, Skills and Abilities
Ability to:
Reason about, and work, with data across a variety of potentially unfamiliar domains
Apply creative problem-solving when considering how to best address technical challenges
Communicate technical nuances in plain language with partners of varying technical background
Write clearly and concisely with organized structure
Write descriptive, technical policies and/or documentation
Manage time effectively between various projects and responsibilities
Knowledge and experience with:
Building and maintaining data pipelines / ETL’s
Building software with Python (intermediate/advanced), Bash and Docker
SQL – intermediate/advanced
Databases, Oracle/SQL Server, intermediate to advanced experience with Postgres preferred
Continuous Integration/Deployment
Classifying data sensitivity and managing secure data environments
Working in a hybrid environment (on-premises and Amazon Web Services)
Qualifications
At least 3 years’ experience in any of the following areas preferred:
Developing software
Data engineering/integration
Architecting and maintaining relational databases
Moving a variety of data types between a variety of systems (ie. Flat file, FTP server, AWS S3, Relational Databases, APIs)
Transforming and enriching data (ie. De-normalizing, geocoding, re-projecting, anonymizing)
Establishing ETL workflows, processes, and documentation for organizations
Completion of a bachelor’s degree program at an accredited college or university, which has included major course work in computer science, information science, geography, GIS or a closely related field.
A Master’s Degree in computer science, information science, geography, GIS or a closely related field can substitute for one year of professional experience.
Education requirements can be waived by the hiring committee for applicants that are found to be exceptionally strong in the technical areas described above.
Additional Information
Salary Not to Exceed: $85,000
Please submit a resume and cover letter with your application.
Did you know?
- We are a Public Service Loan Forgiveness Program qualified employer
- 25% tuition discount program for City employees (and sometimes spouses and dependents as well) in partnership with area colleges and universities
- We offer Comprehensive health coverage for employees and their eligible dependents
- Our wellness program offers eligibility into the discounted medical plan
- Employees receive paid vacation, sick leave, and holidays
- Generous retirement savings options are available
*The successful candidate must be a city of Philadelphia resident within six months of hire
Please note that effective September 1, 2021 the City of Philadelphia is requiring all new employees to present proof of vaccination against COVID-19.
The City of Philadelphia is an Equal Opportunity employer and does not permit discrimination based on race, ethnicity, color, sex, sexual orientation, gender identity, religion, national origin, ancestry, age, disability, marital status, source of income, familial status, genetic information or domestic or sexual violence victim status. If you believe you were discriminated against, call the Philadelphia Commission on Human Relations at 215-686-4670 or send an email to faqpchr @phila.gov. For more information, go to: Human Relations Website: http://www.phila.gov/humanrelations/Pages/default.aspx