Data Engineer
- Full-time
- Engagement type: Permanent - Employee
- Department: Digital Technology
Job Description
About Us
If you are an engineer who likes to push the boundaries of what’s possible but also likes to actually get things done, then Vector might just be the perfect place for you. We’ve been going for 100 years which means we have the scale and support to know how to make things happen and deliver critical infrastructure projects across Auckland.
Right now, the engineering challenges of creating a fit for purpose electricity network to serve a low carbon economy mean you’ll need to bring your problem solving, innovative best to work here. So whilst we’re 100 years old, it can feel like we’re just beginning.
About the role
As a Data Engineer in our DevOps team, you will be responsible for the design and implementation of Vector's cloud-based data solutions. You will design, develop, and release Behavior Driven Development (BDD) and Test-Driven Development (TDD) data solutions to enable our data engineering, BI and data science teams to perform data transformation and analytical activities. You will design, build and/or implement:
- Data flow and processing pipelines for ingestion into AWS data lake using cloud-based tools and services such as Apache Spark, AWS Glue, EMR and Lambda
- Unit tests for the data processing pipelines as well as CI/CD pipelines in AWS cloud
- Data integration between machine learning models and other data processing pipelines using cloud-based services such as AWS Sage Maker
From time to time, you will also be required to provide DevOps support to the Enterprise Data Warehouse and BI Platform team as well as 24x7 on call support as and when is needed.
Who we are looking for
Ideally you’re a seasoned Developer with at least 5 years experience in a Data Engineer, Software Developer or BI Developer role and have worked in an agile environment. You’ve got experience and are capable of leading AWS cloud-based data engineering, development, along with hands on and technical experience with modern data and analytics services and technologies, such as:
- Snowflake
- Distributed Data Processing Framework such as Apache Spark
- AWS data lake technology stack (S3, Glue, Athena).
- Data integration services (AWS EMR, Lambda)
- CI/CD pipeline development (CodePipeline, Terraform)
- Programming languages (Python, Scala and/or Java)
- Unit testing frameworks
- Machine Learning service such as AWS SageMaker
- Experience providing on-Call support.
- Experience with Agile methodologies
- Experience working in a DevOps or DataOps environment
- Comprehensive knowledge of Behaviour-Driven Development and Test-Driven Development.
In addition, you’ve worked on large-scale data engineering projects including requirements gathering, design and have working knowledge of Data Warehouse and Business Intelligence stack (SQL, ETL, database and dimensional modelling).
What we can offer you
We love working together to create a new energy future. If you want to be part of an organisation with a real and tangible purpose, Vector could be the place for you. We offer:
- A fun, flexible and stimulating work environment, full of interesting work being done by smart and innovative people.
- Flexibility, including the choice of working from home a couple of days a week, and flexible start and finish times so you can balance work and life.
- Health and wellbeing support
- Great benefits that reflect the importance of family.
Vector engages a workforce made up of people and teams with diverse skills, values, backgrounds, and experiences. We respect and value people who are different from each other. We are proud of what we have achieved so far -
- First corporate Living Wage Accredited employer
- First Accessibility Tick employer
- Rainbow Tick accredited
- Winner of three gongs at the 2019 Diversity Awards NZ, including the Supreme Award in 2019 and 2020.
If you require assistance or accessibility support from us during the application and selection process, please do get in touch with us on 09 978 7636.
Interviews may be held as applications are received, so do not delay - apply today!