Databricks Engineer

  • Full-time
  • Department: Digital Development & Architecture

Company Description

Just this month winners of the IT Company of the Year in Northern Ireland and now we were proud to be named as a 'Best Large Workplace'. Read more on our remarkable UK journey at: http://bit.ly/2LvSDkR #gptw #UKBestWorkplaces 

We pledge "to prove IT can make a real difference to our customer's businesses". We work hard to ensure we understand what our customers need from their technology solutions and then we deliver. 

We are an award-winning company who provide world class customer service; we think big and we hire great people. Version 1 are more than just another IT services company - we are leaders in implementing and supporting Oracle, Microsoft and AWS technologies. 

Invest in us and we’ll invest in you; if you are driven, committed and up for a challenge, we want to meet you. 

Job Description

With increasingly vast seas of digital information, smart organisations can today do things that were up to now impossible; spot unseen business trends, prevent diseases, make our roads safer and so on. At Version 1 our Mission is to prove IT can make a real difference. We prove this every day and due to continued expansion, we’re searching for like-minded individuals to help us ‘take it to the next level’.

This is an exciting opportunity for an experienced developer of large scale data solutions. You will join a team delivering a transformative cloud hosted data platform for a key Version 1 customer. 

The ideal candidate will have a proven track record in implementing data ingestion and transformation pipelines for large scale organisations. We are seeking someone with deep technical skills in a variety of technologies to play an important role in developing and delivering early proofs of concept and production implementation.

You will good exerience in building solutions using a variety of open source tools a Microsoft Azure services and a proven track record in delivering high quality work to tight deadlines.


Your main responsibilities will be:

  • Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Databricks
  • Delivering and presenting proofs of concept to of key technology components to project stakeholders.
  • Developing scalable and re-usable frameworks for ingesting of geospatial data sets
  • Integrating the end to end data pipleline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
  • Working with event based / streaming technologies to ingest and process data
  • Working with other members of the project team to support delivery of additional project components (API interfaces, Search)
  • Evaluating the performance and applicability of multiple tools against customer requirements
  • Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.

Qualifications

You will have:

  • Strong knowledge of Data Management principles
  • Experience in building ETL / data warehouse transformation processes
  • Direct experience of building data piplines using Azure Data Factory and Apache Spark (preferably Databricks).
  • Experience using geospatial frameworks on Apache Spark and associated design and development patterns
  • Microsoft Azure Big Data Architecture certification.
  • Hands on experience designing and delivering solutions using the Azure Data Analytics platform (Cortana Intelligence Platform) including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
  • Experience with Apache Kafka / Nifi for use with streaming data / event-based data
  • Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)
  • Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
  • Experience working with structured and unstructured data including imaging & geospatial data.
  • Experience working in a Dev/Ops environment with tools such as Microsoft Visual Studio Team Services, Chef, Puppet or Terraform

 

… and you’ll be able to provide evidence and examples of where you have demonstrated these skills in the past. If so, we’d love to speak with you.

Additional Information

This is an opportunity to join one of the biggest Big-Data Projects in the UK and join a growing Microsoft Azure Capability team. As we grow and build this expert team in BI Cloud we are looking ideally for permanent applications but will consider contractors.

Roles will be based on customer site in Southampton. Roles are envisaged to be full-time on-site. Expenses will be covered. London office base thereafter. Limited remote working might be possible but should not be assumed.

Suzanne Whelan,
Version 1 Talent Acquisition Team Manager
Tel: 00353872200625

We are an equal opportunities employer. Please refer to our Diversity & Inclusion statement on the website.

Privacy Policy