Big Data Engineer
- Contract
Job Description
• Big Data Software development /support services to help deliver outcomes sought in relation to the CCP stress testing project.
• On-premises 2nd Generation Data Platform based on Hadoop Ecosystem (Spark, Atlas etc.)
• Python for data ingestion pipelines.
• Airflow for manage scalable workflows.
• MS SQL and Hive for persistent data storage.
• Test Driven Development.
Typical development skills: Spark, Hive and Bedrock and knowledge or R
DevOps skills: Jenkins, Git
Delivery: Agile experience
Need Security Clearance
Additional Information
All your information will be t confidential according to EEO guidelines.