Who is HelloKindred?

HelloKindred are specialists in staffing marketing, creative and technology roles, offering a range of talent solutions that can be delivered on-site, remotely or hybrid.

Our vision is to make work accessible and people’s lives better. We do this by disrupting traditional employment barriers – connecting ambitious talent to flexible opportunities with trusted brands.

Anticipated Contract End Date/Length: Approximately 4.5 months
Work Set Up: Hybrid (60% office, 40% remote)
Clearance Required: BPSS

Our client in the Information Technology and Services industry is looking for a Platform Architect to enhance and optimise an enterprise Data Platform through hands-on Azure Databricks architecture, engineering, operational support, and platform optimisation. The role will focus on enabling Databricks Serverless capabilities, strengthening FinOps practices and platform governance, and enhancing the Databricks Discovery Zone to support migration and consolidation of POSIT/RStudio workloads.

What you will do:

• Assess existing workloads and select appropriate compute models, including Serverless, Jobs Compute, Interactive Compute, and Classic Clusters, based on workload, SLA, performance, utilisation, and cost.
• Configure and implement Databricks Serverless capabilities across notebooks, jobs, SQL workloads, analytical processing, and data pipelines.
• Develop workload placement standards and guidance, identifying scenarios where Serverless may not provide the most cost-effective solution.
• Implement compute policies, autoscaling, quotas, budget controls, and operational guardrails.
• Monitor platform performance and costs, identifying oversized, underutilised, idle, or inefficient resources.
• Define and implement a practical FinOps operating model covering ownership, accountability, environments, projects, applications, cost centres, and teams.
• Establish mandatory tagging standards and integrate automated validation into CI/CD pipelines.
• Deliver granular cost attribution and reporting across workspaces, projects, applications, workloads, and business teams.
• Configure budgets, spend thresholds, alerts, and usage monitoring to proactively manage platform costs.
• Analyse usage and billing data to identify cost anomalies, inefficient workloads, excessive storage, unnecessary data movement, and underutilised resources.
• Enhance the Databricks Discovery Zone to support migration and modernisation of analytics and data science workloads currently running on POSIT/RStudio.
• Enable secure API and external system integrations, external data ingestion, BI and reporting connectivity, scheduling and orchestration, local IDE-based development, and LLM and AI integration.
• Define reusable onboarding, migration, and delivery patterns that reduce technology sprawl while improving platform security, supportability, and delivery speed.
• Design, build, and optimise scalable data ingestion and transformation solutions using Python, PySpark, SQL, and Delta Lake.
• Implement batch and incremental processing patterns, including CDC, schema evolution, reconciliation, data quality controls, and error handling.
• Develop reusable integration frameworks for REST APIs, SaaS platforms, databases, files, object storage, document repositories, enterprise systems, and external data sources.
• Implement secure authentication, secrets management, and credential handling practices.
• Deliver end-to-end data flows from source ingestion through governed and curated data layers supporting analytics, BI, machine learning, and application consumption.
• Support production platforms and drive continuous optimisation across architecture, engineering, implementation, and operational support.
• Document standards, patterns, operational procedures, and architectural decisions for technical and business stakeholders.

• Possess deep hands-on experience implementing, administering, and troubleshooting enterprise-scale Azure Databricks platforms in production.
• Have strong expertise in Databricks Serverless architecture, workload placement, compute optimisation, Databricks SQL, Delta Lake, and query and workload performance optimisation.
• Have strong knowledge of Azure networking, identity, security, monitoring, secrets management, and private connectivity.
• Possess advanced Python, PySpark, and SQL development skills within enterprise-scale data engineering environments.
• Have experience with jobs, workflows, orchestration, incremental processing, CDC, data quality, reconciliation, and operational monitoring.
• Have experience implementing FinOps practices, tagging strategies, cost attribution, monitoring, budget management, and cost optimisation.
• Have experience integrating REST APIs and external data sources within enterprise data platforms.
• Demonstrate strong understanding of platform governance, security, compliance, operational support, and controlled delivery within complex or regulated organisations.
• Collaborate effectively with data engineers, data scientists, architects, security teams, platform teams, and business stakeholders.
• Demonstrate the ability to work across architecture, engineering, implementation, optimisation, and operational support without dependence on specialist teams.
• Communicate effectively and produce clear technical documentation, standards, operational procedures, and architectural decisions.
• Have experience migrating analytics and data science workloads from POSIT/RStudio to Databricks; highly desirable.
• Have experience converting R-based workloads and libraries to modern data platform solutions; highly desirable.
• Possess AI/ML experience, including LLM integration, RAG/vector retrieval, model serving, and model lifecycle management; highly desirable.
• Have experience delivering large-scale platform transformation programmes; highly desirable.
• Have experience operating within highly regulated or complex enterprise environments; highly desirable.

Candidates must be legally authorized to live and work in the country where the position is based, without requiring employer sponsorship.

HelloKindred is committed to fair, transparent, and inclusive hiring practices. We assess candidates based on skills, experience, and role-related requirements.

We appreciate your interest in this opportunity. While we review every application carefully, only candidates selected for an interview will be contacted.

HelloKindred is an equal opportunity employer. We welcome applicants of all backgrounds and do not discriminate on the basis of race, colour, religion, sex, gender identity or expression, sexual orientation, age, national origin, disability, veteran status, or any other protected characteristic under applicable law.