3-6 years of experience as a Data Engineer (or similar). London- hybrid- 3 days per week on-site - 6 Months + UMBRELLA only- Inside IR35 - Key Responsibilities * Design, develop, and maintain ETL/ELT pipelines using Airflow for orchestration and scheduling. * Build and manage data transformation workflows in DBT running on Databricks. * Optimize data models in Delta Lake for performance, scalability, and cost efficiency. * Collaborate with analytics, BI, and data science teams to deliver clean, reliable datasets. * Implement data quality checks (dbt tests, monitoring) and ensure governance standards. * Manage and monitor Databricks clusters & SQL Warehouses to support workloads. * Contribute to CI/CD practices for data pipelines (version control, testing, deployments). * Troubleshoot pipeline failures, performance bottlenecks, and scaling challenges.
more