Senior Data Engineer

Senior Data Engineer

Salary- Up to £90k

Location- London (Hybrid)

This is an exciting opportunity to work with cutting-edge cloud-native data platforms (AWS, Azure, Databricks) and technologies like Spark, Delta, Kafka, and Terraform. You’ll collaborate with cross-functional teams, mentor junior engineers, and drive best practices in data engineering.

Key Responsibilities:

• Design & Build Data Pipelines: Develop and optimize batch and real-time data pipelines using Python, SQL, Spark, and Delta.
• Data Transformation & Processing: Implement ELT workflows to ingest, enrich, and publish data for analytics and AI/ML.
• Security & Compliance: Ensure data privacy, governance, and regulatory compliance.
• Performance Optimization: Tune queries and pipelines for cost-efficiency and speed.
• Monitoring & Observability: Integrate real-time monitoring to detect and resolve issues proactively.
• CI/CD & DevOps: Implement Terraform, GitHub, and DevOps pipelines for automation and deployment.
• Collaboration & Mentorship: Work closely with product owners, analysts, and data scientists, while mentoring junior engineers.
• Research & Innovation: Stay ahead of the curve with the latest data technologies and best practices.

What We’re Looking For:

• Strong experience in Python & SQL for Databricks and Spark (including Delta, Delta Live Tables, PyTest, Great Expectations).
• Expertise in batch and streaming data processing using Kafka, AWS Kinesis, or Azure Stream Analytics.
• Hands-on experience in cloud-native data platforms (AWS, Azure, or Databricks).
• Knowledge of data governance, privacy laws (e.g., GDPR), and security best practices.
• Experience with DevOps for data engineering – Terraform, GitHub, CI/CD.
• Agile mindset – experience working in Scrum/Kanban teams.
• Ability to mentor junior engineers and conduct code reviews (PR reviews).

Please reach out to- daniel.wexler@xcede.com for more information!

Unfortunately, they do not offer sponsorship