Senior Data Engineer - London

£700 - £800 per day
Ref: 35 Date Posted: Wednesday 19 Feb 2025

Azure Data Engineer

Reading – 1 day a week

£700-800 per day inside IR35

About the Role:
We are seeking a highly skilled Data Engineer with strong expertise in Microsoft Azure, Databricks and SQL to join our data engineering team. This role requires someone who is proactive, solution-oriented, and able to work autonomously within a busy team environment. You will play a crucial role in structuring and integrating core data systems, ensuring high-quality data flows into and out of our Master Data Management (MDM) platform.

This is a fantastic opportunity to join a business that has just secured $50 million in investment and are working on a huge utilities project of national importance. This consultancy is building an elite team of data professionals, already outpacing competitors in the data consultancy space.

 Contract or Permanent – Your Choice

  • Initial 6-month inside IR35 contract, but huge opportunity to stay for longer as this is a multi-year project with huge growth potential
  • Option to go perm if you want long-term stability

Key Responsibilities:

  • Azure Data Engineering: Utilize Azure services to manage data pipelines, storage, and processing. Work with Azure Data Lake, Synapse Analytics, and other relevant Azure tools.
  • MDM Integration: Ensure data is structured and processed correctly within the MDM platform to produce a golden set of data assets, which serve as an enhanced and high-quality version of source data.
  • Data Quality & Analytics: Implement data quality measurement processes, generating analytics to feed into dashboarding and reporting solutions.
  • Multi-Source Data Processing: Handle data from four primary sources, managing its ingestion through the data lake, transforming it in Prophecy, applying data quality rules, and enriching it with additional insights.
  • Data Pipeline Development: Build and maintain scalable ETL/ELT pipelines to facilitate efficient data flow across the platform.
  • Exception Reporting: Develop and maintain exception reporting to monitor inbound data quality and identify discrepancies.
  • Collaboration & Autonomy: Work efficiently within a busy team where full support may not always be available, requiring a proactive and forward-thinking approach to problem-solving.

Required Skills & Experience:

  • Proven experience as a Data Engineer, with a strong focus on MS Azure, Databricks and SQL.
  • Expertise in Azure Data Factory, Synapse, Data Lake, and Prophecy (or similar tools).
  • Experience working with Master Data Management (MDM) systems and structuring data for optimal integration.
  • Strong knowledge of ETL/ELT pipeline development and data transformation best practices.
  • Proficiency in data quality assurance, analytics, and exception handling.
  • Ability to work independently, problem-solve proactively, and drive improvements in data processes.
  • Understanding of property data assets and their integration from multiple sources is a plus.

Preferred Qualifications:

  • Certifications in Microsoft Azure (e.g., Azure Data Engineer Associate)
  • Experience with big data technologies and cloud-based data warehousing solutions.
  • Familiarity with Power BI or other visualization tools for data analytics.
Cookies on this website
We need to ensure that we give you the best experience on our website. If you wish you can restrict or block cookies by changing your browser setting. If you continue without changing your settings, we'll assume that you are happy to receive all cookies on this website.