Azure Data Engineer
Reading – 1 day a week
£700-800 per day inside IR35
About the Role:
We are seeking a highly skilled Data Engineer with strong expertise in Microsoft Azure, Databricks and SQL to join our data engineering team. This role requires someone who is proactive, solution-oriented, and able to work autonomously within a busy team environment. You will play a crucial role in structuring and integrating core data systems, ensuring high-quality data flows into and out of our Master Data Management (MDM) platform.
This is a fantastic opportunity to join a business that has just secured $50 million in investment and are working on a huge utilities project of national importance. This consultancy is building an elite team of data professionals, already outpacing competitors in the data consultancy space.
Contract or Permanent – Your Choice
- Initial 6-month inside IR35 contract, but huge opportunity to stay for longer as this is a multi-year project with huge growth potential
- Option to go perm if you want long-term stability
Key Responsibilities:
- Azure Data Engineering: Utilize Azure services to manage data pipelines, storage, and processing. Work with Azure Data Lake, Synapse Analytics, and other relevant Azure tools.
- MDM Integration: Ensure data is structured and processed correctly within the MDM platform to produce a golden set of data assets, which serve as an enhanced and high-quality version of source data.
- Data Quality & Analytics: Implement data quality measurement processes, generating analytics to feed into dashboarding and reporting solutions.
- Multi-Source Data Processing: Handle data from four primary sources, managing its ingestion through the data lake, transforming it in Prophecy, applying data quality rules, and enriching it with additional insights.
- Data Pipeline Development: Build and maintain scalable ETL/ELT pipelines to facilitate efficient data flow across the platform.
- Exception Reporting: Develop and maintain exception reporting to monitor inbound data quality and identify discrepancies.
- Collaboration & Autonomy: Work efficiently within a busy team where full support may not always be available, requiring a proactive and forward-thinking approach to problem-solving.
Required Skills & Experience:
- Proven experience as a Data Engineer, with a strong focus on MS Azure, Databricks and SQL.
- Expertise in Azure Data Factory, Synapse, Data Lake, and Prophecy (or similar tools).
- Experience working with Master Data Management (MDM) systems and structuring data for optimal integration.
- Strong knowledge of ETL/ELT pipeline development and data transformation best practices.
- Proficiency in data quality assurance, analytics, and exception handling.
- Ability to work independently, problem-solve proactively, and drive improvements in data processes.
- Understanding of property data assets and their integration from multiple sources is a plus.
Preferred Qualifications:
- Certifications in Microsoft Azure (e.g., Azure Data Engineer Associate)
- Experience with big data technologies and cloud-based data warehousing solutions.
- Familiarity with Power BI or other visualization tools for data analytics.