Senior Data Engineer

Pearson Carter

Senior Data Engineer / Consultant

Position Summary

The Senior Data Engineer / Consultant is responsible for leading the delivery of data engineering engagements, ensuring high-quality technical outcomes, strong client collaboration, and effective support of delivery teams.

This role combines advanced data engineering expertise with consulting leadership. It is highly hands-on and delivery-focused, requiring autonomous execution and a strong billable contribution. The role also includes mentoring team members, contributing to solution design, and maintaining high engineering standards.

Key Responsibilities

Data Engineering

  • Design, build, and maintain data pipelines and integrations across data lakes, lakehouses, and data warehouses
  • Develop and optimise data transformations for reporting, analytics, and AI/ML use cases
  • Apply data modelling techniques, including star and snowflake schemas
  • Write, test, and maintain high-quality, well-documented, and performant code
  • Troubleshoot and resolve technical issues efficiently
  • Implement modern engineering patterns and well-architected principles
  • Work with Databricks and related tools (e.g., Delta Live Tables, workflows)

Infrastructure & DevOps

  • Lead the use of source control systems (e.g., Git, Azure DevOps)
  • Design and maintain CI/CD pipelines and environment management processes
  • Implement automated testing and data validation for pipelines
  • Conduct performance testing and optimisation of data solutions

Testing & Quality Assurance

  • Design and review unit and integration tests
  • Ensure data accuracy, reliability, and performance through robust testing
  • Maintain documentation of issues and resolutions
  • Uphold engineering quality standards and best practices
  • Contribute to improving internal frameworks, templates, and processes

Consulting & Client Engagement

  • Act as the primary engineering contact for client engagements
  • Lead technical discussions and contribute to requirements gathering
  • Translate business needs into technical solutions and documentation
  • Communicate progress, risks, and blockers to stakeholders
  • Support project estimation, scoping, and planning activities

Leadership & Mentoring

  • Lead delivery workstreams and ensure high-quality outputs
  • Mentor and coach junior and intermediate team members
  • Conduct peer reviews and enforce engineering standards
  • Collaborate with architects on solution design

Continuous Improvement

  • Contribute to internal knowledge sharing and team learning
  • Stay up to date with modern data technologies and practices (AI/ML, cloud, visualisation)
  • Support development of internal processes, templates, and documentation
  • Engage in training and certification aligned with the organisation’s technology stack

Work Planning & Delivery

  • Manage tasks and deliverables to meet timelines and quality expectations
  • Provide accurate effort estimates and project input
  • Maintain clear and timely reporting of progress and dependencies
  • Participate in planning, review, and retrospective sessions

Health & Safety

  • Adhere to workplace health and safety policies
  • Ensure safe work practices for self and others

Key Outcomes

  • Delivery of scalable, maintainable, and high-quality data solutions
  • Strong leadership in planning, estimation, and documentation
  • Effective client communication and collaboration
  • Continuous improvement in engineering standards and practices
  • Reliable, accurate, and well-tested data outputs

Skills & Experience

Required

  • 6 years of experience in data engineering or a related field
  • 4 years of hands-on experience with Databricks (including Delta Live Tables, notebooks, workflows)
  • Strong proficiency in Python and SQL
  • Experience with data modelling (star and snowflake schemas)
  • Understanding of medallion architecture and scalable data design
  • Experience with DevOps practices (CI/CD, version control, automation)
  • Experience with testing and data quality frameworks
  • Familiarity with data visualisation tools (e.g., Power BI or similar)

Preferred

  • Exposure to Microsoft Fabric concepts
  • Relevant certifications (e.g., Databricks Associate/Professional) or ongoing certification efforts

Personal Attributes

  • Strong communication skills (verbal and written)
  • High attention to detail
  • Self-driven, proactive, and results-oriented
  • Ability to manage multiple priorities and deadlines
  • Strong problem-solving mindset
  • Adaptable and flexible in a dynamic environment
  • Excellent interpersonal and relationship-building skills
Login to apply