Back to professions

Lead data engineer

Grade
G6
Download
Word document

As a Lead Data Engineer, you will lead a team of talented data engineers, who are committed to delivering high-impact, secure data solutions to achieve departmental and directorate objectives.

You are an expert in your field and will build and maintain excellence in data engineering development and embed industry standard best practices across the Data Engineering community in DfE.

This role sits in Data Engineering Division. As a division, we provide a full range of engineering capabilities including data sourcing, management, operations, and maintenance of underlying infrastructure.

You will

  • provide technical guidance and leadership for the development of robust automated data pipelines and data management processes.
  • ensure tools and techniques are scalable, secure, and efficient.
  • provide leadership for critical initiatives covering the end-to-end data development lifecycle for data ingestion, cleansing, validation, transformation, curation and presentation.
  • drive the build of data engineering capability by providing technical leadership, mentoring, coaching and career development.
  • embed industry standard best practices across the Data Engineering community in DfE.
  • be part of your business area's leadership team.
  • be part of the data engineering profession leadership team, working with the head of data engineering to develop DfE's data engineering practices.
  • develop internal data engineering capability by developing L&D routeways, shaping careers paths and recruiting talent to the organisation.
  • Supervise the productivity and quality of deliverables by contingent workers.

Essential skills

The skills and expertise we will assess you against are:

  • Leadership. Significant leadership and management experience in a data environment with the ability to motivate individuals to deliver user excellence using highly developed interpersonal communication skills.
  • Strategic Planning. Extensive proven experience of setting the strategic direction for data engineering teams, whilst continuously improving delivery and keeping up to date with industry innovations and changes in your specialism.
  • Data development process. Extensive hands-on experience at a senior level of setting up and delivering data pipelines in a cloud environment. Strong understanding of modern data architecture and data management practices. Ability to provide the strategic vision for data services and products, championing self-service and modernising data delivery.
  • Agile working. Strong experience of mentoring and leading teams in Agile/DevOps delivery and working across multi-disciplinary teams with delivery, architecture, testing, infrastructure, and operations stakeholders.
  • Collaboration. Ability to mediate between people and mend relationships, communicating with stakeholders at all levels. You know how to manage stakeholders’ expectations and facilitate discussions across high risk or complexity or under constrained timescales.
  • Programming and build. Extensive experience of programming and using data engineering tools, including SQL, Python, Azure Data Factory, Informatica DEI, and/or Databricks.
  • Problem resolution. In-depth knowledge of the types of common problems in data structures, process, and products, and their resolution techniques. Strong problem-solving skills, including assessing and mitigating risks while identifying opportunities for innovation.
  • Coaching, mentoring, and learning. Coaching and mentoring junior members of the team on their professional development. Developing and keeping yourself up to date with the latest data engineering trends and being able to translate the learning into your build where applicable.

Desirable skills

  • Community development. Experience of leading a community of practice in a large complex organisation, setting up practice and community events and fostering a culture of learning and continuously improving ways of working.