Back to professions

Senior data engineer

Grade
G7
Download
Word document

As a Senior Data Engineer, you will be experienced practitioner who can plan and lead data engineering activities in larger teams and across complex programmes of work.

This role sits in Data Engineering Division. As a division, we provide a full range of engineering capabilities including data sourcing, management, operations, and maintenance of underlying infrastructure.

You will

  • deliver secure, robust, and scalable data engineering solutions in multi-disciplinary teams for complex data products.
  • implement modern data engineering practices to assure high-quality data is made available for use cases like analysis and machine learning.
  • build complex data pipelines and guide other members of the team to achieve optimal code performance.
  • effectively communicate technical concepts to non-technical stakeholders, and influence decision making to enable sound business decisions.
  • taking an active role and leading initiatives in the DfE data engineering community, to help develop and improve our practice across the department.

Essential skills

The skills and expertise we will assess you against are:

  • Data development process. Strong experience of building data pipelines on cloud platforms. Experience of working with a wide variety of data structures such as data warehouses, data lakes etc. to service business data needs.
  • Agile working. Experience of delivering data products in an agile delivery environment through DevOps working practices – CI/CD, automation etc.
  • Collaboration. Ability to work alongside others to ensure the stability, robustness, and resilience of products you build. Strong experience in managing team dynamics and facilitating difficult discussions within the team or with a diverse set of stakeholders.
  • Programming and build. Proficiency in programming and data engineering tools, including SQL, Python, Azure Data Factory, Informatica DEI, and/or Databricks.
  • Problem resolution. In-depth knowledge of the types of common problems in data structures, process, and products, and their resolution techniques. Strong problem-solving skills, including assessing and mitigating risks while identifying opportunities for innovation.
  • Data innovation. Understanding of business requirements and spotting opportunities to innovate. Willingness to step out of comfort zone and learn modern data engineering practices and trends.
  • Coaching, mentoring, and learning. Coaching and mentoring junior members of the team on their professional development. Developing and keeping yourself up to date with the latest data engineering trends and being able to translate the learning into your build where applicable.

Desirable skills

  • Data Visualisation and Machine Learning. Experience of creating data visualisation products and/or implementing machine learning models on large scale