Lead Data Engineer

Job title:

Lead Data Engineer

Company

Methods Business and Digital Technology

Job description

Methods Business and Digital Technology LimitedMethods is a £100M+ IT Services Consultancy who has partnered with a range of central government departments and agencies to transform the way the public sector operates in the UK. Established over 30 years ago and UK-based, we apply our skills in transformation, delivery, and collaboration from across the Methods Group, to create end-to-end business and technical solutions that are people-centred, safe, and designed for the future.Our human touch sets us apart from other consultancies, system integrators and software houses – with people, technology, and data at the heart of who we are, we believe in creating value and sustainability through everything we do for our clients, staff, communities, and the planet.We support our clients in the success of their projects while working collaboratively to share skill sets and solve problems. At Methods we have fun while working hard; we are not afraid of making mistakes and learning from them.Predominantly focused on the public-sector, Methods is now building a significant private sector client portfolio.Methods was acquired by the Alten Group in early 2022.As a Lead Data Engineer at Methods you will be expected to get involved with projects from conception to delivery – leading and supervising other engineers to meet requirements and deliver quality data products. On a day-to-day basis, you will own best practice for data management and data engineering – providing support to other members of the team and raising awareness across the organisation for data issues and solutions.When contributing towards projects, you’ll make effective long-term design solutions and take an active part in sprint reviews and other agile processes. You’ll often perform code reviews and provide constructive feedback to other developers.Often, you will be called upon to shape and review bids as required based on your expertise.In your downtime or in preparation to get involved in new and exciting opportunities, you’ll be able to use personal development time to keep up to date on the latest trends and technologies.You will also have line management responsibilities for other data engineers and be involved in the recruitment process.RequirementsThe Lead Data Engineer will:

• Lead and supervise other Data Engineers to meet requirements and deliver quality data products.
• Communicate with a range of stakeholders, be they technical, non-technical, junior or very senior and be able to explain a range of technical issues to them.
• Be involved with the architectural design and implementation of data pipelines and solutions.
• Perform code reviews and provide constructive feedback to other Data Engineers.
• Champion best practice and take a pragmatic view on the pros and cons of such processes, adapting where needed.
• Contribute and review bids as required based on your expertise.
• Be a point of knowledge across the organisation in Data Engineering.
• Stay up-to-date with the latest technologies and trends in Data Engineering.
• Develop and maintain a library of documentation and best practice within Data Engineering.
• Develop and build the Data Engineering team.
• Line manage Data Engineers within the team.
• Provide technical leadership, mentorship, and guidance to the Data Engineering team.

Key Requirements:

• Extensive experience working with Azure Data Engineering tools and other cloud platforms.
• Bachelor’s or Master’s degree in Computer Science, Mathematics, or related field.
• Experience leading or mentoring a team of Data Engineers.
• 8+ years of experience in Data Engineering or a related field.
• Strong experience with data modelling, data warehousing, and ETL/ELT processes.
• A fluency and development experience in at least one of the following: Java, Python, PySpark or Scala.
• Experience working with a variety of data formats such as JSON, Parquet, XML etc.
• Experience with or developed understanding of the application of ETL/ELT tools.
• Experience with NoSQL type environments, Data Lakes, Lake-Houses (Cassandra, MongoDB or Neptune).
• Experience with distributed storage, processing engines such as Apache Hadoop and Apache Spark.
• Experience with message brokering/stream processing services such as Apache Kafka, Confluent, Azure Stream Analytics.
• Experience in Test Driven Development (TDD) and Behaviour Driven Development (BDD).
• Ability to translate business requirements into technical specifications.
• Experience in developing, testing and documenting reliable, scalable and secure APIs.
• Experience in Continuous Integration and Continuous Delivery (CI/CD).
• Experience of iterative/agile development methodologies such as SCRUM.
• Knowledge of standard cloud security principles.
• Openness to new technologies and willingness to keep learning.
• Experience of working in an agile environment.
• Good communication and problem solving skills.
• Willingness to travel as required.

Desirable Requirements:

• Experience working in other cloud computing platforms such as Amazon Web Services or Google Cloud Platform.
• Understanding of Graph Database technologies.
• Experience of working within a consultancy environment.
• Experience of IoT and applications.

This role will require you to have or be willing to go through Security Clearance. As part of the onboarding process candidates will be asked to complete a Baseline Personnel Security Standard; details of the evidence required to apply may be found on the government website Gov.UK. If you are unable to meet this and any associated criteria, then your employment may be delayed, or rejected . Details of this will be discussed with you at interviewBenefitsMethods is passionate about its people; we want our colleagues to develop the things they are good at and enjoy.By joining us you can expect

• Autonomy to develop and grow your skills and experience
• Be part of exciting project work that is making a difference in society
• Strong, inspiring and thought-provoking leadership
• A supportive and collaborative environment

Development – access to LinkedIn Learning, a management development programme, and trainingWellness – 24/7 confidential employee assistance programmeFlexible Working – including home working and part timeSocial – office parties, breakfast Tuesdays, monthly pizza Thursdays, Thirsty Thursdays, and commitment to charitable causesTime Off – 25 days of annual leave a year, plus bank holidays, with the option to buy 5 extra days each yearVolunteering – 2 paid days per year to volunteer in our local communities or within a charity organisationPension – Salary Exchange Scheme with 4% employer contribution and 5% employee contributionDiscretionary Company Bonus – based on company and individual performanceLife Assurance – of 4 times base salaryPrivate Medical Insurance – which is non-contributory (spouse and dependants included)Worldwide Travel Insurance – which is non-contributory (spouse and dependants included)Enhanced Maternity and Paternity PayTravel – season ticket loan, cycle to work schemeFor a full list of benefits please visit our website ( )

Expected salary

£70000 – 80000 per year

Location

London

Job date

Tue, 05 Mar 2024 23:20:08 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (jobsnear.net) you saw this job posting.