Senior Data Engineer

Job title:

Senior Data Engineer

Company

Brightwater

Job description

Our client is a leading multinational technology services company renowned for their data teams and is known for being a great place to work.About the Position
We are seeking a talented senior Data Engineer with strong Azure skills who is looking to work with a dynamic team on very interesting data challenges. You will be designing and managing data engineering infrastructure using cloud based tools.Responsibilities:
As a Senior Data Engineer, your primary responsibilities will include:

• Designing and managing data engineering infrastructure using cloud-based tools like Azure Data Factory, Databricks, and Azure Data Lake.
• Building and maintaining secure, scalable data pipelines for data ingestion and exposure (e.g., APIs).
• Implementing and enforcing data quality standards across all workflows.
• Automating infrastructure deployment and management using Terraform and other Infrastructure-as-Code (IaC) tools.
• Managing Azure cloud environments for production and non-production workloads.
• Identifying and resolving security and compliance vulnerabilities in cloud environments.
• Troubleshooting and resolving production issues and implementing long-term solutions.
• Collaborating with cross-functional teams, including Data Science and Product teams, to establish best practices for analytics pipeline development.
• Mentoring junior team members to foster growth and knowledge sharing.
• Documenting processes, standards, and architecture to ensure operational efficiency.

Experience/RequirementsRequired:Hands-on experience designing and developing data pipelines with a strong foundation in software engineering principles.Proficiency with Azure services (e.g., Blobs, Functions, Data Factory, Service Principal, Containers, Key Vault).Expertise in real-time and batch data processing, with the ability to select the appropriate approach based on requirements.Strong programming skills in Python, SQL, and Spark.Proven experience developing and deploying APIs.In-depth knowledge of designing and implementing best practices for data architecture, quality, and security.Experience with Terraform for automating and managing cloud infrastructure.Critical thinking skills and the ability to solve complex problems in dynamic environments.Flexibility and adaptability to contribute to various aspects of a project in a start-up-like atmosphere.Preferred:Experience with DevOps tools, CI/CD pipelines, and Git workflows.Familiarity with agile/scrum methodologies.Understanding of artificial intelligence, machine learning, and their applications.Experience working on proof-of-concept projects for Big Data and Data Science.Please contact on 01 5927853 or or simply click the apply button.To view all live jobs with Brightwater and market insights, please visit our website

Expected salary

€90000 – 120000 per year

Location

Dublin

Job date

Sat, 18 Jan 2025 03:34:39 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (jobsnear.net) you saw this job posting.