Data Engineer

About the role

The Data Engineer is responsible for designing, building, and maintaining robust data solutions to support the organisation’s analytics, business intelligence, and operational processes.

This role plays a critical part in architecting scalable data pipelines, managing data warehousing solutions, and integrating data from various internal and external sources. You’ll collaborate closely with data scientists, analysts, and product teams to ensure data infrastructure meets business needs and enables advanced data-driven use cases. The role also involves mentoring junior engineers and driving best practices in data engineering and governance.

Responsibilities

Data Architecture and Engineering

• Design and implement scalable data architectures to support data analytics, machine learning, and business intelligence initiatives
• Build, maintain, and optimize ETL/ELT pipelines to ingest, transform, and process data from third-party systems, APIs, and internal databases
• Develop and maintain data warehouse solutions, ensuring efficient storage, transformation, and retrieval of data across multiple platforms

Cloud Infrastructure

• Implement cloud-based data infrastructure using technologies such as AWS (e.g., Redshift, S3, Glue), ensuring scalability and performance
• Optimize cloud solutions for cost-efficiency and performance across large-scale distributed systems

Data Integration and Management

• Lead the integration of diverse data sources, ensuring data pipelines are reliable, efficient, and scalable
• Work closely with stakeholders to ensure data solutions align with organizational needs, enabling advanced analytics and machine learning applications

Data Governance and Security

• Ensure compliance with data security and privacy regulations, applying best practices in data governance
• Monitor and enforce data quality standards to maintain accuracy and consistency across the data platform

Collaboration and Mentorship

• Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver solutions that support predictive analytics and machine learning models
• Provide mentorship to junior data engineers, fostering an environment of continuous learning and improvement

Requirements

• Proficiency in data engineering tools and technologies such as Python, SQL, Apache Spark, and ETL/ELT processes
• Experience building and maintaining data warehouses and cloud-based data solutions, particularly on AWS (e.g., Redshift, S3, Glue, Lambda) and Snowflake
• Strong understanding of data integration, including API development, data lakes, and batch/stream processing
• Expertise in data governance and ensuring compliance with data privacy and security regulations
• Experience in optimizing data pipelines and architectures for cost-effectiveness and performance
• Excellent problem-solving skills with the ability to work independently and as part of a team
• Experience mentoring and leading junior engineers