Brief Description 

• The data engineer willoversee the expansion and optimization of our data architecture and datapipeline with the purpose of improving data analysis and ML workflows. The dataengineer will be handling the design and construction of scalable data systemsand researching new use cases for data acquisition. 

Detailed Description        

Technical 

• Help to design and develop a data estate that is performant, accessible, secure, scalable, maintainable, and extensible. 
• Help to implement true CI/CD. 
• Design and develop EDW using Snowflake, DBT etc. 
• Design and develop AWS/Azure Data Lake  
• Design and develop data ingestion pipelines  
• Model EDW entities and ensure all data is complete, accurate, timely, and well documented 
• Work towards the implementation of a true Self-Service analytics platform. 

Governance & DataProtection 

• Ensure that all work follows best security practices and fully adheres to DPA and other data regulations. 
• Ensure that all work follows the correct approval and sign-off process before it is pushed into Production. 
• Ensure that all work is documented and if needed, has a runbook in order to guarantee business continuity and support 
• Work with others in the team to keep the Data Dictionary and Ubiquitous Language complete and up todate. 

PROCESS

• Follow existing processes and work to improve/identify gaps in these processes. 
• Ensure the correct SDLC promotion processes are followed. 
• Follow the correct sign-off processes to ensure that only approved releases are deployed into Production. 
• Ensure that all AWS/Azure development follows CI/CD processes and is repeatable. 

Team/People 

• Evangelize about the data team across the business. 
• Build relationships with members of the data team and the wider business team. 
• Work closely and collaboratively with all members of the data team. 
• Work closely with and learn from tech and team leads and challenge proposed solutions with your own ideas. 

Job Requirements        

Job Specifications 

• Bachelor’s degree in computer science or related fields   
• Some experience and knowledge of coding language such as Python.
• Good experience and knowledge of the SQL query language.
• Some understanding of star schemas and data warehouse concepts.
• Some knowledge of AWS/Azure tools and technologies 
• Beneficial - ETL and ELT experience - both batch and microservices-led.
• Minimum three 3 years’ experience in a data engineer role or similar.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.