Are you an HR-professional recruiting new members for your organisation’s data team? If you are looking for talent to fill a data engineer role, you have probably faced the challenge of writing a proper job description. Your CTO/Director of Engineering/Head of Data is very busy, and gave you only some high-level points, but nothing that fits into a JD. Don't worry, you are not alone.
Below you can find some boilerplate wording for the 'Tasks & Responsibilities' section of a data engineer job description. Adjust, adapt and complete it the way you see fit, the point is really to make you think less about the wording and more about finding the right person for the job.
One hint: be selective in what you write into the job ad, more does not equal better in this case.
Work closely with the product team on the platform
Develop and scale the recommendation engine and comparable algorithms
Interface with the infrastructure team
Scale the architecture to manage increased traffic
Work on the backend structure, the API and partner integrations
Start integrating the recommendation engine
Measure and ensure data quality across the organisation
Use agile software development processes to deliver features
Build and document solid data pipelines
Clean, transform, and aggregate data from different sources
Develop and document processes for data mining, data modeling and data warehousing
Support building complex algorithms that provide business value to the customers
Ingest and aggregate data from both internal and external data sources
Help streamline the data science processes
Plan data models and architecture
Implement customer lifecycle and retention models based on an existing methodology
Employ an array of coding languages and tools to set up the data infrastructure
Interface with the reporting team and support their objectives with infrastructure tweaks
Execute/implement new data product features end-to-end
Engage with the machine learning team and uncover hidden efficiencies in the pipelines
Turn high-volume activity data into a highly accessible resource
Ingest and aggregate data from both internal and external data sources
Transform the available data into meaningful insights working with the analyst team
Develop simple models and integrate them with the visualisation tools
Work closely with the data science and business intelligence teams to develop data models and pipelines for research, reporting, and machine learning
Integrate state-of-the-art data management and software engineering technologies
Tap into new data streams from third-party APIs
Create custom software components for the data platform
Collaborate with the stakeholders in an interdisciplinary team
Research new approaches for making data accessible to customers
Connect legacy and new data systems together
Define basic tooling, metrics and other solutions and maintain quality to the highest standards
Write, extend and debug microservices for planned features
Collaborate with other engineers, data analysts, and product managers
Lead data strategy and inform the product strategy team on how world-class data experiences are built
Take leadership opportunities and shape the data culture within the organisation
Build near real-time and batch data processing pipelines
Design and optimise low latency systems
Build highly reliable but flexible service infrastructure
Maintain tooling and enable algorithms to move into production faster
Relate and match entities from different data streams
Increase the implementation speed of data tools
Creating secure processes to keep the data pipelines safe
Improve data models and foster data-driven decision making
Draw a comprehensive picture of user flows and enable deeper analysis
Specify data requirements and pre-processing routines
Hold companywide data trainings
Develop solutions for automatic labeling of data
Model front end and back end data sources
Design and optimise complex queries and deliver user value
Work closely with data scientists on modeling
Grow the data competency across the company