The ETL Developer will design solutions for extracting large data sets from the analytics data stores, supporting delivery of data products. The data sets could exceed one billion records and will require full documentation, testing, automation, and optimization.
These capabilities will enable the client to increase growth and manage operations more efficiently through the application of cutting edge analytics.
The successful candidate will work closely with data scientists, platform and application specialists, and data source owners to deliver foundational data sets, enabling analytics solutions driving successful business outcomes. This role will provide leadership defining and implementing best practices for efficiently sourcing and extracting data from data stores.
Accountabilities / Responsibilities:
Required Skills, Experience, and Qualifications:
- Work closely with the Advanced Analytics team, business partners, and other stakeholders to define analytics processing requirements for data sets
- Develop robust data models and ETL scripts for data products
- Develop data models for SQL and no SQL data sources
- Develop transformation scripts for converting non-Structured and semi-Structured data sets into structured models
- Develop scripts to automate validation, logging, and alerting
- Identify next generation opportunities for acquiring data from SQL, non-SQL, streaming, batch, pub/sub, and request/response sources of data
- Transform data to support the needs of advanced analytics modeling including Machine Learning, and AI processing.
- Thrive in an innovative environment that is guiding client’s entry into advanced analytics and technologies
- Work collaboratively across client teams to put models into production
- 8+ years’ relevant industry experience with a focus on data engineering preferred
- Undergraduate degree in a field linked to data engineering, business analytics, applied mathematics, computer science, IT, computer applications, engineering or related field is required; advanced degree is preferred
- Good understanding of multiple database technologies and the ability to define optimal processes and scripts
- Experience in core data engineering activities:
- Database optimization (partitioning, group and sort keys, indexes, query optimization)
- Data cleansing, loading, transfer, and management tools
- Familiarity with a broad base of analytical methods (one or more of the following):
- Data modeling (variable transformation & summarization, algorithm development)
- Data processing:
- Programming and/or scripting experience: