Dimensions of Automated ETL Management: A Contemporary Literature Review

Abstract

Emerging dynamics of data systems and reliance on quality data sources, data processing for informed and strategic
decision-making enhance the scope of using the ETL solutions. In the current scenario, one of the critical aspects
focused on software engineering is about focusing on using the data management tools that can help gain insights
for functional and operational aspects. While many academic and industrial research studies have focused on data
management dynamics and the application of ETL tools as a profound solution, there is an imperative need to
upscaling the ETL efficiency over real-time applications. In this literature review, the scope of the current ETL
frameworks, limitations, and scope are discussed. Categorically, the objective is to explore if the machine learning
models are adapted in the ETL systems. However, from the literature review, it is evident that many academic
studies have advocated using machine learning models to improve and optimize the use of ETL solutions. But very
few tools in the market are using the comprehensive range of machine learning models in ETL processing. Focusing
on the current constraints and the scope for improvement, this study advocates the need for designing and
developing machine learning-based models for ETL-based data management optimization. If such processes could
be developed, it can help the organizations have potential systems in place for decision-making.