Applying Genetic Algorithm to Reduce Response Time in ETL Service Composition

Data ETL1 tools, through the creation of workflow (including graphs of data conversion tasks), make it possible to process high volume of data. This workflow includes set of activities that extract data from various sources, transform and load it into a data warehouse. ETL development constitutes the most costly in both time and resources. It can take up as much as 80% of development time in a data warehouse project. Any error in designing of ETL, leads to increase data volume and then rising data access in data warehouse so that impacts on business decisions. Currently ETL workflows are designed manually, which is time consuming and sometimes with errors. Having more Complex workflows, users who can design and make them, must have a lot of knowledge to produce solutions tailored to the business objectives of the working groups. Also available ETL workflows, depend largely on a platform where they have been implemented and it is not possible to reuse them between ETL tools. In service Oriented ETL, we are following the adoption of service oriented ETL capabilities in BI systems, as can be reused ETL workflows turnover, establish interoperability between ETL tools and perform these workflows independent of the platform. Our goal in this study is optimizing ETL workflow dynamically which we implemented as a web service. To do this, we applied genetic algorithm, as an evolutionary approach to service oriented ETL, so it can be considered as an algorithm in designing of ETL tools.