Showing posts with label Dataware Housing. Show all posts
Showing posts with label Dataware Housing. Show all posts

How do the ETL tools work ? |ETL Tools Selection in Data Warehousing


How do the ETL tools work?

The first task is data extraction from internal or external sources. After sending queries to the source system data may go indirectly to the database. However usually there is a need to monitor or gather more information and then go to Staging Area . Some tools extract only new or changed information automatically so we dont have to update it by our own. 
The second task is transformation which is a broad category: 
-transforming data into a stucture wich is required to continue the operation (extracted data has usually a sructure typicall to the source) 
-sorting data 
-connecting or separating 
-cleansing 
-checking quality 

The third task is loading into a data warehouse

As you can see the ETL Tools have many other capabilities (next to the main three: extraction , transformation and loading) like for instance sorting , filtering , data profiling , quality control, cleansing , monitoring , synchronization and consolidation.

ETL Process ,ETL Tools Selection in Data Warehousing




ETL PROCESS

ETL process
The three-stage ETL process and the ETL tools implementing the concept might be a response for the needs described above.

The ‘ETL’ shortcut comes from 'Extract, transform, and load' – the words that describe the idea of the system. The ETL tools were created to improve and facilitate data warehousing.
    The Etl process consists of the following steps:
  1. Initiation
  2. Build reference data
  3. Extract from sources
  4. Validate
  5. Transform
  6. Load into stages tables
  7. Audit reports
  8. Publish
  9. Archive
  10. Clean up
Sometimes those steps are supervised and performed indirectly but its very time-consuming and may be not so accurate.
The purpose of using ETL Tools is to save the time and make the whole process more reliable.