A deeper look of how DataForge is organized and processes data.
As DataForge operates differently than traditional ETL tools, this section provides a deeper dive of how DataForge processes data, how data flows through DataForge, and how DataForge is logically organized. All of this information is relevant whether DataForge is used with Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP). The information presented in this section is helpful to know when architecting a new implementation leveraging DataForge as the data processing tool of choice.
It is helpful to think of DataForge as modifications of SQL clauses. In some ways DataForge can be thought of as a front end wrapper for SQL, as each change in the interface modifies the ultimate SQL that is executed. Each step in the logical architecture modifies a different portion of the overall SQL code that executes from ingest through output.
The Logical Data Flow
The Logical Component Architecture
Audience
The intended audiences for this section are the following:
- Data architects who have experience with traditional ETL tools and solutions and are seeking to understand how DataForge works in anticipation of needing to lead a new DataForge implementation.
- Cloud and application architects who are looking to understand how data moves through DataForge.
- Experienced DataForge configurators who are looking to understand more deeply how DataForge works (the how and why behind the configuration).
- Developers looking to gain a deeper understanding of what DataForge does in anticipation of needing to work on DataForge core processing code.
Updated