Data Pipeline Framework
Author: Emilia Colonese Figure 1: Data Pipeline Framework (by author). The figure above presents a data pipeline framework. It encompasses generic data processing phases used by Business Information (BI) Systems, either in traditional or big data context. Traditional Systems: They use structured data, relacional schema and databases. Big Data Systems: They use structured, simi-structured and/or unstructured data; relational and/or non-relational schemas; SQL and/or NoSQL data storages, and databases servers . These BI Systems uses data modeling process and tools for both, on premisse and cloud environments. The fundamentals for data modeling used are related to the data storage. A traditional system uses relational data model or SQL model, either for OLTP (transational/operational) or OLAP (analytical) systems. The data schema contains tables and their relationsips that will store data. On the other hand, a big data system uses non-relational data model or NoSQL ...