![]() ![]() I have used other ETL tools before, but not to the same extent as NiFi. The Role of ETL in the Oracle Communications Data Model Figure 2-1, 'Layers of an Oracle Communications Data Model Warehouse' illustrated the three layers in Oracle Communications Data Model warehouse environment: the optional staging layer, the foundation layer, and the access layer. Where needed you can use other tools to complement it.Įxtra strong full disclosure: I am an employee of Cloudera, the company that supports NiFi and other projects such as Spark and Flink. NiFi is a great tool, you just need to make sure you use it for the right usecase. A typical thing that you would not want to do in NiFi is joining two dynamic data sources.įor joining tables, tools like Spark, Hive, or classical ETL alternatives are often used.įor joining streams, tools like Flink and Spark Streaming are often used. 1ETL Overview This chapter provides an overview of the fundamental concepts related to Oracle BI Applications ETL processes. NiFi is really a tool for moving data around, you can do enrichments of individual records but it is typically mentioned to do 'EtL' with a small t. Schema aware, and can share schema with solutions like Kafka, Flink, Spark.It can handle any format, not only limited to SQL tables, but can also move log files etc.Low latency, you can support both batch and streaming usecases.To save time and increase efficiency, use change data capture to automate the process to only. Intuitive gui, which allows for easy inspection of the data The sources could include MySQL, PostgreSQL, Oracle and others. Transfer Steps of ETL Processes, From an IBM Mainframe to an Oracle Server Tom,I'm looking for a way to decrease the total run time for a set of ETL processes that, by one measure, operate on about 60 GB of information in each, notionally daily, processing cycle. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |