批流一体中的“批流”是指批处理与流处理:中国数字化转型网www.szhzxw.cn
计算一体
同一套计算逻辑可以同时应用于流处理与批处理两种模式,且在最终结果上一致。
存储一体
流处理与批处理过程中全程数据存储在同一介质,且同一数据只需要存储一遍。

流批一体就是同时支持离线和实时计算,我们假设数据湖是一个堰塞湖,堰塞湖的水顺着山体不断向下流淌,山体有各种洼沟,水流向第一层洼沟后,水会先横向渗透洼沟,这个过程可理解为批处理过程。
当第一层漕沟填满水后会流向下一层,同样下一层横向渗透洼沟的批处理过程。就这样水流一层一层往下传递(类似ODS, DWD,ADS各层)。中国数字化转型网www.szhzxw.cn
当各层洼沟都溢满水后,如果上游水流持续流淌,那么就看到各层也实时有水垂直到下一层(实时),同时也会横向洼沟波动(批处理),效果上看就是实时处理和同步处理同时处理,融合为一个体系。
翻译:
What is flow in one?
“Batch stream” in batch stream integration refers to batch processing and stream processing: China Digital Transformation network www.szhzxw.cn中国数字化转型网www.szhzxw.cn
Computing in one
The same set of computational logic can be applied to both stream and batch processing modes, and the final result is the same.
Memory integration
In the process of stream processing and batch processing, the whole data is stored in the same medium, and the same data needs to be stored only once.中国数字化转型网www.szhzxw.cn
Flow batch integration is to support both offline and real-time computing, we assume that the data lake is a barrier lake, the water of the barrier lake continues to flow down the mountain, the mountain has a variety of gullies, after the water flows to the first layer of the gullies, the water will first transverse penetration of the gullies, this process can be understood as a batch process.
When the first trench is filled with water, it will flow to the next layer, and the next layer will also penetrate the trench horizontally. In this way, the water flow is passed down layer by layer (similar to ODS, DWD,ADS layers). China Digital Transformation network www.szhzxw.cn
When each layer of gully is full of water, if the upstream water continues to flow, then you can see that each layer also has water vertical to the next layer (real-time), but also lateral gully fluctuations (batch processing), the effect is real-time processing and synchronous processing at the same time, integrated into a system.中国数字化转型网www.szhzxw.cn

