CIO之家-StarRocks+极速统一的湖仓分析888.pptx

S匧鸟ta絡r?R涸o廩c?k?s區廩??區

StarRcoks廩???胜兞StarRocks廩???鵳?StarRocks廩???劢勻錞?

01StarRcoks廩???胜兞

欽?汥挿鵴畮??涸靀》?鵷?偽岁忘駈??傞/?傞/???叅霧?銳?欽侨??鸟叅霧→呍????廩??鄬??絶??劥侨??荝??????剣呔??欽侨??鸟侨?廩?區涸欽?汥挿侨??區涸欽?汥挿

???銳廩???剣廩???涸????腊?絡?涸??侨?屛椚SingleSourceofTruth:?剣??侨???湱ず涸侨?勻雦皾皍??助卹匬??鵘絶?劥

StarRocksLakeHouse湡叻DatawarehouseDataLake侨?餘?Dataquality叅霧?腊Performance?傞?區Realtime侨?屛椚Governance?佞欰?Open抳崞絡?Singlesourceoftruth?的??Scalability???嫱CostefficiencyStarRocks?匧鸟絡??佞涸廩?匧鸟?區絡?涸catalog絡?叅霧ㄤ侨?屛椚佅??猫OpenTableFormat匧鸟LakeHouse絡??佞

02StarRcoks廩???鵳?

絡??佞涸LakeHouse卹匬StarRocksFESpark/Flink?ComputeengineCatalogTableformatFileformatStorageHMSIcebergParquetHDFSObjectstorageORC?Hudi?StarRocksFileGlue/DLF?StarRocksTableStarRocksCN统一开放的Lakehouse架构，分层解耦设计Lakehouse??卹匬??鷥??ず?彂鲱???????休涸?饆叅呔霧??乾??湬?叅霧ぐ碫邍呔?ㄤ侨?Catalog?侨?盗椚荈剣?????傞????兞涸邉?佅?HDFSㄤ?韌??

呍?暵?ㄤ鵛劍??CBOOptimizerVectorengineComputeEngineMVExternalTableStatisticsTrinoCompatibleCatalogExternalCatalogUnifiedCatalogTableFormatHive/HudiMetadataCacheIcebergMetadataCacheIcebergColocate/BucketJoinFileFormatNativeReaderIOCoalesceLateMaterializationCost-BasedLateMaterializationDataCacheFooter/PageCache

DataCacheDataCache?鵴畮侨?Cache?雦皾蒜挿?欰Cache偽???絆?絈磜?綕??韌Blocklevel佅???+煂渹幊ざ???荝?Hash涸???綕?鼙???蒜挿傞??Cache偽岁ㄐ务LakeBEExecut EngineDataCacheBEExecut EngineDataCacheBEExecut EngineDataCache?BalancedDistributionGranularCachingScanRange0CN0CN1ConsistentHashScanRange1CN3CN2ScanRangeN

DataCache鷳僈?鸟偽????腊??侨???DataCache??腊??邍?騄?12%佅?Cache荈鷓?煂渹ふち??緸絞ふち吐?梡餏??煂渹籖?傞??叅霧崨??叅霧鵴畮??ExcutionEngineDataCacheHDFS/S3ScanBusyScan

ParquetReaderCol1?剣靘霒傞???靀》Col2ㄤCol3涸??侨????助IOざ?RowGroup务湱龇Column靀》ざ?鼙??籖IO霼宠Footer/PageCache綕?鍒?/????た涸Page/Footer?鵷暟??????涸??銳IO

ParquetReader-?鵷暟?叅霧务??靘霒涸?兞??錛Col1?ㄤCol2?剣靘霒傞?銳靀》???侨?Col2涸靘霒剣刿?涸鷥??Col1偽?靀》SKIPScanandFilterAND

ParquetReader-?鵷暟???靀》鷥??刿?涸靘霒?Col2?鵂応?卓Mask?瑟?騥鵂?欽?靀Col1?Mask?瑟傞??絩絯靀》Col1?卐?靘霒???鷥?去?剒?涸Enumeratetheorder

Hive/Hudi?侨??銳