![© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
Case Name IOs
WDDL-EXT Query 6
CII Query 5+[(property number)*5]
Salesforce Query 9](https://image.slidesharecdn.com/wddlandextdesign-150528012304-lva1-app6892/85/Cisco-Webex-Distributed-Framework-and-Data-Store-Design-24-320.jpg)
Cisco Webex Distributed Framework and Data Store Design
- 1. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1Cisco Confidential© 2011 Cisco and/or its affiliates. All rights reserved. 1 WDDL And EXT Design Grant Pan, Jason Ding, Ken Huang, Philip Zhong
- 2. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2 • Resolve Problems • Industry Solutions • WDDL Architecture • WDDL APP Design • Ext DB Design • Ext Database HA and Read-Write Separation • Ext Performance
- 3. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3 • Cross Data Store Operate • Extensible Data
- 4. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4 Solutions Feather Twitter Gizzard 1. Mappings Table shading strategy 2. Relational database, Lucene, Redis and Others 3. Not support query tuning 4. Not support cross data merge 5. Open Source (Apache License, Version 2.0) TaoBao TDDL 1. Mappings table shading strategy 2. Support Oracle and MYSQL 3. Not support query tuning 4. Not support cross data merge 5. Not Open Source WDDL 1. Mappings Table shading strategy 2. Relational database, K/V and others 3. Support query tuning 4. Support cross data merge
- 5. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5 Solutions Feather Salesforce 1. metadata-driven design model 2. One index table with database Partition(ORGID) 3. Provide data optimizer mechanism CII 1. Store as K/V model 2. Table self-join 3. Base on database optimizer WDDL-EXT 1. metadata-driven design model 2. Use many index tables 3. Provide data optimizer mechanism
- 6. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
- 7. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7 Store 1 Store 2 Applications Director Node Data Node Data Node Data Node Director Node … Store N Store WDDL2.0 Application
- 8. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8 APP Parser Optimizer Executor Store SQL-style command and query scope SyntaxNode Tree ExecutionNode Tree Result Operate user customized data Statistics data Director Node Data Node Parser Optimizer Executor Validation Check Generate SQL Sub RowSet Result
- 9. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9 User Command: Select frstName, lastname, homephone From User Where firstName=‘John’ and lastName=‘Fuller’ and username=‘Fuller.john@cisco.com’ Syntax tree (where clause part): And / | firstName l lastName usename Execution tree: join / Join username / firstName lastName Execution tree: Join / Firstname lastname Oracle MYSQL Director Node Data Node(Ext) Parser Optimizer Executor Executor SQL Optimizer Optimizer SQL Execution tree: usename Data Node
- 10. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
- 11. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
- 12. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12 Table Type Description Metadata Table Maps Custom fields to big table columns, index tables, data type etc. Big Table Save the no-relation data, total column count is 500 Index Table Save all marked for indexing fields data, the index table total count is 500 Statistics Table Make the index table data distributed uniformly and calculate the fields selectivity
- 13. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13 Metadata Table Big Table Index Table Statistics Table
- 14. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
- 15. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15 Parser Executor Result MYSQL(n) Optimizer Meta data SQL with scope id Generate Write SQL
- 16. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16
- 17. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17 Parser Executor Result MYSQL(n) Optimizer Meta data SQL with scope id Generate Query SQL
- 18. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18 Rule name Where Condition Rule Description Rule1 FirstName=‘John’ 1/NDV(number of distinct value) Rule2 Friendcount>26 ((High-Value)/(High- low)) Rule3 Friendcount>=26 ((High-Value)/(High- low))+ 1/NDV Rule4 FirstName like ‘John%’ ((High(john*)-Value)/(High- low)) +1/NDV
- 19. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19 SELECT b.value0 as firstname, b.value1 as lastname FROM wbx_xqengine_data_00 b, ( SELECT guid FROM wbx_xqengine_index_01 a WHERE a.stringvalue ='John' ) c WHERE c.guid = b.guid and b.value1 = 'Fuller'
- 20. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20
- 21. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
- 22. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
- 23. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
- 24. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24 Case Name IOs WDDL-EXT Query 6 CII Query 5+[(property number)*5] Salesforce Query 9
- 25. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25 ORACLE: wbxcii records=64400978 MYSQL: mt_data records= 9000415 , mt_index1 records =9000415 ORACLE: wbxcii_wtng records=7129777 MYSQL: mt_data records= 5000415, mt_index1 records =5000415
- 26. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26 Memory Read CII Query WDDL-EXT Query Return rows Join rows Case 1 2.61 s 55 ms 92 37017 Case 2 930 ms 40 ms 92 37017
- 27. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27 Only 18 columns contain test data in 500 columns Each row length is 2.5 MB and all data is distributed in 500 columns by random
- 28. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28 Memory Read WDDL-Ext query Return rows Join rows Case 1 395.6ms 4443 95840 Case 2 757.2ms 4443 95840
- 29. © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29 Thank you.
Editor's Notes
- #25: A.Definition: We define one Organization and one Organization include 9 million objects data. Every object include 20 properties and one property size is 30byte. Define CII’s NAMESPACEID size is 4byte and PREDPATH size is 1 byte. Database index block pointer and record offset size is 10 byte. query SQL(by value to query the object) is “Select objected from user where FIRSTNAME=’john’ and LASTNAME=’mark’” So CII business data number is:9000000*20=180000000 XQueryEngine and salesforce data number is:big table (9000000)+index data(9000000*20)= 189000000 B.CII query used IO number: 1)change the common SQL to CII query SQL: SELECT ii1.objectid FROM wbxcii ii1 INNER JOIN wbxcii ii2 ON ii1.objectid = ii2.objectid AND ii2.pathaliasid = '/usr/lastname' AND ii2.VALUE =' mark’' WHERE ii1.pathaliasid = '/usr/firstname' AND ii1.VALUE =' john' 2) from query SQL, we can get the SQL use following index: Index1:(PATHALIASID, VALUE, OBJECTID, PREDPATH) Index2:(OBJECTID, PATHALIASID, VALUE, PREDPATH) 3)CII use ORACLE database, so the block size is 8K. 4)calculate one index block include index node number, the formula as following: (Index entry size)n+(index block pointer+ record offset size)*(n+1)=block size Index 1 :(26byte+30byte+36byte+1byte)N1+10*(N1+1)=8192 Index 2:( 36byte+26byte+30byte+1byte)N2+10*(N2+1)=8192 N1=79 N2=79 CII business data number is180000000, so the index tree level must equal to 5(79*79*79*79*79=3077056399>180000000), That’s mean query the index 1 need 5 IO and query the index 2 need 5 IO, total query cost is 10 IO. Because the CII is self join to support other properties query, so add one property, the query cost will add 5 IO. C. XQueryEngine used IO number: 1) change the common SQL to XQueryEngine query SQL: SELECT DT.GUID FROM wbx_xqengine_data_00 DT JOIN (SELECT GUID FROM WBX_XQENGINE_INDEX_03 WHERE stringvalue = 'john' and pathaliasid = '/usr/firstname') IT ON DT.guid = IT.guid WHERE (DT.VALUE6 = 'mark') 2)from query SQL, we can get the SQL use following index: Index 1:(guid) Index 2: (stringvalue, pathaliasid)+PK(guid, pathaliasid) 3) XQueryEngine use MYSQL database, so the block size is 16k. 4)calculate one index block include index node number, the formula as following: (Index entry size)n+(index block pointer+ record offset size)*(n+1)=block size Index 1 :( 36byte)N1+10*(N1+1)=16*1024 Index 2: (30byte+26byte+36 byte+26BYte)N2+10*(N2+1)= 16*1024 N1=356 N2=127 XQueryEngine business data number is :big table (9000000) and every index table is data(9000000*20/500=360000) so the big table index tree level must equal to 3(356*356*356= 45118016>9000000), the index table index tree level must equal to 3(127*127*127= 2048383>360000) That’s mean query the index 1 need 3IO and query the index 2 need 3 IO, total query cost is 6 IO , doesn’t use self join to support other properties query, so will not add query cost when add one property. D. salesforce used IO number: 1) change the common SQL to salesforce query SQL: SELECT b.GUID FROM mt_data b, (SELECT guid, orgid FROM mt_indexes1 a WHERE a.stringvalue = 'john' AND a.tabid = 3 AND a.fieldnum = 1 ) c WHERE c.guid = b.guid AND c.orgid = b.orgid AND value2 = 'mark' 2) from query SQL, we can get the SQL use following index: Index1:(guid, orgid) Index2:( stringvalue, tabid, fieldnum, orgid, guid) 3) salesforce use ORACLE database, so the block size is 8k. 4)calculate one index block include index node number, the formula as following: (Index entry size)n+(index block pointer+ record offset size)*(n+1)=block size Index 1 :( 36byte+8 BYTE)N1+10*(N1+1)=8*1024 Index 2: (30byte+8byte+4byte+8byte+36 byte)N2+10*(N2+1)= 8*1024 N1=151 N2=85 salesforce business data number is :big table (9000000) and use one index table with multiple partitions(by ORGID),now one Organization include 9 million objects data, so all index data in one partition, the index table data size 180000000. so the big table index tree level must equal to 4(151*151*151*151= 519885601>9000000), the index table index tree level must equal to 5(95*95*95*95*95=7737809375>180000000) That’s mean query the index 1 need 4 IO and query the index 2 need 5 IO, total query cost is 9 IO , doesn’t use self join to support other properties query, so will not add query cost when add one property.
- #27: Case 1: ORACLE: wbxcii records=64400978 MYSQL: mt_data records= 9000415 , mt_index1 records =9000415 Case 2: ORACLE: wbxcii_wtng records=7129777 MYSQL: mt_data records= 5000415, mt_index1 records =5000415 --oracle CII SQL SELECT UNIQUE 'Q2UD35OD4EFZ2Q0M65G1NQ5MCJ', II1.OBJECTID FROM WBXCII II1 LEFT OUTER JOIN WBXCII II9 ON II9.OBJECTID = II1.OBJECTID AND II9.PATHALIASID = 'Y9JYBIG34BFVHEABYEVAI6IWJ3' and II9.VALUE != 'dfNull' LEFT OUTER JOIN WBXCII II2 ON II2.OBJECTID = II1.OBJECTID AND II2.PATHALIASID = 'Y7H9NW4KCE6ZKH9L9TLQUNUB5K' and II2.VALUE = 'DF_Calendar_Event__c' LEFT OUTER JOIN WBXCII II4 ON II4.OBJECTID = II1.OBJECTID AND II4.PATHALIASID = 'YB0UWDMYMGC59OMI7PD0HB10NH' and ( (II4.VALUE = 'dfnull') OR (II4.VALUE = 'meetings') OR (II4.VALUE = 'pto') OR (II4.VALUE = 'sporting events') OR (II4.VALUE = 'training') OR (II4.VALUE = 'travel') ) LEFT OUTER JOIN WBXCII II7 ON II7.OBJECTID = II1.OBJECTID AND II7.PATHALIASID = 'Y7FC5NAQ1LPO71XOJR26QJF6D2' and (II7.VALUE != 'true') LEFT OUTER JOIN WBXCII II6 ON II6.OBJECTID = II1.OBJECTID AND II6.PATHALIASID = 'YB3953YMA2XMQ56KHMU47DPHJB' and (II6.VALUE = 'false') LEFT OUTER JOIN WBXCII II5 ON II5.OBJECTID = II1.OBJECTID AND II5.PATHALIASID = 'Y3BX69G62ZWN44WYL8D4460C9X' and (II5.VALUE >= '2011-01-10') AND (II5.VALUE < '2011-03-01') LEFT OUTER JOIN WBXCII II10 ON II10.OBJECTID = II1.OBJECTID AND II10.PATHALIASID = 'YB9L34ZZ9WKTJ6ZOF7YQWS9W3A' and (II10.VALUE != 'dfNull') LEFT OUTER JOIN WBXCII II8 ON II8.OBJECTID = II1.OBJECTID AND II8.PATHALIASID = 'YBA7SM4BFSLWL0UA90QV5UYWEK' and (II8.VALUE != 'true') LEFT OUTER JOIN WBXCII II3 ON II3.OBJECTID = II1.OBJECTID AND II3.PATHALIASID = 'Y358Z3VQ5A4NMUTX0FAO4ATX2I' and ( (II3.VALUE = 'dfnull') OR (II3.VALUE = 'gmm') OR (II3.VALUE = 'hr') OR (II3.VALUE = 'marketing') OR (II3.VALUE = 'operations') OR (II3.VALUE = 'visual') ) WHERE (II1.PATHALIASID = 'Y3EQ7KDX0S2BANPPSVH3TVB7X2') AND (II1.VALUE = 'W5Q14UZ3KQTP9H2OQGYQIRMNSB') AND ( II2.VALUE = 'DF_Calendar_Event__c') AND ( ( (II3.VALUE = 'dfnull') OR (II3.VALUE = 'gmm') OR (II3.VALUE = 'hr') OR (II3.VALUE = 'marketing') OR (II3.VALUE = 'operations') OR (II3.VALUE = 'visual') ) AND ( (II4.VALUE = 'dfnull') OR (II4.VALUE = 'meetings') OR (II4.VALUE = 'pto') OR (II4.VALUE = 'sporting events') OR (II4.VALUE = 'training') OR (II4.VALUE = 'travel') ) AND ( (II5.VALUE >= '2011-01-10') AND (II5.VALUE < '2011-03-01') ) AND (II6.VALUE = 'false') AND ( (II7.VALUE != 'true') AND (II8.VALUE != 'true') AND ( (II9.VALUE != 'dfNull') OR (II10.VALUE != 'dfNull') ) )) --mysql big table SQL SELECT DISTINCT b.value0 FROM mt_data b, (SELECT guid, orgid FROM mt_indexes1 a WHERE a.stringvalue = 'W5Q14UZ3KQTP9H2OQGYQIRMNSB' AND a.tabid = 3 AND a.fieldnum = 1 ) c WHERE c.guid = b.guid AND c.orgid = b.orgid AND value2 = 'DF_Calendar_Event__c' AND ( value3 = 'dfnull' OR value3 = 'gmm' OR value3 = 'hr' OR value3 = 'marketing' OR value3 = 'operations' OR value3 = 'visual' ) AND ( value4 = 'dfnull' OR value4 = 'meetings' OR value4 = 'pto' OR value4 = 'sporting events' OR value4 = 'training' OR value4 = 'travel' ) AND (value5 >= '2011-01-10' AND value5 < '2011-03-01') AND value6 = 'false' AND ( (value7 != 'true') AND (value8 != 'true') AND ((value9 != 'dfNull') OR (value10 != 'dfNull')) )
- #29: Case 1: 500 VARCCHAR(250) columns, but 18 columns contain test data, other columns are null Use big table name is wbx_xqengine_data_02 Case 2: 500 VARCCHAR(250) columns, the average length of each row is 2500 byte, And the data is distributed in 500 columns by random Use big table name is wbx_xqengine_data_01 wbx_xqengine_data_01 records :9000305 wbx_xqengine_data_02 records :9000305 wbx_xqengine_index_03 records : 947885 Use SQL: SELECT SQL_NO_CACHE DT.GUID FROM wbx_xqengine_data_01 DT JOIN (SELECT GUID FROM WBX_XQENGINE_INDEX_03 WHERE stringvalue LIKE 'J%') IT ON DT.guid = IT.guid WHERE (DT.VALUE6 LIKE 'D%') ORDER BY DT.VALUE5 DESC, DT.VALUE6 DESC