Multidimensional or tabular points to consider
- 1. Tabular Cube Solutions Introduction and comparison to MOLAP in GDW Created by: Deepak Kumar( Deepak.Kum@HCL.com)
- 2. Why Tabular Came? • Not a replacement for multidimensional. • Push for self service BI, compatibility with Power BI. • Demand for Low latency(real time BI) solutions. • Push from competitors for in memory solution. • Design more adept for OLAP workloads using relational source. • Less steep learning curve.
- 3. Architecture in brief • Two deployment models: In-Memory and Direct Query. • In-Memory used columnar based data storage in compressed format on disk for backup and the database resides in memory. • Direct Query makes just a semantic model at the top of relational model and no data is stored in memory. • Different from MOLAP as that is mainly disk based storage, designed to be able to return data in a better way for OLAP workload. • Also require a workspace server where database can reside during the development stage. Doing this on desktop could cause performance problem as RAM will be consumed. For multiple developers working on the same server could demand more memory.
- 4. Some more details on In-Memory • We need to evaluate memory need for a tabular database before starting development. There is no direct formula to calculate the memory need as it depends on the data spread. • Memory needed could nearly double up when processing full is needed and we need to keep two copies of data in memory. • Data stored in columnar format could lead to a different storage/memory required based on the distinct values of the fields. • Memory required could also be very high in some cases where the DAX calculation has to keep intermediate result set in memory to do the calculations.
- 5. What makes tabular different • Only Tables, No Dimension, Facts. • Model is the lowest logical entity and not cube. • Only one Model Per Database. • Model can have data pulled from heterogeneous sources. • In memory database. Mostly data remains in RAM. • Columnar storage database using dictionary.
- 6. Missing Functionality in Tabular • Ragged Hierarchy • Role Playing Dimension • Scope Assignments • Many to Many relationship • Drill-through (Query raw data from source) • Versatility of MDX
- 7. Where Tabular has an edge • Processing one table is not affecting other table. So nothing like Process Update of dimension in multidimensional. • No cache affect on querying. • No partitioning affect on query performance. • No security expression bottleneck on dimension security. • No aggregation to maintain. • Distinct Count Measure would benefit immensely as that is what is causing problem in the aggregation size in MOLAP. • Can have data from multiple sources in the same model.
- 8. No Dim Fact, Only Tables • There are only tables and no dim or fact. • No referential integrity between dimension tables and fact tables. • Dim and Fact are stored and partitioned in same way. • Tables can be processed independently in any order. • Measures can be calculated in any of the table.
- 9. Partitioning • Both dimension and fact tables can be partitioned. • Partitioning has no impact on query performance. • Partitioning is useful in refresh of data in memory. • Multiple partitions of the same table not processed in parallel. • Calculated columns and other internal index are not partitioned and are recalculated full any way.
- 10. Time Intelligence in Tabular/ Timing Calc. • Tabular support time intelligence functions but with more limitations than options available in multidimensional. • We can have only one date table that means out of box time intelligence in DAX could be utilized with only one table for the given active relationship. For others we will need to write DAX expressions. • To implement timing calculation without date table or when there are many date tables(role playing dimension) require creation of multiple measures.
- 11. Currency Calculations using DAX • Currency Calculation requires that we are able to convert value of fact using the exchange rate for a given date for the target currency. • As multi column relationship is not directly supported, we need to write DAX expression to do the conversion and we create multiple measures for different combination of currency and currency type. • It could be complex to do as compared to multidimensional but once the pattern is established that could be reused. • Going with approach of doing the currency conversion in database could be expensive because of the columnar storage nature.
- 13. Data Modelling • Having only one model per database in Tabular means that we are going to duplicate dimension data on the server, so it is good to have less number of models, which is in someway not in accordance to what we have currently. • It is also a bit not in accordance of the SSAS guidelines that we are following where we are going to have different solutions having different view names, as here we are going to need less number of views that can serve more and more solutions. • Not having role playing dimension could mean that we will end up duplicating date dimension( or other dimensions) in memory. Also as Time Intelligence functions in DAX requires that date table has to be continuous. • We need to have only those columns in the dimension and fact views which are required as that will impact the storage required and memory consumption. We need to give attention to decrease the distinct values by redesigning to decrease the dictionary size.
- 14. Timing and Currency Calculation • Implementing Timing calculations will be comparatively complex as we can not set multiple tables as date tables. Creating Timing calculations against date dimensions which are not Date Type can be more complex. • As it is not possible to build a timing tool like we have done that in SSAS MOLAP, it will require, creating multiple measures version for the same measure. • Currency calculation implementation will be complex as compared to multidimensional as doing currency calculation in database view will increase storage as we will need to have multiple copies of data in memory. • Doing it in calculation will not be easy as there is no way to edit the measures like we do in multidimensional. Secondly, it will require comparatively complex DAX calculation as there is no support of multi column relationship. • Both of these can be done in Tabular but it will require some initial work from more experienced team members and that can be followed as guidelines by others.
- 15. Other Points • From multidimensional perspective there was a concept of properties( apart from attributes). So the fields which were required just for seeing in the report and not for slicing/dicing can still be managed this way. • Query logging does not seems to be working. • Small adjustments could be moved forward quickly in Tabular as we don’t have dim-fact dependency that that much ingrained in the model. • Default Member setting is missing in Tabular. Could be important from implementing calculations. • Different types of aggregation functions are not supported as setting even on base measure, that needs to be handled in DAX expression for measures. • As there is only one model file, from SVN perspective, managing changes can be a bit different as one developer can work one time.
- 16. Need to discover more • Ways to handle different types of hierarchies : ragged/parent child. • Multiple date tables and time calculation against that. • Dax calculation performance. • Handling different types of aggregation functions. • Row Context and Filter Context in comparative complex DAX expressions.
- 17. Thank You Feedback is appreciated