New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Carl Olofson
3 likes716 views
The document discusses the emergence of a new generation of database management systems (DBMS) driven by the demands of cloud-based applications and the limitations of traditional relational databases. It emphasizes the role of graph databases in managing complex relationships and highlights their scalability and efficiency in handling varied workloads. The conclusions recommend adopting diverse DBMS technologies tailored to specific tasks, particularly advocating for graph databases in areas like customer and social network management.
New Data Technologies, Graph Computing and Relationship Discovery in the Enterprise - Carl Olofson
- 1. Graph Computing and Relationship Discovery in the EnterpriseCarl OlofsonResearch Vice PresidentIDC
- 2. AgendaThe New Generation of DBMSOverview of the movement toward a new generation of DBMS.New demands of emerging Cloud-based applications.“NoSQL” technologies use casesGraph databasesWhat they are, and the role they play in social media and various analytical functions.List of requirements for a graph database.Limitations of RDBMS for handling graph database workloads.How an object-oriented DBMS is well suited to use in deploying a graph database.Conclusions and Recommendations2Source:/Notes:
- 3. A New Generation of DBMSWhy a new generation of DBMS?DBMS users are no longer insisting that one DBMS can handle all workloads.New Cloud-based workloads demand different database architectures and technologies.Requirements vary:Large, informal data collections for ad hoc analysisLarge, complex data collections for ongoing analysisData sharing for clusters of application serversVery large databases (VLDB)Extreme transaction throughputRapid execution of complex relational database queries.3
- 4. Third Generation DBMS TechnologiesRelationalColumnarCellular (or modular)In-memory RDBMSNon-RelationalKey-value pairList-oriented (Hadoop oriented)Graph4
- 5. DBMS Types5GraphDatabaseObjectDatabaseMulti-value (Pick style)DatabaseData ComplexityHadoopDatabaseRelationalDatabaseNetworkDatabaseKey-Value PairDatabaseHierarchicalDatabaseInverted ListDatabaseModel Complexity
- 6. Web AppsPure OO AppsOLTP & DW MF & Dist. AppsGraphDatabaseObjectDatabaseMulti-value (Pick style)DatabaseMF Transactions, Complex SchemaDBMS Types and Workloads6Data ComplexityHadoopDatabaseRelationalDatabaseNetworkDatabaseHigh Throughput Mainframe TransactionsKey-Value PairDatabaseHierarchicalDatabaseInverted ListDatabaseModel Complexity
- 7. Most Flexible Relationship Management7GraphDatabaseObjectDatabaseMulti-value (Pick style)DatabaseData ComplexityHadoopDatabaseRelationalDatabaseNetworkDatabaseKey-Value PairDatabaseHierarchicalDatabaseInverted ListDatabaseModel Complexity
- 8. DBMS Architectures and WorkloadsDifferent kinds of DBMS being employed to take on workloads required by emerging Cloud-based applications.Cloud demands virtualized, limitless data spaceNew kinds of data relationships introduced by website constructs and social media, not supported by conventional DBMSFluctuating user demand requires flexible resource scalability in terms of server capacitySometimes, data to be analyzed is dynamically gathered; users chafe at the requirement that they build a schema and map out storage space before they load it.8
- 10. Graph DatabasesA graph database is used to trace relationships among entities, most commonly people, to any depth. Its characteristics are:Very simple, fixed schemaVery complex data relationshipsUsed to support complex associations among like entities.10Jeff SmithAttribute(s)Nancy JonesJim SmithEdgeNodePaul JonesJane Jones-SmithDoris SmithJohn JonesMeta-ModelInstance Example (simplified)
- 11. Requirements for a Graph DBMSThese are the core requirements of a graph DBMS:Must be capable of supporting recursive relationships.Must be scalable to arbitrary sizes with minimal administration.Provides rapid means of traversing complex relationship structures.Can rapidly search many objects based on their relationships and properties.For applications such as law enforcement, must have ACID (atomicity, consistency, isolation, durability) properties.11
- 12. Limitations of Relational DBMSWhy RDBMS does not support graph databases.The relational model is flat (2 dimensional).A relational database cannot properly capture nested or recursive data.Data based on multiple modes of relationship between entitiesData based on collections of entities of a type under another entity of the same typeExamplesParts database and other “bill of materials” structuresEmployee reporting hierarchyPersonal relationships captured in social media12Person-X-RefKnows-PID (PK)
- 15. Description
- 16. … Requires a separate SELECT for each level of query.
- 17. Object-Oriented DBMS and Graph DatabasesAn object-oriented DBMS is well suited to use in deploying a graph database because it can handle the demands of very complex object relationships.In order to support the various types of relationships demanded by object-oriented models, OODBMS supportsContainer relationshipsRecursionType hierarchiesThe underlying building blocks needed to support these constructs also enableRapid relationship traversal and searchEfficient storage of databases of arbitrary size13
- 18. Conclusions and RecommendationsConclusionsA new generation of DBMS is emerging, prompted by new problems posed by Web applications and scalability options inherent in faster, cheaper, more scalable hardware.Users are more open to the idea of heterogeneous DBMS; choosing the right tool for the job.Graph database is just such a technology, and object technology provides an excellent platform for its management.RecommendationsNow is the time to put appropriate functionality above DBMS uniformity: keep an open mind regarding DBMS products.RDBMS does not adequately address all data needs: choose the technology that’s right for the task at hand.Graph databases offer relationship tracking and analysis capability that can’t be done by other means. Consider such technology as a solution for such problems in areas such as customer relationship management, social network management, supply chain management, etc.14