7 Characteristics of a Bad (Big) Data Platform
Download as PPTX, PDF0 likes1,257 views
The document outlines seven characteristics of a flawed big data platform, emphasizing that it often functions more as a toolbox than a cohesive platform. Key issues highlighted include its schema-less nature, lack of optimization for both read and write operations, and insufficient monitoring and logging capabilities. The author suggests that effective governance and the ability to track data are essential for scalability and performance.
7 Characteristics of a Bad (Big) Data Platform
- 1. 7 Characteristics of a “Bad” (Big) Data Platform Harshal Deo* *Personal Opinions, not endorsed by my employer.
- 2. It supports Analysts and Business Users and Engineers – everyone! This is Not a Platform. It’s a Tool Box.
- 3. It is schema-less. Yet it is ‘Single Source of Truth’ – in production! Schema-less is good for discovery. Production jobs and reports need Catalog and Governance.
- 4. It processes all the data in “Real Time”. Yes, may be you can. But Why? Think ‘Lambda Architecture’ ?
- 5. It moves all the data in memory. Unless you can go back and forth between Disk and Memory, you can not scale in all situations.
- 6. It is optimized for Read and Write. Pick One. The other can be managed – ‘eventually’!
- 7. It can just plug into any monitoring Dashboard. That’s good. But what are we going to monitor? Instrumentation is critical.
- 8. It does not log the request & Metadata. You can not optimize, if you do not log and track. Think ‘self learning’ and enable ‘some’ learning!