Block Chain meets Big Data
18 likes2,906 views
Blockchains can be applied to big data in several ways: 1) Blockchains can add a new data construct to big data systems by providing features like atomicity, immutability, and provenance to data. 2) Blockchain data from cryptocurrencies like Bitcoin can be analyzed alongside other data sources to generate insights for applications in areas like trading, risk scoring, and compliance. 3) Blockchains can be used to build applications that provide data governance, compliance, legal archiving, and chain of custody for big data lakes. They can also enable attribution and licensing of digital works.
![Key Aspects and Trends
Source: Evry
● The Ledger
Industry-level system of record (provides
massive efficiency gain for incumbent)
● Transaction Model
Permissionless and Permissioned
Blockchains, Anonymous and Trusted
Validator based Consensus (new models to
support existing use cases)
● The Asset / Token
Currency, Smart Contracts, Physical Goods
[Proof of Work, Coin, Consensus, Bit,
Transaction, Contract, Block, Sign, Chain,
Proof of Stake, Crypto, Hash, Miner, Curve]](https://image.slidesharecdn.com/03d9b245-8bbd-4c2d-82b3-7e4dfdb2fb08-160516033431/85/Block-Chain-meets-Big-Data-7-320.jpg)
![Challenges from a Computational standpoint
Slow - Code runs much
slower than natively
compiled
Expensive - Memory and
storage costs take us back by
50 years - and costs $$$
Not immediately decisive -
Actions of last 60s to 10 mins
may be reorganized
Comparison with Big Data Stack
Pubkey script: OP_HASH160 <Hash160(redeemScript)> OP_EQUAL
Signature script: <sig> [sig] [sig...] <redeemScript>
Spending a P2SH
Ethereum
Script
Bitcoin
Script](https://image.slidesharecdn.com/03d9b245-8bbd-4c2d-82b3-7e4dfdb2fb08-160516033431/85/Block-Chain-meets-Big-Data-12-320.jpg)
Block Chain meets Big Data
- 1. Block Chain meets Big Data Data Night Singapore - April 2016 Vihang Patel, CTO, Dragon Wealth
- 2. About Me Creating a superior Client Experience for Financial Advisors and Wealth Managers using Data Previous Clients
- 3. Overview ● BlockChains - Key Aspects ● Applications ● Challenges from a Computational standpoint ● A new data construct ● Applications using Block Chain Data ● Block Chain applied to Big Data
- 4. The Block Chain (Bitcoin) This is how the Bitcon Blockchain looks like recently (changing every 10 mins or so) : 471 binary files of 128 MB
- 6. Surprise Surprise ? Source : http://www.righto.com/2014/02/ascii-bernanke-wikileaks-photographs.html Benedictus Sanguis eius pretiosissimus. Benedictus Iesus in sanctissimo altaris Sacramento. Ave Maria, gratia plena, Dominus tecum. Benedicta tu in mulieribus, ... ...and life everlasting, through the merits of Jesus Christ, my Lord and Redeemer. O Heart of Jesus, burning with love for us, inflame our hearts with love for Thee. Jesus, meek and humble of heart, make my heart like unto thine! Wikileaks Cablegate Backup cablegate-201012041811.7z Download the following transactions with Satoshi Nakamoto's download tool which can be found in transaction 6c53cd987119ef797d5adccd76241247988a0a5ef783572a9972e7371c5fb0cc Free speech and free enterprise! Thank you Satoshi! 5c593b7b71063a01f4128c98e36fb407b00a87454e67b39ad5f8820ebc1b2ad5 221d900b5ac701028f9dfab7dfba326f608308386d45c05432e721b7c122cba7 ... 128 lines of transaction ids deleted ...
- 7. Key Aspects and Trends Source: Evry ● The Ledger Industry-level system of record (provides massive efficiency gain for incumbent) ● Transaction Model Permissionless and Permissioned Blockchains, Anonymous and Trusted Validator based Consensus (new models to support existing use cases) ● The Asset / Token Currency, Smart Contracts, Physical Goods [Proof of Work, Coin, Consensus, Bit, Transaction, Contract, Block, Sign, Chain, Proof of Stake, Crypto, Hash, Miner, Curve]
- 8. General Applications Source - ABE EBA Asset-centric technologies ● Exchange of digital representations of existing assets (e.g. currencies, metals, stock, bonds etc.) ● Shared ledger but not on a public ledger ● Trust is organised between participants directly, so not through a blockchain and mining as is the case with bitcoin
- 9. General Applications Source - ABE EBA Application Stack ● Platform for the development and execution of complete applications on top of decentralised networks ● Call it ‘cloud services’ as delivered, in non-crypto technology environments, akin to AWS, Azure, GC.
- 10. General Applications Source - ABE EBA Asset registries ● Public ledgers to register assets other than the ‘coins’ these public ledgers use ● Tokens reference an existing asset (e.g. stocks, vehicles, buildings, domain names), the ownership of that particular asset is publicly registered without the need for a central authority
- 11. General Applications Source - ABE EBA Currencies ● Applied in Bitcoin, Litecoin, Peercoin and Dogecoin focus on creating so-called currencies and an associated transfer mechanism ● Their applications range from speculations, online and POS transacting and the storage of value ● No regulatory oversight (yet) ● (Potentially new Crypto-Fiat Currency options coming up)
- 12. Challenges from a Computational standpoint Slow - Code runs much slower than natively compiled Expensive - Memory and storage costs take us back by 50 years - and costs $$$ Not immediately decisive - Actions of last 60s to 10 mins may be reorganized Comparison with Big Data Stack Pubkey script: OP_HASH160 <Hash160(redeemScript)> OP_EQUAL Signature script: <sig> [sig] [sig...] <redeemScript> Spending a P2SH Ethereum Script Bitcoin Script
- 13. Adding a new Data Construct (collective property across different block chains) Atomicity - Entire Operation runs or nothing does Synchronous - No two operations can interfere with each other Provenance* - All messages can be inspected to determine caller address Immutability - Object’s code can never be changed Immortality - Object can never be externally deleted Permanence - Object’s data are permanent Verifiable and Auditable Authenticity - All interactions are cryptographically signed *depending on implementation
- 14. Applications using Block Chain Data ● Early Stage for Big Data Applications (a good time to get in?) ● Bitcoin and the Blockchain are great “research objects” for data scientists ● General commercial focus around Bitcoin and Cryptocurrency analysis ○ Currency Volatility and Price Trends (E.g. using Social Media data in conjunction with the Price movements) ○ Identify which transactions mark Bitcoin currency movements by key stakeholders ○ The ability to identify trends, track data from its source to its endpoint, and making inferred correlations between data points
- 15. Applications using Block Chain Data ● Real-time Search and Data analytics ● Scoring Systems (Risk Pricing) ● Data Warehouses - (BlockChain + Other Sources) ● Forensic compliance intelligence ● https://www.elliptic.co/bigbang-v1.html
- 16. Applications using Block Chain Data ● Forensic compliance intelligence ○ Compliance / AML - Expose funds derived from illicit activities and detect complex money laundering activities ○ Capital Markets - Combine multiple data sources and discover actionable insights from sparse data ○ Trader Oversight - Gain insights into risk exposure in real-time and detect anomalous behavior ○ Performance Monitoring - Monitor smart contracts in real-time and quickly pinpoint the root of the issue ○ Transaction Risk Scoring - Combining source of funds features with behavioral analytics to detect suspicious activity
- 17. BlockChains applied to Big Data ADEPT ● A Take on IoT and BlockChain that can be a data goldmine ● Protocols + Services + Applications ● (TeleHash, BitTorrent, Ethereum)
- 18. BlockChains applied to Big Data A Big Data Lake with ● Data Governance and Regulatory Compliance ● Legal Hold and Long Term Archive ● Chain-of-Custody ● Forensic Readiness and eDiscovery
- 19. BlockChains applied to Big Data Interesting take on Notary and Attribution ● Which one among thousand spread across is the original art source? ● Lock in attribution ● Securely share and trace where digital work spreads ● Certificate Of Authenticity ● License Your Work
- 20. Thanks