Mining of massive datasets
Download as PPTX, PDF0 likes1,280 views
The document discusses techniques in stream processing and data mining, specifically focusing on Bloom filters for guaranteed negative detection with potential false positives. It explores MapReduce methods for multiway joins, matrix multiplication, and various clustering algorithms, including K-means and hierarchical clustering, along with dimensionality reduction techniques like SVD and CUR. The content is based on the textbook 'Mining of Massive Datasets' and includes practical approaches to optimize data processing and clustering for large datasets.
![Map Reduce – Multiway Joins
Hash B to b buckets, c to C buckets.
bc = k
Cost ~ r + 2s + t + 2 * sqrt(krt)
Usually, can neglect r + t compared to the k term. So,
2s + 2*sqrt(krt)
[Single MR job]](https://image.slidesharecdn.com/miningofmassivedatasets-141129174357-conversion-gate02/85/Mining-of-massive-datasets-12-320.jpg)
![Map Reduce – Multiway Joins
Hash B to b buckets, c to C buckets.
bc = k
Cost ~ r + 2s + t + 2 * sqrt(krt)
Usually, can neglect r + t compared to the k term. So,
2s + 2*sqrt(krt)
[Single MR job]
vs (r + s + t + prs)
[Two MR jobs]](https://image.slidesharecdn.com/miningofmassivedatasets-141129174357-conversion-gate02/85/Mining-of-massive-datasets-13-320.jpg)
![Similarity - Shingling
“abcdef” -> [“abc”, “bcd”, “cde”…]
Jaccard similarity - > N(intersection) / N(union)](https://image.slidesharecdn.com/miningofmassivedatasets-141129174357-conversion-gate02/85/Mining-of-massive-datasets-20-320.jpg)
![Similarity - Shingling
“abcdef” -> [“abc”, “bcd”, “cde”…]
Jaccard similarity - > N(intersection) / N(union)
Problem?
Size](https://image.slidesharecdn.com/miningofmassivedatasets-141129174357-conversion-gate02/85/Mining-of-massive-datasets-21-320.jpg)
![Thanks
Mining of Massive Datasets
Leskovec, Rajaraman, Ullman
Coursera / Stanford Course
Book: http://www.mmds.org/ [free]](https://image.slidesharecdn.com/miningofmassivedatasets-141129174357-conversion-gate02/85/Mining-of-massive-datasets-45-320.jpg)
Mining of massive datasets
- 1. Mining of Massive Datasets Ashic Mahtab @ashic www.heartysoft.com
- 3. Stream Processing Have I already processed this? How many distinct queries were made? How many hits did I get?
- 4. Stream Processing – Bloom Filters Guaranteed detection of negatives. Possible false positive.
- 5. Stream Processing – Bloom Filters Have a collection of hash functions (h1, h2, h3…). For an input, run the hash functions. Map to bit array. If all bits are lit in working store, might have been processed (possibility of false positives). If any of the lit bits in hashed array are not lit in working store, need to process this. (Guaranteed…no false negatives).
- 6. Stream Processing – Bloom Filters 1 0 0 1 1 0 1 1 1 0 0 0 1 0 0 1 1 0 0 0 0 0 1 1 1 0 0 0 0 1 0 1 0 0 0 1 0 0 0 1 Input 1: “Foo” hashes to: 1 0 0 1 1 0 0 0 0 0 Input 2: “Bar” hashes to: 1 0 1 1 1 0 0 0 0 0
- 7. Stream Processing – Bloom Filters Not just for streams (everything is a stream, right?) Cassandra uses bloom filters to detect if some data is in a low level storage file.
- 8. Map Reduce A little smarts goes a l-o-o-o-n-g way.
- 9. Map Reduce – Multiway Joins R join S join T size(R) = r, size(S) = s, size(T) = t Probability of match for R and S = p Probability of match for S and T = p Which do we join first?
- 10. Map Reduce – Multiway Joins R (A, B) join S(B, C) join T(C, D) size(R) = r, size(S) = s, size(T) = t Probability of match for R and S = p Probability of match for S and T = p Communication cost: * If we join R and S first: O(r + s + t + pst) * If we join S and T first: O(r + s + t + prs)
- 11. Map Reduce – Multiway Joins Can we do better?
- 12. Map Reduce – Multiway Joins Hash B to b buckets, c to C buckets. bc = k Cost ~ r + 2s + t + 2 * sqrt(krt) Usually, can neglect r + t compared to the k term. So, 2s + 2*sqrt(krt) [Single MR job]
- 13. Map Reduce – Multiway Joins Hash B to b buckets, c to C buckets. bc = k Cost ~ r + 2s + t + 2 * sqrt(krt) Usually, can neglect r + t compared to the k term. So, 2s + 2*sqrt(krt) [Single MR job] vs (r + s + t + prs) [Two MR jobs]
- 14. Map Reduce – Multiway Joins So…is this always better?
- 15. Map Reduce – Complexity Replication Rate (r): Number of outputs by all Map tasks / number of inputs Reducer Size (q): Max number of items per key at reducers p = number of inputs For nxn: qr >= 2n^2 r >= p / q
- 16. Map Reduce – Matrix Multiplication Approach 1 Matrix M, N M(i, j), N(j, k) Map1: Map matrices to (j, (M, i, mij)), (j, (N, k, njk)) Reduce1: for each key, output ((i, k), mij*njk) Map2: Identity Reduce2: For each key, (i, k) get the sum of values.
- 17. Map Reduce – Matrix Multiplication Approach 2 One step: Map: For M, produce ((i, k), (M, j, mij)) for k = 1…Ncolumns_in_N For M, produce ((i, k), (N, j, njk)) for k = 1…Nrows_in_M Reduce: For each key (i, k), multiple values, and sum.
- 18. Map Reduce – Matrix Multiplication Approach 3 Two steps again.
- 19. Map Reduce – Matrix Multiplication One pass: (4n^4) / q Two pass: (4n^3) / sqrt(q)
- 20. Similarity - Shingling “abcdef” -> [“abc”, “bcd”, “cde”…] Jaccard similarity - > N(intersection) / N(union)
- 21. Similarity - Shingling “abcdef” -> [“abc”, “bcd”, “cde”…] Jaccard similarity - > N(intersection) / N(union) Problem? Size
- 23. Similarity - Minhashing h(S1) = a, h(S2) = c, h(S3) = b, h(S4) = a
- 24. Similarity - Minhashing Problem? h(S1) = a, h(S2) = c, h(S3) = b, h(S4) = a
- 25. Similarity – Minhash Signatures
- 26. Similarity – Minhash Signatures Problem? Still can’t find pairs with greatest similarity efficiently
- 27. Similarity – LSH for Minhash Signatures
- 29. Clustering – K Means 1. Pick k points (centroids) 2. Assign points to clusters 3. Shift centroids to “centre”. 4. Repeat
- 30. Clustering – K Means
- 31. Clustering – FBR • 3 sets – Discard, Compressed and Retained • First two have summaries. N, sum per dimension, sum of squares per dimension • High dimensional Euclidian space Mahalanobis Distance
- 33. Clustering – CURE • Sample. Run clustering on sample. • Pick “representatives” from each sample. • Move representatives about 20% or so to the centre. • Merge of close.
- 36. Dimentionality Reduction - SVD
- 37. Dimentionality Reduction - SVD
- 38. Dimensionality Reduction - CUR SVD results in U and V being dense, even when M is sparse. O(n^3)
- 39. Dimensionality Reduction - CUR Choose r. Choose r rows and r columns of M. Intersection is W. Run SVD on W (much smaller than M). W = XΣY’ Compute Σ+, the Moore-Penrose pseudoinverse of Σ. Then, U = Y * (Σ+)^2 * X’
- 40. Dimensionality Reduction – CUR Choosing Rows and Columns Random, but with bias for importance. (Frobenius Norm)^2 Probability of picking a row or column: Sum of squares for row or column / Sum of squares of all elements
- 41. Dimensionality Reduction – CUR Choosing Rows and Columns Same row / column may get picked (selection with replacement). Reduces rank.
- 42. Dimensionality Reduction – CUR Choosing Rows and Columns Same row / column may get picked (selection with replacement). Reduces rank. Can be combined: multiply vector by sqrt(k) if it appears k times.
- 43. Dimensionality Reduction – CUR Choosing Rows and Columns Same row / column may get picked (selection with replacement). Reduces rank. Can be combined: multiply vector by sqrt(k) if it appears k times. Compute pseudo-inverse as before, but transpose the result.
- 44. Dimensionality Reduction – CUR Choosing Rows and Columns Same row / column may get picked (selection with replacement). Reduces rank. Can be combined: multiply vector by sqrt(k) if it appears k times. Compute pseudo-inverse as before, but transpose the result.
- 45. Thanks Mining of Massive Datasets Leskovec, Rajaraman, Ullman Coursera / Stanford Course Book: http://www.mmds.org/ [free]