Tone deaf: finding structure in Last.fm data
- 1. Как мне медведь на ухо наступил ищем структуру в данных Last.fm (Tone deaf: finding structure in Last.fm data) Andrei Zhabinski hadoop engineer @ adform.com
- 4. “raw” features name = fry gender = male age = 1029 height = 171 weight = 68 occupation = delivery boy income = $510 balance = $4.3 billion
- 5. transformed / combined features log(age) = 6.94 income^2 = 260100 height * weight = 11628
- 6. expert knowledge (domain specific) planet(GeoIP) = Earth working_hours(delivery boy) = 8
- 7. expert knowledge is really useful Part-of-Speech Tags Stemming/Lemmatization Gabor filters Haar wavelets SURF descriptors
- 8. but what if I’m not an expert?
- 10. Helps to find structure in “raw” data
- 11. Clustering
- 13. Autoencoders RBM
- 14. consider image data raw pixel matrix structured patterns
- 15. deep learning
- 16. Images are too mainstream!
- 17. Last.Fm dataset 360K (http://mtg.upf.edu/node/1671) ● userid, artname, plays ● data from 2001-2009 ● 360K unique users ● 290K unique artists ● 17M records
- 18. Most popular artname audience 1 radiohead 77348 2 the beatles 76339 3 coldplay 66738 4 red hot chili peppers 48989 5 muse 47015 6 metallica 45301 7 pink floyd 44506 8 the killers 41280 9 linkin park 39833 10 nirvana 39534
- 19. # of plays statistics julia> describe(dataset) plays Min 0.0 1st Qu. 35.0 Median 94.0 Mean 215.18519944356137 3rd Qu. 224.0 Max 419157.0 NAs 0 NA% 0.0%
- 20. 419157 * 3 min = 873 days
- 21. 2.39 years
- 22. NOFX
- 24. Noisy and computationally hard Let’s reduce dataset 290K => 1K 17M => 8M
- 25. As simple as possible >= 10 < 10 1 0
- 26. Like-matrix user #1 user #2 ... user #n artist #1 0 0 ... 1 artist #2 1 0 ... 0 ... ... ... ... ... artist #m 1 1 ... 0
- 27. Restricted Boltzmann Machines Two formulations: ● neural network - vertices are “neurons” connected via weighted edges ● Markov random field - vertices are random variables, edges show dependencies
- 28. feature #1 feature #2 feature #3 feature #4 “mega”- feature #1 “mega”- feature #2 “mega”- feature #3 RBM “compresses” input, converting “raw” data into more high-level representation
- 29. short probability theory review
- 30. Random Variable Not a single value, but a set of values with associated probabilities
- 31. Probability Distribution probability distribution X=0 X=1 0.3 0.7 joint probability distribution X1=0 X1=1 X2=0 0.15 0.25 X2=1 0.4 0.3
- 32. Like-matrix user #1 user #2 ... user #n artist #1 0 0 ... 1 artist #2 1 0 ... 0 ... ... ... ... ... artist #m 1 1 ... 0 observation (object)
- 33. Like-matrix user #1 user #2 ... user #n artist #1 0 0 ... 1 artist #2 1 0 ... 0 ... ... ... ... ... artist #m 1 1 ... 0 binary random variable (feature)
- 34. artist #1 artist #2 artist #3 artist #4 ? ? ? 2 sets of random variables: ● visible variables - we know their values (e.g. “likes”) ● hidden variables - some hidden counterparts of visible ones (genres? countries? decades?)
- 35. Ideally, we would like to get probability distribution table V0 V1 V2 ... H1 H2 ... P(...) 0 0 0 ... 0 0 ... 0.002 0 0 0 ... 0 1 ... 0.0004 0 0 0 ... 1 0 ... 0.007
- 36. total number of combinations 2^1100 = 13582985290493858492773514283592667786034938469 31744549748519669727813092754241848720539208320 75605922985782629538473834750387255432349299711 55548342800628721885763499406390331782864144164 68073076683716052622317651279843577212995655335 52860322030803807757597323201989850948840040691 16123084147875437183658467465148948790552744165 376L
- 37. RBM Training: step 1 Sample hiddens given visibles
- 38. RBM Training: step 2 Sample visibles given hiddens (reconstruct)
- 39. RBM Training: step 3 Calculate error and adjust weights dW training result - weight matrix that minimizes reconstruction error
- 40. Hidden variables after training ● each hidden units activates visible ones with different weights ● weights represent strength of some concepts W1 W2 W3 W4
- 41. Learned concepts: pop / metal VS. Beyonce Metallica
- 42. Learned concepts: hard rock / epatage VS. Dream Theater Lady Gaga
- 43. Learned concepts: classic / alternative VS. Bob Dylan Linkin Park
- 44. Artist portrait each artist may be described via weighted vector of concepts
- 45. Radiohead Artist portrait axis weight classic rock/grunge 0.23 pop/rock 0.24 electronic/country -0.23 pop rock/hard rock 0.37 hardcore/alternative -0.1
- 46. Daft Punk Artist portrait axis weight classic rock/grunge 0.09 pop/rock 0.18 electronic/country -0.82 famous/little-known -0.56 girl band/boy band 0.15