A hierarchical attention model for social contextual image recommendation
Download as DOCX, PDF0 likes92 views
This document describes a hierarchical attention model for social contextual image recommendation. It analyzes three key aspects from social media data - upload history, social influence, and owner ratings. These are used to generate matrices representing users' relationships to images. A convolutional neural network is trained on these matrices along with image data to generate recommendations. The model was tested on a dataset of 100 users with images and social connections. It was able to provide personalized image recommendations to users based on both image content and social relationships.
A hierarchical attention model for social contextual image recommendation
- 1. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com A Hierarchical Attention Model for SocialContextual ImageRecommendation Now-a-days all social networking sites provide support for image upload and sharing with other users, to allow user to share images various social networking sites used various recommendation techniques such as content base recommendation (based on past history), collaborative recommendation (based on user and his friends similarity) and personalized recommendation etc. All this previous techniques where not using complex social aspects such as Upload History, Social Influence and Owner Admiration, by using this 3 key aspects we can get context relationship between users and images which helps in perfect recommendation based on relationships. A hierarchical attention model can be generated with combination of 3 key aspects and Convolution Neural Network (CNN) where CNN represents image visual model for user and 3 key aspects will represents Users upload history, social influence and owner matrix. From social network dataset first we extract 3 key aspects and then generate a matrix (vector) 1) Upload History: This represents all images uploaded by this current user, with all social images a matrix will form up and if user upload any image from that matrix then that matrix column value will be updated with 1, if user not uploaded that image then column value will be updated with 0. For all users and images a matrix will be generated and this matrix called as upload history matrix. 2) Social Influence: This represents relationship of user with other users, if current user is friend of another user then this matrix will be updated with 1, if current user not a friend of other user then this matrix column updated with 0. 3) Owner: This represents image owner who uploaded image and this matrix will contains rating values of that image, if any user give rating to this image then matrix will have value 1 in its column for that image, if image not got any rating then matrix column will contain 0. Using above 3 key aspects will form 3 matrixes and then build CNN model with all images. CNN represents image visual data and matrixes will contains user social data. Both this will be used to generate train model by using concept called ‘HETEROGENEOUS (different) DATA EMBEDDING’. This technique will combine different features such as images and 3 key aspects to build single model. Inside CNN all images and matrix data will combine to generate train model. If we give new test image then CNN will apply test image on train model to find best image match and relationship match to give recommended new images to user. HASC algorithm will take 3 matrixes as input and then calculate positive and negative predicted
- 2. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com recommendation from CNN model. All positive predicted images will be shown to user as recommendation. To implement this project we used social networking dataset with image sharing and below are some records from dataset. For security reasons this dataset is anonymised (hide) user details with numeric values. User ID, Social Friends, Uploaded Images, Rating Images 1,19 52 16 2 36 31 81 42 87 51,00000058.jpg 00000073.jpg,00000040.jpg-2 00000069.jpg-4 2,71 7 40 83,00000159.jpg 00000073.jpg 00000162.jpg 00000146.jpg,00000077.jpg-4 00000050.jpg-3 00000053.jpg-5 00000056.jpg-1 3,90 26 6,00000160.jpg00000074.jpg 00000044.jpg 00000041.jpg 00000035.jpg 00000150.jpg 00000155.jpg 00000071.jpg 00000056.jpg 00000042.jpg,00000146.jpg-4 00000145.jpg-3 00000062.jpg-1 00000040.jpg-5 00000034.jpg-3 00000051.jpg-4 00000065.jpg-2 00000159.jpg-3 00000162.jpg-4 00000059.jpg-2 4,42 9,00000064.jpg 00000075.jpg 00000036.jpg,00000158.jpg-3 00000147.jpg-3 00000035.jpg-4 In above dataset all bold names are the column names and all values are comma separated. First value is the user id and second column values are the friends of this user and each user separated by space. Third column value is images uploaded or owned by this user and each image separated by space. Fourth column value is the rating for the image, here each image separated by space and between image name and rating hyphen symbol is there. For example in fourth column 00000040.jpg-2 00000069.jpg-4 00000040.jpg = image name and 2 is the ratingof that image and 00000069.jpg is theimage name and 4 is the rating. All images used in this project are available inside ‘img’ folder and above dataset available inside ‘dataset’ folder. Convolution Neural Network Working Procedure To demonstrate how to build a convolutional neural network based image classifier, we shall build a 7 layer neural network that will identify and separate one image from other. This network that we shall build is a very small network that we can run on a CPU as well. Traditional neural networks that are very good at doing image classification have many more parameters and take a lot of time if trained on normal CPU. However, our objective is to show how to build a real-world convolutional neural network using TENSORFLOW. Neural Networks are essentially mathematical models to solve an optimization problem. They are made of neurons, the basic computation unit of neural networks. A neuron takes an input (say x), do some computation on it (say: multiply it with a
- 3. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com variable w and adds another variable b) to produce a value (say; z= wx + b). This value is passed to a non-linear function called activation function (f) to produce the final output (activation) of a neuron. There are many kinds of activation functions. One of the popular activation function is Sigmoid. The neuron which uses sigmoid function as an activation function will be called sigmoid neuron. Depending on the activation functions, neurons are named and there are many kinds of them like RELU, TanH. If you stack neurons in a single line, it’s called a layer; which is the next building block of neural networks. See below image with layers To predict image class multiple layers operate on each other to get best match layer and this process continues till no more improvement left. Screen shots To run this project double click on ‘run.bat’ file to get below screen
- 4. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com In above screen click on ‘Upload Social Image Sharing Dataset’ button to upload dataset In above screen uploading social network dataset called ‘dataset.txt’ after uploading dataset will get below screen
- 5. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com In above screen we can see dataset loaded and this dataset has total 100 user’s records. Now click on ‘Identify 3 Key Aspects From Dataset’ button to identify 3 keys aspects such as Upload History, Social Influence and Owner, after identifying 3 aspects a matrix will form up and mark the matrix values with 1 or 0, if user upload any image then that matrix will have 1 for that image column other matrix will contains 0. Similarly same will apply for social influence and ratings. See below matrix for above 3 aspects
- 6. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com In above screen we can see for each user a matrix rows and columns are generated and in that matrix we can see 0 and 1 values updated based on 3 identified key aspects. All values greater than 1 are the ratings of that image. Now click on ‘Run CNN Embedding Images & Vector’ button to build CNN model with images and 3 key aspects matrix. See below screen of CNN model generation In above screen CNN model generated and in below console screen we can see all CNN details
- 7. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com In above console we can see total 4 layers are created with first layer image size as 126,126 and second layer image size as 63 and 63 and other layers information also there. Now click on ‘Run HASC Algorithm & Image Recommendation’ button to upload test image and then allow CNN and HASC algorithm to predict positive best match images from train model as recommendation images. In above screen I am uploading new test image called ‘5.jpg’ and below are the recommendation based on image content similarity and relationships data which we calculated using 3 key aspects.
- 8. Venkat Java Projects Mobile:+91 9966499110 Visit:www.venkatjavaprojects.com Email:venkatjavaprojects@gmail.com Above are the recommendation images for uploaded test image based on image content similarity and relationship similarity, similarly you can upload other images and test. All above recommendation images are coming from ‘img’ folder. Now click on ‘Rating For Each User Graph’ to see rating for each image given by users In above graph x-axis represents image name and y-axis represents rating for that image.