Actively Learning to Rank Semantic Associations for Personalized Contextual Exploration of Knowledge Graphs

Editor's Notes

• #9: To generalize
• #10: To generalize
• #28: We defined an heuristic measure but we don’t know if this is the best mesasure for everybody and if a better measure exists
• #29: Motivated by the fact that different users might be interested in different kind of SAs We thus want to learn a ranking funcion from explicit user rating, asking them few feedback as possible, since ranking an entire set of SAs might be tedious and boring Moreover we want a model that is able to quickly imporve and speed up the learning task
• #30: Active sampling method proposed in different context More loops of ranking refinition!
• #31: Active sampling method proposed in different context More loops of ranking refinition!
• #32: Active sampling method proposed in different context More loops of ranking refinition!
• #33: Active sampling method proposed in different context More loops of ranking refinition!
• #34: Active sampling method proposed in different context More loops of ranking refinition!
• #35: Active sampling method proposed in different context More loops of ranking refinition!
• #36: Active sampling method proposed in different context More loops of ranking refinition!
• #37: Active sampling method proposed in different context More loops of ranking refinition!
• #44: The example is built over the complete set of ratings given by the user for an article
• #45: The example is built over the complete set of ratings given by the user for an article
• #46: The example is built over the complete set of ratings given by the user for an article
• #58: Contextual: rating on the whole dataset and testing on the remaining part of the dataset -> replicate user behaviour on the application Cross Validation: evaluate the robustness of our model
• #60: These are the configuration we tested, they can be divided in two major categories: the first one consists of the combination of the divverent bootstrapping methods and of active sampling. The bootstrapping has been tested with two different clustering algorithms that are Gaussian Mixture Model and the Dirichlet Gaussian Mixutre Model. The second category contains our baselines: the first one, namely Random Random, select random associations for both initialization and active sampling, is thus used to see if active learning is really useful in this context. The two remaining configuration do not use a learning to rank algorithm, so there’s no learning behind, and are used to see if ranking models can perform better than a simple heuristic, namely serendipity, and a random approach.
• #61: The first result we tested was the validity of the personalization hypotesis within this context. We used Inter rater reliability measures that are usually used to assess the level of agreement between users. A 0 value can mean that there’s a complete disagreemeent between users while a 1 means that user give unanimous rates. Since value for both krippendorff’s apha and kendall’s W are distant for came to the conclusion that our hypotesis is valid
• #62: These are the results that we obtained in five iterations of the loop I’ve shone you before. Results are computed using nDCG on the top ten results of the ranking list and are aggregated, so we took the mean value for each iteration with respect to each user. We can see that the algorithm that performs better for both the experiments is the one that uses the serendipity heuristic and the AUC based sampling: this is interesting for two reason: in our application we are able to provide an ordered set of probably interesting semantic associations since the first iteration, moreover is the sub-optimal algorithms that is able to gain the best results. This si again importatn because you can see that the graphs on the right lack of the second algorithm for sampling, this is because the algorithms wasn’t able to compute the results in a time that could be considered good for user interaction, thus making in unusable in a live setting. The last thing that is really interesting to notice is that active learning performs better then the baseline we have defined. The learning to rank approach feeded with random observations is always the least algorithms in the learning to rank group. The same goes for the last two baselines considered, those that did not use active learning.
• #63: These are the results that we obtained in five iterations of the loop I’ve shone you before. Results are computed using nDCG on the top ten results of the ranking list and are aggregated, so we took the mean value for each iteration with respect to each user. We can see that the algorithm that performs better for both the experiments is the one that uses the serendipity heuristic and the AUC based sampling: this is interesting for two reason: in our application we are able to provide an ordered set of probably interesting semantic associations since the first iteration, moreover is the sub-optimal algorithms that is able to gain the best results. This si again importatn because you can see that the graphs on the right lack of the second algorithm for sampling, this is because the algorithms wasn’t able to compute the results in a time that could be considered good for user interaction, thus making in unusable in a live setting. The last thing that is really interesting to notice is that active learning performs better then the baseline we have defined. The learning to rank approach feeded with random observations is always the least algorithms in the learning to rank group. The same goes for the last two baselines considered, those that did not use active learning.
• #64: These are the results that we obtained in five iterations of the loop I’ve shone you before. Results are computed using nDCG on the top ten results of the ranking list and are aggregated, so we took the mean value for each iteration with respect to each user. We can see that the algorithm that performs better for both the experiments is the one that uses the serendipity heuristic and the AUC based sampling: this is interesting for two reason: in our application we are able to provide an ordered set of probably interesting semantic associations since the first iteration, moreover is the sub-optimal algorithms that is able to gain the best results. This si again importatn because you can see that the graphs on the right lack of the second algorithm for sampling, this is because the algorithms wasn’t able to compute the results in a time that could be considered good for user interaction, thus making in unusable in a live setting. The last thing that is really interesting to notice is that active learning performs better then the baseline we have defined. The learning to rank approach feeded with random observations is always the least algorithms in the learning to rank group. The same goes for the last two baselines considered, those that did not use active learning.
• #65: These are the results that we obtained in five iterations of the loop I’ve shone you before. Results are computed using nDCG on the top ten results of the ranking list and are aggregated, so we took the mean value for each iteration with respect to each user. We can see that the algorithm that performs better for both the experiments is the one that uses the serendipity heuristic and the AUC based sampling: this is interesting for two reason: in our application we are able to provide an ordered set of probably interesting semantic associations since the first iteration, moreover is the sub-optimal algorithms that is able to gain the best results. This si again importatn because you can see that the graphs on the right lack of the second algorithm for sampling, this is because the algorithms wasn’t able to compute the results in a time that could be considered good for user interaction, thus making in unusable in a live setting. The last thing that is really interesting to notice is that active learning performs better then the baseline we have defined. The learning to rank approach feeded with random observations is always the least algorithms in the learning to rank group. The same goes for the last two baselines considered, those that did not use active learning.
• #66: These are the results that we obtained in five iterations of the loop I’ve shone you before. Results are computed using nDCG on the top ten results of the ranking list and are aggregated, so we took the mean value for each iteration with respect to each user. We can see that the algorithm that performs better for both the experiments is the one that uses the serendipity heuristic and the AUC based sampling: this is interesting for two reason: in our application we are able to provide an ordered set of probably interesting semantic associations since the first iteration, moreover is the sub-optimal algorithms that is able to gain the best results. This si again importatn because you can see that the graphs on the right lack of the second algorithm for sampling, this is because the algorithms wasn’t able to compute the results in a time that could be considered good for user interaction, thus making in unusable in a live setting. The last thing that is really interesting to notice is that active learning performs better then the baseline we have defined. The learning to rank approach feeded with random observations is always the least algorithms in the learning to rank group. The same goes for the last two baselines considered, those that did not use active learning.
• #67: These are the results that we obtained in five iterations of the loop I’ve shone you before. Results are computed using nDCG on the top ten results of the ranking list and are aggregated, so we took the mean value for each iteration with respect to each user. We can see that the algorithm that performs better for both the experiments is the one that uses the serendipity heuristic and the AUC based sampling: this is interesting for two reason: in our application we are able to provide an ordered set of probably interesting semantic associations since the first iteration, moreover is the sub-optimal algorithms that is able to gain the best results. This si again importatn because you can see that the graphs on the right lack of the second algorithm for sampling, this is because the algorithms wasn’t able to compute the results in a time that could be considered good for user interaction, thus making in unusable in a live setting. The last thing that is really interesting to notice is that active learning performs better then the baseline we have defined. The learning to rank approach feeded with random observations is always the least algorithms in the learning to rank group. The same goes for the last two baselines considered, those that did not use active learning.