SELF LEARNING METHODS TO FIND NOVEL INFORMATION ON THE WWW



Marco Wiering
Inst of Information & Computing Sciences
University Utrecht
Netherlands


Abstract
This talk will discuss self-learning search methods for finding novel interesting information on the world wide web (WWW). The methods employ reinforcement learning (RL) agents which can optimize their behavior by interacting with an environment and learn from the obtained feedback (reward signals). The reward for finding a novel WWW-page is given by a supervisedly trained naive Bayesian classifier. The goal of the agent is to maximize the cumulative reward obtained in its limited life time, and thus it will learn to find as much novel interesting WWW-pages as possible. The agents learn value functions for evaluating many possible WWW-pages as starting address for its search. It this way it can also learn to use particular WWW-pages as starting
place which lead to finding many novel interesting pages. As soon as the agent has exhaustively explored a WWW-page and its links, it does not get any more reward, and it will learn to search for different WWW-pages. The idea has been proposed by Andrew McCallum, and we provide extensions to this idea.

Biography
Dr. Marco Wiering (marco@cs.uu.nl) finished his Computer Science degree "Cum Laude" from the university of Amsterdam in 1995. Then he did his PhD research at IDSIA (Lugano, CH). He graduated on the PhD-thesis "Explorations in Efficient Reinforcement Learning" in 1999. Since then we was a postdoc at the UvA from May until October, 1999. Since January 2000, he works as a university researcher/teacher at the University Utrecht. He published about 20 papers on the topics of reinforcement learning, machine learning, and multi-agent systems.