Seminar
Reinforcement Learning

WS 07/08

Abstract:

Reinforcement Learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting online with a complex, uncertain environment.

In this seminar, we will look at a few selected interesting directions in reinforcement learning by studying recently published papers and text book chapters. We will start with classical approaches and progress to more recent, complex methods.

Some of the topics will be:

• Classical Reinforcement Learning
• Actor-Critic Methods
• Off-Policy Learning
• Model-Based Reinforcement Learning
• Hierarchical Methods
• Policy Gradients
• Solving POMDPs

Presentation: Each student presents one topic, usually a few papers or a book chapter. The presentation should be about 40 minutes followed by 5 to 10 minutes for questions and discussion. The presentation language is either German or English. You might want to check out these suggestions for your presentation. Talk to your advisor at least 2 weeks before your scheduled talk and show him your presentation.

Audience: The presenter will give his talk to the whole group, not just the instructors. Guests, who are not joining the seminar but are interested in the topics, are welcome, too. So bring your friends and fans as well. This is a good opportunity to practice your presentation skills in front of a larger audience.

Composition: You also must write a summary of your talk. It should be about 10-20 pages. Hand it in until the end of the semester (but better finish your summary before you give your talk, because trying to write things down in your own words will help you realize which parts of the paper(s) are important). We will make the written summaries available to all participants.

Grading: In order to get the credits (ECTS/Schein), you must give a presentation, write a summary and attend the seminar meetings. The seminar is worth 4 credits (ECTS) or 2 SWS.

Organizer: Professor Jürgen Schmidhuber.

Contact: Thomas Rückstieß (email)
Please direct questions, suggestions, etc. regarding this seminar to Thomas.

Registration: If you want to join the seminar, please contact Thomas. The topics are assigned on a "First come, first served" basis.

Coming soon: The schedule will be announced soon. The seminar will be held either as a block seminar or on a weekly basis, depending on the participants.

Book recommendation: "Reinforcement Learning" (online version available here) by R. Sutton and A. Barto is probably the most-cited book in the field of reinforcement learning. It is very well-written and not difficult to understand, yet gives a good introduction to the basic algorithms including chapters on continuous state and action spaces and model-based methods.
Everyone interested in reinforcement learning should read this book (probably several times).