Seminar
Recent Advances in Reinforcement Learning

Abstract: Reinforcement Learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment.

In this seminar, we will study a few very interesting, only recently published papers about reinforcement learning. Most of these publications are selected from the NIPS (Neural Information Processing Systems) workshop 2006, held in Vancouver, Canada last December.

The topics are (but may still change)

• Natural Actor-Critic
  Jan Peters, Sethu Vijayakumar, Stefan Schaal
• Maximum Margin Planning
  Nathan Ratliff, Andrew Bagnell
• Policies Based on Trajectory Libraries
  Martin Stolle
• Learning Operational Space Control
  Jan Peters, Stefan Schaal
• Exploration and Apprenticeship Learning in Reinforcement Learning
  Pieter Abbeel, Andrew Ng
• No-regret Algorithms for Online Convex Programs
  Geoff Gordon
• Reducing T-Step Reinforcement Learning to Classification
  John Langford, Bianca Zadronzny

Presentation: Each student presents one topic, usually one main paper and serveral references. The presentation should be about 40 minutes followed by 5 to 10 minutes for questions and discussion. The presentation language is either German or English. You might want to check out these suggestions for your presentation. Talk to your advisor at least 2 weeks before your scheduled talk and show him your presentation.

Audience: The presenter will give his talk to the whole group, not just the instructors. Guests, who are not joining the seminar but are interested in the topics, are welcome, too. So bring your friends and fans as well. This is a good opportunity to practice your presentation skills in front of a larger audience.

Composition: You also must write a summary of your talk. It should be about 10-20 pages. Hand it in until the end of the semester (but better finish your summary before you give your talk, because trying to write things down in your own words will help you realize which parts of the paper(s) are important). We will make the written summaries available to all participants.

Grading: In order to get the credits (ECTS/Schein), you must give a presentation, write a summary and attend the seminar meetings. The seminar is worth 4 credits (ECTS) or 2 SWS.

Organizer: Professor Jürgen Schmidhuber.

Contact: Thomas Rückstieß (email)
Please direct questions, suggestions, etc. regarding this seminar to Thomas.

Registration: If you want to join the seminar, please contact Thomas. The topics are assigned on a "First come, first served" basis.

Coming soon: The schedule will be announced soon. The seminar will be held either as a block seminar or on a weekly basis, depending on the participants.

Book recommendation: "Reinforcement Learning" (online version available here) by R. Sutton and A. Barto is probably the most-cited book in the field of reinforcement learning. It is very well-written and not difficult to understand, yet gives a good introduction to the basic algorithms including chapters on continuous state and action spaces and model-based methods.
Everyone interested in reinforcement learning should read this book (probably several times).