Intelligent agents
CS-430
This course covers the part of Artificial Intelligence that deals with rational decision and action, as is required for intelligent agents. It starts with techniques for programming and learning optimal policies, for planning and planning with adversaries, and scaling to complex problems. The next part deals with multi-agent planning, both centralized and distributed. The third part of the course is dedicated to game theory, which provides the theoretical underpinning for multi-agent systems.
The course is complemented with exercises in Java, using an agent platform. The exercises form a project where students learn to implement agents and multi-agent systems of increasing complexity. For the third part of the course on game theory, paper exercises are provided.
Documents for the course are found on Moodle, and students are expected to check Moodle regularly for information. Schedules and organizational details may change in response to circumstances. The recorded lectures from 2020 (or later) are available for each week and are not very different from the live lectures.
Grading:
There will be one midterm quiz that counts for 30% of the final grade.
There will be a tournament where each student can submit an agent and can win bonus points depending on the performance up to a maximum of 10% of the grade. The bonus points are not required to get a perfect grade in the course
There will be a final examination during the last week of the semester and it will count for 70% of the grade.
Organization:
The lectures on Wednesdays will generally last about 2 hours; the 3rd hour is dedicated to discussion of course and exercise material, and presenting exercises.
The exercise hours on Thursday are used purely to provide answer questions and provide assistance with exercises.
Several rooms have been allocated for the exercises. However, all the assistants will initially be in BC01 and we will use the other rooms only if there is not enough space in BC01. Please come to BC01 first.
The quiz will be held on Thursday, November 7th, 13:15-14:00, in rooms BC01, BC02, and INJ218.
The quiz is open-book: all documents are allowed. Please do not print excessive amounts of material.
The final exam will be held on Thursday, December 19th, 13:15-16:00. It is closed-book, but you are allowed one cheat sheet (A4, single-sided, two-sided if handwritten).
Please make sure you do not have a conflict with either of these dates. No special arrangements will be possible.
- Course Content (Page)
- References & Docs (Page)
- Description of the Intelligent Agents course (Page)
- News & Announcements (Forum)
- Q&A (Forum)
- Logist platform manual (File)
- logist javadoc (File)
- Repast javadoc (File)
- Quiz from 2022 (File)
- Quiz from 2022 with solutions (File)
- Seating Plan BC01 (File)
- Seating Plan BC02 (File)
- Seating plan INJ218 (File)
- Seat assignments for midterm exam (File)
- Midterm quiz solved (File)
- Seating plan for final exam (File)
8-14 September: Introduction and Overview
Introduction and Overview
- Introductory Remarks (URL)
- Recording of 1st week lecture (Introduction) (URL)
- Slides: Introduction and Overview (File)
- Bibliography (Page)
- Introduction Exercise: Setting up the Environment (Page)
- Supplementary reading: Computational Rationality (File)
15-21 September:Reactive Agents
Reactive Agents
- Recording of 2nd week lecture (Reactive Agents) (URL)
- Slides: Reactive Agents (File)
- Bibliography (Page)
- Programming Exercise 1: Reactive Agents (Folder)
- Supplementary reading: Reinforcement Learning Survey (File)
- Supplementary reading: POMDP Policy Iteration Paper (File)
- Reinforcement Renaissance (File)
22-28 September: Learning Agents
Learning Agents
- Recording of 3rd week lecture (Learning Agents) (URL)
- Slides: Machine learning for agents (File)
- Bibliography (Page)
- Supplementary Reading: Multi-armed Bandit Problem (File)
- Supplementary Reading: Adversarial Bandit Algorithms (File)
29 September - 5 October: Deliberative Agents
Deliberative Agents
- Slides: Deliberative Agents (File)
- Recording of 4th week lecture (Deliberative Agents) (URL)
- Bibliography (Page)
- Supplementary Reading: Survey of Monte Carlo Search (File)
- AlphaGo paper from Nature (File)
- Programming Exercise 1 Model Solution (File)
- Programming Exercise 1 Model Solution Slides (File)
- Programming Exercise 2: Deliberative Agent (Folder)
- Supplementary reading: Libratus Poker (File)
6 -12 October: Planning with Factored Representations
Planning with Factored Representations
- Slides: Planning with Factored Representations (File)
- Recording of 5th week lecture (Factored Representations) (URL)
- Bibliography (Page)
- Supplementary Reading : Efficient Solution Algorithms for Factored MDPs (File)
- Supplementary Reading: The Fast Forward Planning System (File)
13 - 19 October: Deep Learning Agents
Deep Learning Agents
- Slides: deep learning agents (File)
- Lecture recording (URL)
- Policy Gradients in a Nutshell (URL)
- Background reading on Policy Gradients (File)
- Background reading on Proximal Policy Optimization (File)
- Background reading on offline reinforcement learning (File)
- Deliberative Agent Solution Presentation Slides (File)
- Solution for deliberative agent exercise (Folder)
- Programming Exercise 3: Centralized Coordination (Folder)
20 - 26 October: Semester Break
27 October - 2 November: Multi-Agent Systems
Multi-Agent Systems
- Slides: Multiagent Systems (File)
- Recording of 7th week lecture (Multiagent Systems) (URL)
- Bibliography (Page)
- Supplementary reading: Algorithms for multiple reactive agents (File)
- Supplementary reading: Partial Global Planning (File)
- Supplementary reading: Market-based Contract Nets (File)
3 -9 November: Distributed Multi-Agent Systems; Mid-Term Exam
Distributed Multi-Agent Systems
Nov. 7th, 13:15-14:00: Quiz
- Slides: Distributed Multi-Agent Systems (File)
- Recording of 8th week lecture (Distributed Multi-Agent Systems) (URL)
- Supplementary reading: Traconet (File)
- Survey paper: Distributed Constraint Programming (File)
- Supplementary reading: Distributed UCT Algorithm (File)
10 - 16 November: Self-interested Agents (Game-Theory)
Self-interested Agents (Game-Theory)
- Slides: Self-interested Agents (Game Theory) (File)
- Recording of 9th week lecture (Introduction to Game Theory) (URL)
- Bibliography (Page)
- Exercise: Game Theory (File)
- Solutions to the game theory exercise (File)
- Supplementary reading: Economic reasoning and artificial intelligence (File)
- Programming Exercise 3 Model Solution (File)
- Programming Exercise 4: Decentralized Coordination (Folder)
- Programming Exercise 3: Centralizated Coordination - Model solution presentation (File)
17 - 23 November: Agent Mechanism Design
Agent Mechanism Design
- Slides: Auctions and Mechanism Design (File)
- Recording of 10th week lecture (Auctions and Mechanism Design) (URL)
- Bibliography (Page)
- Exercise: Mechanism Design and Auctions (File)
- Solutions: Mechanism design and Auctions (File)
24 -30 November: Equilibria and Negotiation
Equilibria and Negotiation
- Slides: Real-world Games and Negotiation (File)
- Recording of 11th week lecture (Real World Games) (URL)
- Bibliography (Page)
- Supplementary reading: Negotation in State-Oriented Domains (File)
- Programming Exercise 4 Model Solution (File)
- Tournament Format (File)
- Exercises: Uncertainty and negotiation (File)
- Solutions: Uncertainty and negotiation (File)
1 - 7 December: Coalitions and Group Decisions
Coalitions and Group Decisions
- Slides: Coalitions and Group Decisions (File)
- Recording of 12th week lecture (Coalitions and Group Decisions) (URL)
- Bibliography (Page)
- Exercise: Coalitions and Group Decisions (File)
- Solutions: Coalitions and Group Decisions (File)
- Guest Lecture on Transaction Fee Mechanisms (File)
8 - 14 December
Implementation Techniques and Applications of Agents
- Slides: Application of Agents (File)
- Recording of lecture: Agent Applications (URL)
- Slides from DaimlerChrysler (File)
- Recording of lecture: Agents at DaimlerChrysler (URL)
- Supplementary Reading: Developing Multiagent Applications with JADE. Tutorial for Beginners. (File)
- Supplementary Reading: Engineering jade agents with the Gaia methodology (File)
- Tournament 2024 (File)
15 December - 21 December: Q&A/Final Exam
Final Exam: Thursday, December 19th, 13:15-16:00
Wednesday, December 18th: Q&A Session
To prepare, please note the week and slides in the course that your question refers to on the spreadsheet (link is below). Deadline: Monday, December 16th, 19:00!
I will go through the slides you reference during the QA session and you can ask you question (or put the question in the spreadsheet already).
If you have questions regarding the exercises, ask during the exercise sessions but you can mention them here as well - if enough time, I will answer them as well.
During the exam, one A4 sheet of notes is allowed - single-sided if printed, double-sided (or two sheets) if handwritten. A calculator is also allowed (although calculations are generally simple and can be done by hand).
- Exam sample (File)
- Exam sample solutions (short) (File)
- Final exam of 2017 (File)
- Question for QA session December 18 (URL)
- Seating plan for final exam (File)
No course this week
Wednesday, 1.15 PM: presentation of the decentralized coordination + corrections of two paper exercizes (game theory and VCG tax)
Thursday, 1.15 PM: group presentations of the centralized coordination