• Introduction
The goal of this assignment is to do experiment with Dynamic Programming(DP), including iterative policy evaluation, policy iteration and value iteration. Your goal is to implement DP methods and test them in the small gridworld men-tioned in the slides of Lecture 3.
• Small Gridworld
Figure 1: Gridworld
As shown in Fig.1, each grid in the Gridworld represents a certain state. Let st denotes the state at grid t. Hence the state space can be denoted as S = fstjt 2 0; ::; 35g. S1 and S35 are terminal states, where the others are non-terminal states and can move one grid to north, east, south and west. Hence the action space is A = fn; e; s; w g. Note that actions leading out of the Gridworld leave state unchanged. Each movement get a reward of -1 until the terminal state is reached.
A good policy should be able to nd the shortest way to the terminal state randomly given an initial non-terminal state.
• Experiment Requirments
Programming language: python3
You should build the Gridworld environment and implement iterative pol-icy evaluation methods and policy iteration methods. Then run the two methods to evaluate and improve an uniform random policy (nj ) = (ej ) = (sj ) = (wj ) = 0:25
2
• Report and Submission
Your report and source code should be compressed and named after \stu-dentID+name".
The les should be submitted on Canvas before Apr. 3, 2020.
3
