CS 451 Assignment # 3 – Reinforcement Learning and Self Organizing Maps Solution

Assignment # 3 – Reinforcement Learning and Self Organizing Maps

Objective: The objective of this assignment is to enable students to implement a basic reinforcement learning agent. The assignment also trains them to use a specialized form of artificial neural networks (called Self Organizing Maps) for clustering and visualization.

Question 1 [25 points]
You have to write your value iteration agent that navigates in a 10 * 10 grid. The grid world offers reward (+100) at different points. Your agent is moving around (via Up, Down, Left, Right actions) to maximize its reward. Determine a policy via value iteration that guides the agent while navigating in the grid. There are certain obstacles in the grid that restrict movement of the agent.

A sample grid is shown above where obstacles are represented as red cells (-100) and rewards as green cell(s) (+100). You want to minimize unnecessary movement of agent and make them take optimal actions. You have to try with different configurations of grid and ensure that your agent is working fine in each case.
Grading:
The grading will be based on the following:
Correct Implementation of algorithm 45%
Adjustable grid configuration 15
Convergence of value function 15%
Visualisation of learned policy 25%

Question 2 [25 points] Clustering World dataset using SOM
In this question, you will apply Self Organizing Maps (SOM) on a world dataset of your choice (such as world poverty data, country-wise environmental data, COVID19 data etc) to build an effective visualization of that dataset. You have to make sure that there are multiple attributes in that dataset that make the cluster analysis meaningful.
SOM will result in the following type of clustering:
(Note: The sample is taken from World Poverty Map showing clustering based on poverty related dataset)

Where each color represents a cluster and its varying shades denotes nearby/similar clusters.
Your implementation of the algorithm and the dataset integration should be generic that support updated data files with more rows and/or columns.
Bonus:
The representative color of each country will be used to color that country on world map resulting in a visualization similar to the following:

Enhance your SOM visualization by making use of timeseries data to show changes/progression of the underlying factors over time.

Grading:
The grading will be based on the following components:
Component Weight
Correct Implementation of algorithm 50%
Generic dataset integration 20%
Visualization of SOM grid 20%
Dataset selection (relevance, uniqueness, comprehensiveness of dataset) 10%
Bonus: Visualization on map + datewise Progression 20%

Submission: You will submit a zip file containing separate folders for the source code of two questions and a combined report talking about your problem formulation and results/visualization.