CS7638-Problem Set 2 Detecting Traffic Signs and Lights Solved

Problem Set 2 is aimed at introducing basic building blocks of image processing.  Key areas that we wish to see you implement are: loading and manipulating images, producing some valued output of images, and comprehension of the structural and semantic aspects of what makes an image.  Relevant Modules are 1-2.

 

Methods to be used: In this assignment you are to use methods that work with the Hough Transform and line finding. You should also analyze how each image is composed and what patterns can be used to identify objects in a scene.   

 

You have just started working for a self-driving car company.  As your first assignment, you are asked about how the car would process traffic rules.  That is, you are tasked with detection of both traffic lights and traffic signs.  Your job is to design and implement a program that would solve both aims.

1.  Traffic Light  
First off, you are given a generic traffic light to detect from a scene.  For the sake of the problem, assume that traffic lights are shown as below: (with red, yellow, and green) lights that are vertically stacked.  You may also assume that there is no occlusion of the traffic light.

                                       

         

It is your goal to find a way to determine the state of each traffic light and position in a scene.  Position is measured from the center of the traffic light. Given that this image presents symmetry, the position of the traffic light matches the center of the yellow circle.   

 

Complete your python ps2.py such that traffic_light_detection returns the traffic light center coordinates (x, y) ie (col, row) and the color of the light that is activated (‘red’, ‘yellow’, or ‘green’). Read the function description for more details.

 

Testing:

A traffic light scene that we will test will be randomly generated, like in the following pictures and examples in the github repo.   

  

 

  

Functional assumptions:

For the sake of simplicity, we are using a basic color scheme, but assume that the scene may have different color objects and backgrounds [relevant for part 2 and 3].  The shape of the traffic light will not change, nor will the size of the individual lights relative to the traffic light.  Size range of the lights can be reasonably expected to be between 10-30 pixels in radius.  There will only be one traffic light per scene, but its size and location will be generated at random (that is, a traffic light could appear in the sky or in the road--no assumptions should be made as to its logical position).  While the traffic light will not be occluded, the objects in the background may be.   

 

Code: Complete traffic_light_detection(img_in, radii_range) 

 

Report: For each of the following images, find the traffic light. Place its coordinates and state using cv2.putText before saving your output images.

-        Input: simple_tl.png. Output: ps2-1-a-1.png 

-        Input: scene_tl_1.png. Output: ps2-1-a-2.png 

-        Input: scene_tl_2.png. Output: ps2-1-a-3.png 

-        Input: scene_tl_3.png. Output: ps2-1-a-4.png 

2.  Traffic Signs one per scene  
Now that you have detected a basic traffic light, see if you can detect road signs.  Below are five common road signs that you would see in the United States (apologies to those outside the United States)

 

Implement a way to recognize these signs:

  

 

Similar to the traffic light, you are tasked with detecting the sign in a scene and finding the (x, y) i.e (col, row) coordinates that represent the polygon’s centroid.

  

Functional assumptions:

Like above, assume that the scene may have different color objects and backgrounds.  The size and location of the traffic sign will be generated at random.  While the traffic signs will not be occluded, objects in the background may be.

 

Code: Complete the following functions. Read their documentation in ps2.py for more details.

-   yield_sign_detection(img_in) 

-   stop_sign_detection(img_in) 

-   warning_sign_detection(img_in) 

-   construction_sign_detection(img_in) 

-   do_not_enter_sign_detection(img_in) 

 

Report: For each of the following images, find the traffic sign. Place its coordinates using cv2.putText before saving your output images.

-   Input: scene_dne_1.png. Output: ps2-2-a-1.png

-   Input: scene_stp_1.png. Output: ps2-2-a-2.png

-   Input: scene_constr_1.png. Output: ps2-2-a-3.png  

-   Input: scene_wrng_1.png. Output: ps2-2-a-4.png 

-   Input: scene_yld_1.png. Output: ps2-2-a-5.png 

3.  Multiple signs in a scene  
The next task is to detect multiple traffic signs and the traffic light in one scene. Find where each sign is in the scene, below is a randomly generated example:

 

  

 

Functional assumptions:

Like above, assume that the scene may have different color objects and backgrounds.  There will be n instances of each sign and/or traffic light, where n is 0 or 1.  The size and location of each will be generated at random.  While the traffic signs will not be occluded, objects in the background may be.

 

Code: Complete traffic_sign_detection(img_in).

 

Report: Find all traffic signs and the traffic light using the images below. Place their coordinates and name (in a easy to read area) using cv2.putText before saving your output images.

-        Input: scene_some_signs.png. Output: ps2-3-a-1.png  

-        Input: scene_all_signs.png. Output: ps2-3-a-2.png 

4.  Poor Weather Conditions  
Anyone with driving experience knows that signs and lights may be harder to detect in times of inclement weather.  Though, as a CV student, you are unphased.  Your program will be given a scene like below.  Test your traffic light and sign detection against images that have a certain amount of noise.   

 

  

 

Code: Complete traffic_sign_detection_noisy(img_in).

 

Report: Find all traffic signs and the traffic light  using the images below. Place their coordinates and name using cv2.putText before saving your output images.

-        Input: scene_some_signs_noisy.png. Output: ps2-4-a-1.png  

-        Input: scene_all_signs_noisy.png. Output: ps2-4-a-2.png 

5.  Challenge problem: Using a real image  
Let’s try your code using real world images. In this section you are encouraged to go out and take some photographs of street signs like the ones above. See how well your method performs with these. In case you are not able to obtain such images, you should look for these images online. You are to use two type of images:

A.     Three images with one street sign in the scene. Label them img-5-a-1.png, img-5-a-2.png, img-5-a-

3.png.  

Three images with multiple street signs. Label them img-5-b-1.png, img-5-b-2.png, img-5-b-3.png.