How I Solve Finding Lane Line Project in Udacity Self Driving Car Program

Udacity Self Driving Car Nano-degree program is using OpenCV and Jupyter Notebook as the tools. I only have a little experience using OpenCV when I was developing dPCR instrument at Formulatrix. I had no experience in using Pyhon and Jupyter Notebook, before but both are simple to understand and play with.

I spent quite some time to setup the development environment because the package manager which is conda always failed with conflicts. Finally, I tried to install all the packages manually and it works.

After succeeding setting up my environment then I start working with the test videos. At first, I succeed recognizing the first one but not perfectly working for the second one and kept iterating until I succeed with both. I was using the recommended pipeline which is also involving grayscale pipeline. But the recommended pipeline kept failing with the last video, the optional challenge video.

Finally, I gave up using the recommended pipeline and start building my own code base. And to be able to do that I need to understand all the image format in OpenCV and how each functions behave. Thankfully there is a guy, an Indian Guy that already created a crash course on OpenCV and his course covers many basic functions that I can use.

So I started to build an image class containing all the basic methods to do computer vision in OpenCV. I also decided to go full blown using OOP to solve this problem. These are all the classes that I wrote:

OOP really shines in helping me to solve this problem. But it is not enough to produce the best pipeline.

After, several days trying to find the best pipeline by coding it directly in the notebook, finally I gave up. I decided to build a simple GUI with sliders so that I can drag the sliders and can see the impact on the image directly. This simple tool really help me to find the best parameters for cv2.inrange() quickly. You can see the simple tool below:

After several iterations by testing the found parameters with all the challenge videos, finally I found the best ones that works for all the three videos:

These are my write up for the submission:

My pipeline consisted of 7 steps. But to find those 7 steps I have to build my own Python class toolkit so I can solve the problem using OOP.

1. I converted the image to HSL
2. I created 2 masks filtered by yellow and white color based on my own tools to find the appropriate cv.inrange() parameters
3. Then I combined both masks into one image using bitwise_or()
4. The next step would be to mask the resulting image with the viewport mask image
5. After that I ran the canny algorithm to detect the boundaries. It should be easy to find boundaries because the remaining pixels should contains only the lane lines
6. Once I got the boundaries I can pass it to houglines() to produce the lines
7. I filtered out the incorrect lines and produce the straight line using the y = ax + b equation and simple statistic to choose the lines that grouped most as the winner. Then I make sure I store the last drawn lines and use it if the next frame I fail to recognize the lines.

And this is one the response line of the reviewer that makes me so proud that someone has appreciate my effort:

Your algorithm did an amazing job finding the lane lines. The left and right lanes were accurately put on the video. I can see how much effort was put into this by the effectiveness of the algorithm.

You can find the source code here.