Object detection based on color (colour) with Python 3, OpenCV 3, Pi Camera, and Raspberry Pi 2

This Python 3 code will mark a blue object in the screen, mark it with a red dot and each time the object moves, the red dot will follow it.

The application of this could range from robot vision to automatic video camera. Feel free to adjust the treshold setting to capture the right RGB (or BGR) value of the blue color. This code is good enough to capture anything between light to dark blue. Or you can also change it so it detects red, green, or any other color ranges.

It’s pretty self explanatory, but if you need more explanation, go ahead and ask in comment section.


# import the necessary packages
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
import numpy as np
 
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 50
camera.hflip = True

rawCapture = PiRGBArray(camera, size=(640, 480))
 
# allow the camera to warmup
time.sleep(0.1)
 
# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
	# grab the raw NumPy array representing the image, then initialize the timestamp
	# and occupied/unoccupied text
        image = frame.array

        blur = cv2.blur(image, (3,3))

        #hsv to complicate things, or stick with BGR
        #hsv = cv2.cvtColor(blur,cv2.COLOR_BGR2HSV)
        #thresh = cv2.inRange(hsv,np.array((0, 200, 200)), np.array((20, 255, 255)))

        lower = np.array([76,31,4],dtype="uint8")
        #upper = np.array([225,88,50], dtype="uint8")
        upper = np.array([210,90,70], dtype="uint8")

        thresh = cv2.inRange(blur, lower, upper)
        thresh2 = thresh.copy()

        # find contours in the threshold image
        image, contours,hierarchy = cv2.findContours(thresh,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)

        # finding contour with maximum area and store it as best_cnt
        max_area = 0
        best_cnt = 1
        for cnt in contours:
                area = cv2.contourArea(cnt)
                if area > max_area:
                        max_area = area
                        best_cnt = cnt

        # finding centroids of best_cnt and draw a circle there
        M = cv2.moments(best_cnt)
        cx,cy = int(M['m10']/M['m00']), int(M['m01']/M['m00'])
        #if best_cnt>1:
        cv2.circle(blur,(cx,cy),10,(0,0,255),-1)
        # show the frame
        cv2.imshow("Frame", blur)
        #cv2.imshow('thresh',thresh2)
        key = cv2.waitKey(1) & 0xFF
 
	# clear the stream in preparation for the next frame
        rawCapture.truncate(0)
 
	# if the `q` key was pressed, break from the loop
        if key == ord("q"):
        	break

Reference

(Visited 21,932 times, 1 visits today)

46 thoughts on “Object detection based on color (colour) with Python 3, OpenCV 3, Pi Camera, and Raspberry Pi 2”

  1. Rohan Kotwani says:

    The algorithm is actually pretty good. And the methods used are really good.

  2. Sydney Bristow says:

    I’ve been looking at this on another site and there are many things that they do not explain. Re OpenCV, do you just install it per the standard commands given to install OpenCV? Another page showed running a build command and various other things like virtual environments. It seemed like it was over complicating things. I need some help with this as I was going to use it in the course of teaching a class.

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      This tutorial was written during Raspbian Wheezy era. In 2016 Raspbian got a new version called Jessie. Please follow this reference for more updated way to install OpenCV3. The codes above should work in Jessie as well. Give it a go. http://www.pyimagesearch.com/2015/10/26/how-to-install-opencv-3-on-raspbian-jessie/

      1. Sydney Bristow says:

        Thank you for your quick response. I looked at the post you mentioned but it more complex than I can manage with kids in a computer class. I may step back and use Wheezy instead. It would accomplish the goal in the time I have. I have a few Pi’s so I’ll give it a go both ways. I just need them to be able to track an object. We have various sensors, including heat sensors. Any suggestions are appreciated and again, thank you for responding quickly.

      2. Dipto Pratyaksa Dipto Pratyaksa says:

        Hi Sydney, please follow my instructions to install Open CV 3 https://www.linuxcircle.com/2015/05/18/open-computer-vision-opencv3-with-python-3-on-raspberry-pi-2/
        They are quite straight-forward copy-paste instructions. What you can do to make it even easier for your students is once you got it working on your Pi, you can clone your SD cards using DD Tool, and let your students load the SD cards clone in their Pis. The focus should be on the image processing theories and practice, and minimize the installation hassle as much as we can, to make your class interesting and engaging.

      3. Sydney Bristow says:

        Sounds good. I was looking into how to clone the image to do just that! Thank You!

  3. alex alex says:

    Hello
    I have Raspberry Pi3 , I installed OpenCv 3.1 and Python 3.4
    What should I do to run the programm? Just copy the code in a file named ‘project’.py ?
    I tryed this and it says :
    pi@raspberrypi:~ $ source ~/.profile

    pi@raspberrypi:~ $ workon cv

    (cv) pi@raspberrypi:~ $ cd programm

    (cv) pi@raspberrypi:~/programm $ python detectare.py

    Traceback (most recent call last):

    File “detectare.py”, line 2, in

    from picamera.array import PiRGBArray

    ImportError: No module named ‘picamera’

    (cv) pi@raspberrypi:~/programm $

    Please help me .
    Thank you
    Have a nice day

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      Just follow what the error says. It said you dont have python module called picamera. Please install it:
      sudo apt-get install python3-picamera

      Read more: http://picamera.readthedocs.io/en/release-1.0/install3.html

      1. alex alex says:

        Same problem …
        Any solution please?
        Thanks

        pi@raspberrypi:~ $ sudo apt-get install python3-picamera

        Reading package lists… Done

        Building dependency tree

        Reading state information… Done

        python3-picamera is already the newest version.

        0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

        pi@raspberrypi:~ $ source ~/.profile

        pi@raspberrypi:~ $ workon cv

        (cv) pi@raspberrypi:~ $ cd programm

        (cv) pi@raspberrypi:~/programm $ python detectare.py

        Traceback (most recent call last):

        File “detectare.py”, line 2, in

        from picamera.array import PiRGBArray

        ImportError: No module named ‘picamera’

      2. Dipto Pratyaksa Dipto Pratyaksa says:

        you are using virtual environment called (cv) when you run it, yet th path is on the normal environment. Makensure you install picamera in the right virtual environment.
        http://stackoverflow.com/questions/4757178/how-do-you-set-your-pythonpath-in-an-already-created-virtualenv

        i’d just abandon virtual environment completely

      3. alex alex says:

        Thank you !
        Now it’s working
        I added the fallowing command :
        export PYTHONPATH=/usr/local/lib/python3.4/site-packages:$PYTHONPATH

        Thank you

      4. Try this:-
        sudo modprobe bcm2835-v4l2

  4. David says:

    What framerate are you getting with this? Surely not 50? Also, does it cover a large spectrum/ could it cover a large spectrum of just one colour? Say, green? Or is it very specific?

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      The frame rate depends on your camera and your dedicated GPU. You can write logic to cover large spectrum, using matrix, for instance, from dark green to light green like: RGB 50, 255, 50 to RGB 80, 200, 80.

      that is where these lines come to play. You can define the lower and upper thresholds:
      lower = np.array([76,31,4],dtype=”uint8″)
      upper = np.array([210,90,70], dtype=”uint8″)

      thresh = cv2.inRange(blur, lower, upper)

      Try it anyway and let us know how you go

  5. rexry says:

    hi, may i ask that izit possible i convert this algorithm for detect red, green and blue color??
    i am doing line follower robot that able to detect different color path and make decision which path it should enter.
    hope u can response to my question asap.

    Thanks a lot and your algorithm is interesting and accurate in detecting the color. 🙂

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      Just modify the following lines to match your desired colors. You need lower and upper value for each of red, green, and blue
      lower = np.array([76,31,4],dtype=”uint8″)
      upper = np.array([210,90,70], dtype=”uint8″)

      These are hsv values. But you can use BGR without converting to HSV, if it is easier for you so:
      red_lower = np.array([30,30,200],dtype=”uint8″)
      red_upper =np.array([0,0,255],dtype=”uint8″)

      green_lower = np.array([30,200,30],dtype=”uint8″)
      green_upper =np.array([0,255,0],dtype=”uint8″)

      blue_lower = np.array([200,30,30],dtype=”uint8″)
      blue_upper =np.array([2550,0,0],dtype=”uint8″)

      And then create a function to detect each color.

      1. rexry says:

        Hi Dipto Pratyaksa,
        thanks for your help! you have solved my problem 🙂
        i want to ask that raspberry pi able to save the color of path that it has passed through??
        lets said the robot has passed through red color, where should i insert the function to save the red color ??
        and do u know that how my robot’s motor able to follow the centroids generated as it able to keep in the middle of path.

        Thanks & Regards
        Rexry

      2. Dipto Pratyaksa Dipto Pratyaksa says:

        To save the colour you just need to toggle a boolean value each time it has passed a colour. You can also use a list or array to store the colour name of each colour and then check if the list has already contained a particular one.

        You can have a motor class and control it during line checking. E.g.
        x=linelocationonthescreen ()
        If x>400:
        Motor.left ()
        Elif x<200:
        Motor.right ()
        Else:
        Motor.forward ()

      3. rexry says:

        can i change the direction of motor from centroids function??
        # finding centroids of best_cnt and draw a circle there
        M = cv2.moments(best_cnt)
        cx,cy = int(M[‘m10’]/M[‘m00’]), int(M[‘m01’]/M[‘m00’])
        #if best_cnt>1:
        cv2.circle(blur,(cx,cy),10,(0,0,255),-1)

        If M>400:
        Motor.left ()
        Elif M<200:
        Motor.right ()
        Else:
        Motor.forward ()

        izit possible ?? if possible, how i define the value of moment(best_cnt) ??
        sorry for troubleshoot u, because i am new in this field of project. 🙂

      4. Dipto Pratyaksa Dipto Pratyaksa says:

        cx and cy are the position of the center of the object.

        So you use it like this:

        If cx>400:
        Motor.left ()
        Elif cx<200:
        Motor.right ()
        Else:
        Motor.forward ()

  6. Apiz Skemmatixx says:

    how to solve this error

    Traceback (most recent call last):
    File “objectdetection.py”, line 9, in
    camera = PiCamera()
    File “/usr/lib/python2.7/dist-packages/picamera/camera.py”, line 345, in __init__
    }[(GPIO.RPI_REVISION, camera_num)]
    AttributeError: ‘module’ object has no attribute ‘RPI_REVISION’

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      $ sudo apt-get update
      $ sudo apt-get dist-upgrade
      $ sudo apt-get python-rpi.gpio

      Failing that. Just purge the package and reinstall

      1. Apiz Skemmatixx says:

        get this error:

        E: Invalid operation python-rpi.gpio

      2. Dipto Pratyaksa Dipto Pratyaksa says:

        The blog article was for python 3.
        You should install the proper gpio for python3 and run the code with python3 compiler.

  7. Aizuddin Ibnu Al-Muwari says:

    this code keep giving me, need two values to unpack, i hv no idea, i hv no experience using python…i just copy this code, i cannot trace the problem

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      You need Python 3 not Python 2.

  8. Star Lee says:

    Hi there, is it possible to return the X, Y Coordinates of the dot that detects color?

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      cx and cy will give you the coordinate. You can modify and try different algorithms and approaches, and share it with us.

  9. Star Lee says:

    Have another question, when I use this code to detect blue color, the red dot goes all over the place, is there a way to detect specifically blue color?

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      You can try using RBG, instead of HSV when detecting the blue colour, and do lots of testing with different range of blue objects to get the right detection parameters for blue. Human vision have different values to computer vision. It also depends on the luminosity of the room and your camera quality. What you see blue, does not always mean blue to computers.
      Read more about colour theory here: https://www.labnol.org/home/hue-saturation-luminosity/20104/

    2. Try converting to HSV and look for the equivalent formula for blue.

  10. Jesse Miller says:

    Awesome thanks! However, my picamera is sensitive to lighting changes which causes the overall color of the image to change. This makes color tracking difficult as even under well lit conditions, the image often has high color variability. Any idea how to prevent this?

    1. You can try other cameras, or you can change the colour scheme from RGB to HSL http://www.rapidtables.com/convert/color/rgb-to-hsl.htm
      or HSV https://en.wikipedia.org/wiki/HSL_and_HSV

  11. spospider says:

    can i make it so that if it detects blue then BLUE_DETECTED = True

    and also is there a way to open a window that shows the video stream ?

    1. Yes you can use OpenCV window for it, or use Python QT library,
      let me know if you are successful in porting this C++ code into Python: https://github.com/LaurentBerger/wxOpenCV/blob/master/wxOpenCVMain/CameraOpenCV.cpp#L521

  12. Sudharsan Sudha says:

    value error:need more than two values to unpack in line 40

  13. Anbazhagan S says:

    ValueError: need more than 2 values to unpack
    how to solve this error ??

    1. You need to supply a list of more than 2 data.

  14. Valeria Martínez says:

    Hi!

    How can I put multiple colors?

    1. Sure. Can you try and solve that algorithm to detect multiple colours? Maybe use array of colours, or 1 colour per function.

  15. tofik hodayat says:

    Sorry ,, i am still a beginner in this field
    What if you want to detect orange color?

  16. Emre Uysal says:

    hi!
    how can i detect 12 different color at the same time??????

    1. Dipto Pratyaksa Dipto Pratyaksa says:

      Just run the loop 12 times with dfferent color parameters.

      1. Emre Uysal says:

        https://uploads.disquscdn.com/images/407706c4d5b140032817df9022793ed1473121250a421b4d8ea2fc0f4123a57b.png
        hi! again i listen your advice and i did it but it doesn’t work. what is wrong in your opinion? thanks for helping 🙂

  17. dilawar cuteboy says:

    In opencv2, findContours returns just two values, contours and hierarchy. Therefore line 38 should be corrected to be:

    contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

  18. Lance__x says:

    Hi! If i don´t use the camera module and use a USB camera instead what should I change to the code for it to work?

Comments are closed.