Object detection based on color (colour) with Python 3, OpenCV 3, Pi Camera, and Raspberry Pi 2

This Python 3 code will mark a blue object in the screen, mark it with a red dot and each time the object moves, the red dot will follow it.

The application of this could range from robot vision to automatic video camera. Feel free to adjust the treshold setting to capture the right RGB (or BGR) value of the blue color. This code is good enough to capture anything between light to dark blue. Or you can also change it so it detects red, green, or any other color ranges.

It’s pretty self explanatory, but if you need more explanation, go ahead and ask in comment section.


# import the necessary packages
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
import numpy as np
 
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
camera.resolution = (640, 480)
camera.framerate = 50
camera.hflip = True

rawCapture = PiRGBArray(camera, size=(640, 480))
 
# allow the camera to warmup
time.sleep(0.1)
 
# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
	# grab the raw NumPy array representing the image, then initialize the timestamp
	# and occupied/unoccupied text
        image = frame.array

        blur = cv2.blur(image, (3,3))

        #hsv to complicate things, or stick with BGR
        #hsv = cv2.cvtColor(blur,cv2.COLOR_BGR2HSV)
        #thresh = cv2.inRange(hsv,np.array((0, 200, 200)), np.array((20, 255, 255)))

        lower = np.array([76,31,4],dtype="uint8")
        #upper = np.array([225,88,50], dtype="uint8")
        upper = np.array([210,90,70], dtype="uint8")

        thresh = cv2.inRange(blur, lower, upper)
        thresh2 = thresh.copy()

        # find contours in the threshold image
        image, contours,hierarchy = cv2.findContours(thresh,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)

        # finding contour with maximum area and store it as best_cnt
        max_area = 0
        best_cnt = 1
        for cnt in contours:
                area = cv2.contourArea(cnt)
                if area > max_area:
                        max_area = area
                        best_cnt = cnt

        # finding centroids of best_cnt and draw a circle there
        M = cv2.moments(best_cnt)
        cx,cy = int(M['m10']/M['m00']), int(M['m01']/M['m00'])
        #if best_cnt>1:
        cv2.circle(blur,(cx,cy),10,(0,0,255),-1)
        # show the frame
        cv2.imshow("Frame", blur)
        #cv2.imshow('thresh',thresh2)
        key = cv2.waitKey(1) & 0xFF
 
	# clear the stream in preparation for the next frame
        rawCapture.truncate(0)
 
	# if the `q` key was pressed, break from the loop
        if key == ord("q"):
        	break

Reference

(Visited 13,432 times, 19 visits today)
  • Rohan Kotwani

    The algorithm is actually pretty good. And the methods used are really good.

  • Sydney Bristow

    I’ve been looking at this on another site and there are many things that they do not explain. Re OpenCV, do you just install it per the standard commands given to install OpenCV? Another page showed running a build command and various other things like virtual environments. It seemed like it was over complicating things. I need some help with this as I was going to use it in the course of teaching a class.

    • Dipto Pratyaksa

      This tutorial was written during Raspbian Wheezy era. In 2016 Raspbian got a new version called Jessie. Please follow this reference for more updated way to install OpenCV3. The codes above should work in Jessie as well. Give it a go. http://www.pyimagesearch.com/2015/10/26/how-to-install-opencv-3-on-raspbian-jessie/

      • Sydney Bristow

        Thank you for your quick response. I looked at the post you mentioned but it more complex than I can manage with kids in a computer class. I may step back and use Wheezy instead. It would accomplish the goal in the time I have. I have a few Pi’s so I’ll give it a go both ways. I just need them to be able to track an object. We have various sensors, including heat sensors. Any suggestions are appreciated and again, thank you for responding quickly.

        • Dipto Pratyaksa

          Hi Sydney, please follow my instructions to install Open CV 3 http://www.linuxcircle.com/2015/05/18/open-computer-vision-opencv3-with-python-3-on-raspberry-pi-2/
          They are quite straight-forward copy-paste instructions. What you can do to make it even easier for your students is once you got it working on your Pi, you can clone your SD cards using DD Tool, and let your students load the SD cards clone in their Pis. The focus should be on the image processing theories and practice, and minimize the installation hassle as much as we can, to make your class interesting and engaging.

          • Sydney Bristow

            Sounds good. I was looking into how to clone the image to do just that! Thank You!

  • alex alex

    Hello
    I have Raspberry Pi3 , I installed OpenCv 3.1 and Python 3.4
    What should I do to run the programm? Just copy the code in a file named ‘project’.py ?
    I tryed this and it says :
    pi@raspberrypi:~ $ source ~/.profile

    pi@raspberrypi:~ $ workon cv

    (cv) pi@raspberrypi:~ $ cd programm

    (cv) pi@raspberrypi:~/programm $ python detectare.py

    Traceback (most recent call last):

    File “detectare.py”, line 2, in

    from picamera.array import PiRGBArray

    ImportError: No module named ‘picamera’

    (cv) pi@raspberrypi:~/programm $

    Please help me .
    Thank you
    Have a nice day

    • Dipto Pratyaksa

      Just follow what the error says. It said you dont have python module called picamera. Please install it:
      sudo apt-get install python3-picamera

      Read more: http://picamera.readthedocs.io/en/release-1.0/install3.html

      • alex alex

        Same problem …
        Any solution please?
        Thanks

        pi@raspberrypi:~ $ sudo apt-get install python3-picamera

        Reading package lists… Done

        Building dependency tree

        Reading state information… Done

        python3-picamera is already the newest version.

        0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

        pi@raspberrypi:~ $ source ~/.profile

        pi@raspberrypi:~ $ workon cv

        (cv) pi@raspberrypi:~ $ cd programm

        (cv) pi@raspberrypi:~/programm $ python detectare.py

        Traceback (most recent call last):

        File “detectare.py”, line 2, in

        from picamera.array import PiRGBArray

        ImportError: No module named ‘picamera’

  • David

    What framerate are you getting with this? Surely not 50? Also, does it cover a large spectrum/ could it cover a large spectrum of just one colour? Say, green? Or is it very specific?

    • Dipto Pratyaksa

      The frame rate depends on your camera and your dedicated GPU. You can write logic to cover large spectrum, using matrix, for instance, from dark green to light green like: RGB 50, 255, 50 to RGB 80, 200, 80.

      that is where these lines come to play. You can define the lower and upper thresholds:
      lower = np.array([76,31,4],dtype=”uint8″)
      upper = np.array([210,90,70], dtype=”uint8″)

      thresh = cv2.inRange(blur, lower, upper)

      Try it anyway and let us know how you go

  • rexry

    hi, may i ask that izit possible i convert this algorithm for detect red, green and blue color??
    i am doing line follower robot that able to detect different color path and make decision which path it should enter.
    hope u can response to my question asap.

    Thanks a lot and your algorithm is interesting and accurate in detecting the color. 🙂

    • Dipto Pratyaksa

      Just modify the following lines to match your desired colors. You need lower and upper value for each of red, green, and blue
      lower = np.array([76,31,4],dtype=”uint8″)
      upper = np.array([210,90,70], dtype=”uint8″)

      These are hsv values. But you can use BGR without converting to HSV, if it is easier for you so:
      red_lower = np.array([30,30,200],dtype=”uint8″)
      red_upper =np.array([0,0,255],dtype=”uint8″)

      green_lower = np.array([30,200,30],dtype=”uint8″)
      green_upper =np.array([0,255,0],dtype=”uint8″)

      blue_lower = np.array([200,30,30],dtype=”uint8″)
      blue_upper =np.array([2550,0,0],dtype=”uint8″)

      And then create a function to detect each color.

      • rexry

        Hi Dipto Pratyaksa,
        thanks for your help! you have solved my problem 🙂
        i want to ask that raspberry pi able to save the color of path that it has passed through??
        lets said the robot has passed through red color, where should i insert the function to save the red color ??
        and do u know that how my robot’s motor able to follow the centroids generated as it able to keep in the middle of path.

        Thanks & Regards
        Rexry

        • Dipto Pratyaksa

          To save the colour you just need to toggle a boolean value each time it has passed a colour. You can also use a list or array to store the colour name of each colour and then check if the list has already contained a particular one.

          You can have a motor class and control it during line checking. E.g.
          x=linelocationonthescreen ()
          If x>400:
          Motor.left ()
          Elif x<200:
          Motor.right ()
          Else:
          Motor.forward ()

          • rexry

            can i change the direction of motor from centroids function??
            # finding centroids of best_cnt and draw a circle there
            M = cv2.moments(best_cnt)
            cx,cy = int(M[‘m10’]/M[‘m00’]), int(M[‘m01’]/M[‘m00’])
            #if best_cnt>1:
            cv2.circle(blur,(cx,cy),10,(0,0,255),-1)

            If M>400:
            Motor.left ()
            Elif M<200:
            Motor.right ()
            Else:
            Motor.forward ()

            izit possible ?? if possible, how i define the value of moment(best_cnt) ??
            sorry for troubleshoot u, because i am new in this field of project. 🙂

          • Dipto Pratyaksa

            cx and cy are the position of the center of the object.

            So you use it like this:

            If cx>400:
            Motor.left ()
            Elif cx<200:
            Motor.right ()
            Else:
            Motor.forward ()

  • Apiz Skemmatixx

    how to solve this error

    Traceback (most recent call last):
    File “objectdetection.py”, line 9, in
    camera = PiCamera()
    File “/usr/lib/python2.7/dist-packages/picamera/camera.py”, line 345, in __init__
    }[(GPIO.RPI_REVISION, camera_num)]
    AttributeError: ‘module’ object has no attribute ‘RPI_REVISION’

    • Dipto Pratyaksa

      $ sudo apt-get update
      $ sudo apt-get dist-upgrade
      $ sudo apt-get python-rpi.gpio

      Failing that. Just purge the package and reinstall

      • Apiz Skemmatixx

        get this error:

        E: Invalid operation python-rpi.gpio

        • Dipto Pratyaksa

          The blog article was for python 3.
          You should install the proper gpio for python3 and run the code with python3 compiler.

  • Aizuddin Ibnu Al-Muwari

    this code keep giving me, need two values to unpack, i hv no idea, i hv no experience using python…i just copy this code, i cannot trace the problem

    • Dipto Pratyaksa

      You need Python 3 not Python 2.

  • Star Lee

    Hi there, is it possible to return the X, Y Coordinates of the dot that detects color?

    • Dipto Pratyaksa

      cx and cy will give you the coordinate. You can modify and try different algorithms and approaches, and share it with us.

  • Star Lee

    Have another question, when I use this code to detect blue color, the red dot goes all over the place, is there a way to detect specifically blue color?

    • Dipto Pratyaksa

      You can try using RBG, instead of HSV when detecting the blue colour, and do lots of testing with different range of blue objects to get the right detection parameters for blue. Human vision have different values to computer vision. It also depends on the luminosity of the room and your camera quality. What you see blue, does not always mean blue to computers.
      Read more about colour theory here: https://www.labnol.org/home/hue-saturation-luminosity/20104/

    • Try converting to HSV and look for the equivalent formula for blue.

  • Jesse Miller

    Awesome thanks! However, my picamera is sensitive to lighting changes which causes the overall color of the image to change. This makes color tracking difficult as even under well lit conditions, the image often has high color variability. Any idea how to prevent this?

  • spospider

    can i make it so that if it detects blue then BLUE_DETECTED = True

    and also is there a way to open a window that shows the video stream ?

  • Sudharsan Sudha

    value error:need more than two values to unpack in line 40

  • Anbazhagan S

    ValueError: need more than 2 values to unpack
    how to solve this error ??

  • Valeria Martínez

    Hi!

    How can I put multiple colors?

    • Sure. Can you try and solve that algorithm to detect multiple colours? Maybe use array of colours, or 1 colour per function.

  • tofik hodayat

    Sorry ,, i am still a beginner in this field
    What if you want to detect orange color?