Bounding Box Detection

Overview

The Bounding Box Detection Library detects the bounding box of a moving object in the field of view of a MP7 Bioptic scanner using OpenCV.


Bounding Box Detection Configuration

The bounding box background filter type defines how the bounding box is detected, the following table lists the possible configurations for background filter types.

Table 1: Bounding Box Detection Configuration

Background Filter Type Description
Static

This option assumes a static background, the background frame should be saved with initial call of the SetBackgroundFrame method, before any calls to detect bounding box. If the background or lighting conditions change, call SetBackgroundFrame to reset the background frame (where no objects are in the field of view of the MP7).

Dynamic Use this option for dynamic changing background, any objects that are not moving becomes part of the background after some time.
Dynamic Gaussian Use This option for dynamic changing background, any objects that are not moving becomes part of the background after some time. Additional gaussian blurring is applied before background detection.
Mixed Uses a combination of methods 1 (Static) and 2 (Dynamic) to detect background.

Bounding Box Detection Sample

C++ Bounding Box API

The C++ Bounding box detection API is defined in the camera_imaging library:

  • C++ header : camera_bounding_box_factory.h
  • Library : camera_imaging.lib

Include the above header and lib into the target bounding box detection application. Include/Lib directories should be specified in the target C++ project.

Dependencies :The Camera SDK bin DLLs (and OpenCV DLLs) should be in the target application folder.

The following code shows how to detect bounding box:


CameraBoundingBoxDetectorInterface* frame_proc;

//Create an instance of bounding box detector (with the specified type of background filter)
frame_proc = CameraImageProcessingFactory::CreateBoundingBoxDetector(BackgroundFilterType::kStatic);


//Call from continuous/decode/produce/snapshot event handler to get bounding box location
void Render(LPBYTE frame_buffer, LONG buffer_size, int image_width, int image_height, int image_type)
{
    vector bounding_rect; //Point array of bounding box location coordinates

    if (save_background_) //flag to indicate saving of background frame
    {
        //Save background frame
        frame_proc->SetBackgroundFrame(frame_buffer,buffer_size, image_width, image_height, image_type);
        save_background_ = false;
    }
    else
    {
        //Detect bounding box
        bool rect_found = frame_proc->DetectBoundingBox((frame_buffer,buffer_size, image_width,
        image_height, image_type,bounding_rect);

    if (rect_found)
    {
        // Bounding box detected
        int x1 = bounding_rect[0].x; //Bounding box top left x  
        int y1 = bounding_rect[0].y; //Bounding box top left y  
        int x2 = bounding_rect[3].x; //Bounding box bottom right x  
        int y2 = bounding_rect[3].y; //Bounding box bottom right y
    }
}

The SetBackgroundFrame method is used to specify the background image. Initially call this method with an image frame of the background (with no object in the field of view of the MP7). Next call the DetectBoundingBox to detect the bounding box, if a bounding box is found the function returns true. The bounding_rect point array parameter gives the output detected location of the bounding box. The coordinates are as follows:

Figure 1: Bounding Box Coordinates

C# COM API Bounding Box Detection Sample

The C# Bounding box detection API is defined in the native library: image_proc.dll

Dependencies: In addition to the above DLL the Camera SDK bin DLLs (and OpenCV DLLs) should be in the target application folder.

In the C# project include the following native method declarations, to call bounding box detection DLL functions.


using System;
using System.Runtime.InteropServices;

namespace CameraSDKSampleApp
{
    /// <summary>
    /// This class contains native declarations used int the CameraSDK C# sample application.
    /// </summary>
    class NativeMethods
    {
        ///  Bounding box detection image processing library native methods
        /// <summary>
        /// Initialize Bounding Box Detector
        /// </summary>
        /// <param name="background_filter_type">background_filter_type</param>
        [DllImport("image_proc.dll", CallingConvention = CallingConvention.Cdecl)]
        internal static extern void InitBoundingBoxDetector(int background_filter_type);

        /// <summary>
        /// Set Background Frame
        /// </summary>
        /// <param name="background_frame_buffer">Background frame image buffer</param>
        /// <param name="length">length of background frame buffer</param>
        /// <param name="image_width">image width</param>
        /// <param name="image_height">image height</param>
        /// <param name=image_type">image type (reserved BMP:0/JPEG:1)</param>
        [DllImport("image_proc.dll", CallingConvention = CallingConvention.Cdecl)]
        internal static extern void SetBackgroundFrame(IntPtr background_frame_buffer, int length, 
        int image_width, int image_height, int image_type);

        /// <summary>
        /// Detect Bounding Box
        /// </summary>
        /// <param name="input_frame_buffer">input image frame to detect bounding box </param>
        /// <param name="length">length of image frame buffer</param>
        /// <param name="image_width">input image width</param>
        /// <param name="image_height">input image height</param>
        /// <param name=image_type">input image type (reserved BMP:0/JPEG:1)</param>
        /// <param name="x1">detected bounding box location top left x</param>
        /// <param name="y1">detected bounding box location top left y</param>
        /// <param name="x2">detected bounding box location bottom right x</param>
        /// <param name="y2">detected bounding box location bottom right y</param>
        /// <returns>Returns true if bounding box detected</returns>
        [DllImport("image_proc.dll", CallingConvention = CallingConvention.Cdecl)]
        internal static extern bool DetectBoundingBox(IntPtr input_frame_buffer, int length,
        int image_width, int image_height, int image_type, 
        ref int x1, ref int y1, ref int x2, ref int y2);
    }
}

In the C# camera image event handler the following code snippet detects the bounding box: (See C# Camera SDK Sample Application source for complete project)


private void Camera_OnImageEvent(ref byte[] gImage, object imageBuffer, int length, int width, int height, int imageType, string eventType, string decodeData = "")
{
    lock (imageEventLock)
    {
        if (gImage == null || gImage.Length < length)
        {
            gImage = new byte[length];
        }
        ((Array)imageBuffer).CopyTo(gImage, 0);
        msImage.Flush();
        msImage.Seek(0, SeekOrigin.Begin);
        msImage.Write(gImage, 0, length);
        msImage.SetLength(length);
    }
    curForm.BeginInvoke((MethodInvoker)delegate
    {
        lock (imageEventLock)
        {
            if (imageType != ImageTypeRaw)
            {
                if (saveBackground) //If background frame (first frame)
                {
                    byte[] inData = new byte[length];
                    msImage.Seek(0, SeekOrigin.Begin);
                    msImage.Read(inData, 0, length);
                    msImage.Seek(0, SeekOrigin.Begin);
                    msImage.SetLength(length);
                    int size = Marshal.SizeOf(inData[0]) * inData.Length;
                    IntPtr inPtr = Marshal.AllocHGlobal(size);
                    Marshal.Copy(inData, 0, inPtr, inData.Length);
                    // Set Background Frame
                    NativeMethods.SetBackgroundFrame(inPtr, length, width, height, imageType);
                    saveBackground = false;
                    picImage.Image = Image.FromStream(msImage);
                }
                else
                {
                    byte[] inData = new byte[length];
                    msImage.Seek(0, SeekOrigin.Begin);
                    msImage.Read(inData, 0, length);
                    msImage.Seek(0, SeekOrigin.Begin);
                    msImage.SetLength(length);
                    int size = Marshal.SizeOf(inData[0]) * inData.Length;
                    IntPtr inPtr = Marshal.AllocHGlobal(size);
                    Marshal.Copy(inData, 0, inPtr, inData.Length);
                    int x1 = 0;
                    int y1 = 0;
                    int x2 = 0;
                    int y2 = 0;
                    Image img = Image.FromStream(msImage);
                    // Detect Bounding Box
                    if (NativeMethods.DetectBoundingBox(inPtr, length, width, height, imageType, ref x1, ref y1, ref x2, ref y2))
                    {  
                        // Draw Bounding Box
                        Graphics g = Graphics.FromImage(img);
                        g.DrawRectangle(new Pen(Brushes.LightGreen, 5), x1, y1, x2 - x1, y2 - y1);
                    }
                    picImage.Image = img;
                    Marshal.FreeHGlobal(inPtr);
                }
            }
            else
            {
                picImage.Image = Image.FromStream(msImage);
            }
        }
    });
}


Camera SDK COM API

The Color Camera SDK provides a COM service(“bcc_com.exe”) with the following corresponding methods /properties/events, that allows calling the Color Camera SDK API from C# or other COM clients. The COM service allows multiple clients to communicate/manage multiple connected Bioptic cameras simultaneously. It is an out-of-process COM service that gets started on demand.

Figure 2: COM API Methods
Figure 3: COM API Properties and Events

To use this COM object in C#, add a reference to the Camera SDK COM component as follows:

Figure 4: Reference Manager

Add the following namespace to the project:


using BiopticColorCamera;

Create an instance of the Camera SDK COM object as follows and call the required methods/properties.


Camera camera = new Camera();

Following is a simple C# Camera Demo Application (using the Bioptic Color Camera COM service):


/*
File: Program : Program.cs
Description: Color Camera SDK Demo Console Application
*/

using System;
using System.Threading;
using BiopticColorCamera;   // Color Camera SDK namespace

namespace CameraDemoApp
{
    /// <summary>
    ///  Color Camera SDK Demo Console Application
    /// </summary>
    class Program
    {
        const int VideoModeContinuous = 2; // Continuous video mode
        const int Register = 1; // Register for event notification
        const int UnRegister = 0; // Unregister for event notification
        static int frameNumber = 0; // Image frame number
        const int True = 1;
        const int False = 0;

        /// <summary>
        /// Camera SDK Image event handler
        /// </summary>
        /// <param name="imageBuffer">Input frame buffer</param>
        /// <param name="length">Length of frame</param>
        /// <param name="width">Width of image</param>
        /// <param name="height">Height of frame</param>
        /// <param name="imageType">Image type (0:Bmp/1:Jpeg)</param>
        private static void Camera_OnContinuousImageEvent(ref object imageBuffer, int length, int width, int height, int imageType)
        {
            Console.WriteLine("Camera frame received:" + frameNumber);
            frameNumber++;
        }

        /// <summary>
        /// Color Camera SDK Demo Program
        /// </summary>
        /// <param name="args"></param>
        static void Main(string[] args)
        {
            Array devices;
            int deviceIndex = 0; // Bioptic connected camera index

            // Create camera instance
            Camera camera = new Camera();

            // Enumerate cameras
            Console.WriteLine("Discovering cameras.");
            camera.EnumerateDevices(out devices);

            // Open camera (with index 0) 
            Console.WriteLine("Opening first camera");
            camera.Open((string) devices.GetValue(deviceIndex));

            // Get asset information
            Console.WriteLine("Camera Info:");
            Console.WriteLine("Model :" + camera.ModelNumber);
            Console.WriteLine("Serial : " + camera.SerialNumber);
            camera.SetVideoMode(VideoModeContinuous);

            // Get/Set property information
            int propertyValue = -1;
            int defaultValue = -1;
            int min = 0;
            int max = 0;
            int step = 1;
            int isAutoEnabled = False;
            int isAutoSupported = False;

            // Get brightness property value
            camera.GetBrightness(out propertyValue, out isAutoEnabled);
            // Get brightness property info (min/max/default/step/is-auto-supported)
            camera.GetBrightnessInfo(out min, out max, out defaultValue, out step, out isAutoSupported);

            // Display brightness info
            Console.WriteLine("Brightness Value   : " + propertyValue.ToString());
            Console.WriteLine("Brightness Minimum : " + min.ToString());
            Console.WriteLine("Brightness Maximum : " + max.ToString());
            Console.WriteLine("Brightness Default : " + defaultValue.ToString());
            Console.WriteLine("Brightness Step    : " + step.ToString());

            // New brightness value = min value + step
            int newValue = min + step;
            // Set new brightness value
            camera.SetBrightness(newValue, isAutoEnabled);

            // Get exposure property value
            camera.GetExposure(out propertyValue, out isAutoEnabled);
            // Get exposure property info (min/max/default/step/is-auto-supported)
            camera.GetExposureInfo(out min, out max, out defaultValue, out step, out isAutoSupported);

            // Display exposure info
            Console.WriteLine("Exposure Value   : " + propertyValue.ToString());
            Console.WriteLine("Exposure Minimum : " + min.ToString());
            Console.WriteLine("Exposure Maximum : " + max.ToString());
            Console.WriteLine("Exposure Default : " + defaultValue.ToString());
            Console.WriteLine("Exposure Step    : " + step.ToString());

            // New exposure value = min value + step
            newValue = min + step;
            // Set new exposure value
            camera.SetExposure(newValue, isAutoEnabled);


            // Register for image events
            Console.WriteLine("Registering for image events");
            camera.OnContinuousImageEvent += Camera_OnContinuousImageEvent;
            camera.RegisterForContinuousImageEvent(Register);

            // Wait for image events to trigger
            Thread.Sleep(5000);

            // Unregister for image events
            Console.WriteLine("Unregistering for image events");
            camera.RegisterForContinuousImageEvent(UnRegister);

            // Close camera connection
            Console.WriteLine("Closing camera");
            camera.Close();
        }
    }
}