Overview
The Bounding Box Detection Library detects the bounding box of a moving object in the field of view of a MP7 Bioptic scanner using OpenCV.
Bounding Box Detection Configuration
The bounding box background filter type defines how the bounding box is detected, the following table lists the possible configurations for background filter types.
Table 1: Bounding Box Detection Configuration
| Background Filter Type | Description |
|---|---|
| Static | This option assumes a static background, the background frame should be saved with initial call of the SetBackgroundFrame method, before any calls to detect bounding box. If the background or lighting conditions change, call SetBackgroundFrame to reset the background frame (where no objects are in the field of view of the MP7). |
| Dynamic | Use this option for dynamic changing background, any objects that are not moving becomes part of the background after some time. |
| Dynamic Gaussian | Use This option for dynamic changing background, any objects that are not moving becomes part of the background after some time. Additional gaussian blurring is applied before background detection. |
| Mixed | Uses a combination of methods 1 (Static) and 2 (Dynamic) to detect background. |
Bounding Box Detection Sample
C++ Bounding Box API
The C++ Bounding box detection API is defined in the camera_imaging library:
- C++ header : camera_bounding_box_factory.h
- Library : camera_imaging.lib
Include the above header and lib into the target bounding box detection application. Include/Lib directories should be specified in the target C++ project.
Dependencies :The Camera SDK bin DLLs (and OpenCV DLLs) should be in the target application folder.
The following code shows how to detect bounding box:
CameraBoundingBoxDetectorInterface* frame_proc;
//Create an instance of bounding box detector (with the specified type of background filter)
frame_proc = CameraImageProcessingFactory::CreateBoundingBoxDetector(BackgroundFilterType::kStatic);
//Call from continuous/decode/produce/snapshot event handler to get bounding box location
void Render(LPBYTE frame_buffer, LONG buffer_size, int image_width, int image_height, int image_type)
{
vector bounding_rect; //Point array of bounding box location coordinates
if (save_background_) //flag to indicate saving of background frame
{
//Save background frame
frame_proc->SetBackgroundFrame(frame_buffer,buffer_size, image_width, image_height, image_type);
save_background_ = false;
}
else
{
//Detect bounding box
bool rect_found = frame_proc->DetectBoundingBox((frame_buffer,buffer_size, image_width,
image_height, image_type,bounding_rect);
if (rect_found)
{
// Bounding box detected
int x1 = bounding_rect[0].x; //Bounding box top left x
int y1 = bounding_rect[0].y; //Bounding box top left y
int x2 = bounding_rect[3].x; //Bounding box bottom right x
int y2 = bounding_rect[3].y; //Bounding box bottom right y
}
}
The SetBackgroundFrame method is used to specify the background image. Initially call this method with an image frame of the background (with no object in the field of view of the MP7). Next call the DetectBoundingBox to detect the bounding box, if a bounding box is found the function returns true. The bounding_rect point array parameter gives the output detected location of the bounding box. The coordinates are as follows:
Figure 1: Bounding Box Coordinates
C# COM API Bounding Box Detection Sample
The C# Bounding box detection API is defined in the native library: image_proc.dll
Dependencies: In addition to the above DLL the Camera SDK bin DLLs (and OpenCV DLLs) should be in the target application folder.
In the C# project include the following native method declarations, to call bounding box detection DLL functions.
using System;
using System.Runtime.InteropServices;
namespace CameraSDKSampleApp
{
/// <summary>
/// This class contains native declarations used int the CameraSDK C# sample application.
/// </summary>
class NativeMethods
{
/// Bounding box detection image processing library native methods
/// <summary>
/// Initialize Bounding Box Detector
/// </summary>
/// <param name="background_filter_type">background_filter_type</param>
[DllImport("image_proc.dll", CallingConvention = CallingConvention.Cdecl)]
internal static extern void InitBoundingBoxDetector(int background_filter_type);
/// <summary>
/// Set Background Frame
/// </summary>
/// <param name="background_frame_buffer">Background frame image buffer</param>
/// <param name="length">length of background frame buffer</param>
/// <param name="image_width">image width</param>
/// <param name="image_height">image height</param>
/// <param name=image_type">image type (reserved BMP:0/JPEG:1)</param>
[DllImport("image_proc.dll", CallingConvention = CallingConvention.Cdecl)]
internal static extern void SetBackgroundFrame(IntPtr background_frame_buffer, int length,
int image_width, int image_height, int image_type);
/// <summary>
/// Detect Bounding Box
/// </summary>
/// <param name="input_frame_buffer">input image frame to detect bounding box </param>
/// <param name="length">length of image frame buffer</param>
/// <param name="image_width">input image width</param>
/// <param name="image_height">input image height</param>
/// <param name=image_type">input image type (reserved BMP:0/JPEG:1)</param>
/// <param name="x1">detected bounding box location top left x</param>
/// <param name="y1">detected bounding box location top left y</param>
/// <param name="x2">detected bounding box location bottom right x</param>
/// <param name="y2">detected bounding box location bottom right y</param>
/// <returns>Returns true if bounding box detected</returns>
[DllImport("image_proc.dll", CallingConvention = CallingConvention.Cdecl)]
internal static extern bool DetectBoundingBox(IntPtr input_frame_buffer, int length,
int image_width, int image_height, int image_type,
ref int x1, ref int y1, ref int x2, ref int y2);
}
}
In the C# camera image event handler the following code snippet detects the bounding box: (See C# Camera SDK Sample Application source for complete project)
private void Camera_OnImageEvent(ref byte[] gImage, object imageBuffer, int length, int width, int height, int imageType, string eventType, string decodeData = "")
{
lock (imageEventLock)
{
if (gImage == null || gImage.Length < length)
{
gImage = new byte[length];
}
((Array)imageBuffer).CopyTo(gImage, 0);
msImage.Flush();
msImage.Seek(0, SeekOrigin.Begin);
msImage.Write(gImage, 0, length);
msImage.SetLength(length);
}
curForm.BeginInvoke((MethodInvoker)delegate
{
lock (imageEventLock)
{
if (imageType != ImageTypeRaw)
{
if (saveBackground) //If background frame (first frame)
{
byte[] inData = new byte[length];
msImage.Seek(0, SeekOrigin.Begin);
msImage.Read(inData, 0, length);
msImage.Seek(0, SeekOrigin.Begin);
msImage.SetLength(length);
int size = Marshal.SizeOf(inData[0]) * inData.Length;
IntPtr inPtr = Marshal.AllocHGlobal(size);
Marshal.Copy(inData, 0, inPtr, inData.Length);
// Set Background Frame
NativeMethods.SetBackgroundFrame(inPtr, length, width, height, imageType);
saveBackground = false;
picImage.Image = Image.FromStream(msImage);
}
else
{
byte[] inData = new byte[length];
msImage.Seek(0, SeekOrigin.Begin);
msImage.Read(inData, 0, length);
msImage.Seek(0, SeekOrigin.Begin);
msImage.SetLength(length);
int size = Marshal.SizeOf(inData[0]) * inData.Length;
IntPtr inPtr = Marshal.AllocHGlobal(size);
Marshal.Copy(inData, 0, inPtr, inData.Length);
int x1 = 0;
int y1 = 0;
int x2 = 0;
int y2 = 0;
Image img = Image.FromStream(msImage);
// Detect Bounding Box
if (NativeMethods.DetectBoundingBox(inPtr, length, width, height, imageType, ref x1, ref y1, ref x2, ref y2))
{
// Draw Bounding Box
Graphics g = Graphics.FromImage(img);
g.DrawRectangle(new Pen(Brushes.LightGreen, 5), x1, y1, x2 - x1, y2 - y1);
}
picImage.Image = img;
Marshal.FreeHGlobal(inPtr);
}
}
else
{
picImage.Image = Image.FromStream(msImage);
}
}
});
}
Camera SDK COM API
The Color Camera SDK provides a COM service(“bcc_com.exe”) with the following corresponding methods /properties/events, that allows calling the Color Camera SDK API from C# or other COM clients. The COM service allows multiple clients to communicate/manage multiple connected Bioptic cameras simultaneously. It is an out-of-process COM service that gets started on demand.
Figure 2: COM API Methods
Figure 3: COM API Properties and Events
To use this COM object in C#, add a reference to the Camera SDK COM component as follows:
Figure 4: Reference Manager
Add the following namespace to the project:
using BiopticColorCamera;
Create an instance of the Camera SDK COM object as follows and call the required methods/properties.
Camera camera = new Camera();
Following is a simple C# Camera Demo Application (using the Bioptic Color Camera COM service):
/*
File: Program : Program.cs
Description: Color Camera SDK Demo Console Application
*/
using System;
using System.Threading;
using BiopticColorCamera; // Color Camera SDK namespace
namespace CameraDemoApp
{
/// <summary>
/// Color Camera SDK Demo Console Application
/// </summary>
class Program
{
const int VideoModeContinuous = 2; // Continuous video mode
const int Register = 1; // Register for event notification
const int UnRegister = 0; // Unregister for event notification
static int frameNumber = 0; // Image frame number
const int True = 1;
const int False = 0;
/// <summary>
/// Camera SDK Image event handler
/// </summary>
/// <param name="imageBuffer">Input frame buffer</param>
/// <param name="length">Length of frame</param>
/// <param name="width">Width of image</param>
/// <param name="height">Height of frame</param>
/// <param name="imageType">Image type (0:Bmp/1:Jpeg)</param>
private static void Camera_OnContinuousImageEvent(ref object imageBuffer, int length, int width, int height, int imageType)
{
Console.WriteLine("Camera frame received:" + frameNumber);
frameNumber++;
}
/// <summary>
/// Color Camera SDK Demo Program
/// </summary>
/// <param name="args"></param>
static void Main(string[] args)
{
Array devices;
int deviceIndex = 0; // Bioptic connected camera index
// Create camera instance
Camera camera = new Camera();
// Enumerate cameras
Console.WriteLine("Discovering cameras.");
camera.EnumerateDevices(out devices);
// Open camera (with index 0)
Console.WriteLine("Opening first camera");
camera.Open((string) devices.GetValue(deviceIndex));
// Get asset information
Console.WriteLine("Camera Info:");
Console.WriteLine("Model :" + camera.ModelNumber);
Console.WriteLine("Serial : " + camera.SerialNumber);
camera.SetVideoMode(VideoModeContinuous);
// Get/Set property information
int propertyValue = -1;
int defaultValue = -1;
int min = 0;
int max = 0;
int step = 1;
int isAutoEnabled = False;
int isAutoSupported = False;
// Get brightness property value
camera.GetBrightness(out propertyValue, out isAutoEnabled);
// Get brightness property info (min/max/default/step/is-auto-supported)
camera.GetBrightnessInfo(out min, out max, out defaultValue, out step, out isAutoSupported);
// Display brightness info
Console.WriteLine("Brightness Value : " + propertyValue.ToString());
Console.WriteLine("Brightness Minimum : " + min.ToString());
Console.WriteLine("Brightness Maximum : " + max.ToString());
Console.WriteLine("Brightness Default : " + defaultValue.ToString());
Console.WriteLine("Brightness Step : " + step.ToString());
// New brightness value = min value + step
int newValue = min + step;
// Set new brightness value
camera.SetBrightness(newValue, isAutoEnabled);
// Get exposure property value
camera.GetExposure(out propertyValue, out isAutoEnabled);
// Get exposure property info (min/max/default/step/is-auto-supported)
camera.GetExposureInfo(out min, out max, out defaultValue, out step, out isAutoSupported);
// Display exposure info
Console.WriteLine("Exposure Value : " + propertyValue.ToString());
Console.WriteLine("Exposure Minimum : " + min.ToString());
Console.WriteLine("Exposure Maximum : " + max.ToString());
Console.WriteLine("Exposure Default : " + defaultValue.ToString());
Console.WriteLine("Exposure Step : " + step.ToString());
// New exposure value = min value + step
newValue = min + step;
// Set new exposure value
camera.SetExposure(newValue, isAutoEnabled);
// Register for image events
Console.WriteLine("Registering for image events");
camera.OnContinuousImageEvent += Camera_OnContinuousImageEvent;
camera.RegisterForContinuousImageEvent(Register);
// Wait for image events to trigger
Thread.Sleep(5000);
// Unregister for image events
Console.WriteLine("Unregistering for image events");
camera.RegisterForContinuousImageEvent(UnRegister);
// Close camera connection
Console.WriteLine("Closing camera");
camera.Close();
}
}
}