CameraX

AI Data Capture SDK

Overview

CameraX is a Jetpack support library for Android that simplifies camera app development by providing a consistent, easy-to-use API across various Android devices. It offers backward compatibility, facilitates smoother camera operations, and integrates seamlessly with existing camera APIs. CameraX enhances the developer experience with features such as automatic lifecycle management and easy access to use cases such as preview, image capture, and image analysis.

AI Data Capture SDK supports integration with CameraX-based applications by providing:

  • EntityTrackerAnalyzer – A CameraX ImageAnalysis.Analyzer that detects, decodes and tracks Entities.
  • Detectors for BarcodeDecoder and TextOCR – Developers can utilize these detectors to create custom Analyzers.
  • EntityViewfinder – An integrated viewfinder designed to work alongside EntityTrackerAnalyzer.

For details on compatibility, refer to the CameraX requirements.

EntityTrackerAnalyzer

EntityTrackerAnalyzer implements the ImageAnalysis.Analyzer interface for real-time detection and tracking of entities. It integrates seamlessly with CameraX, processing image frames using a list of detectors to deliver aggregated entity tracking results. The analyzer efficiently manages image buffers and executes detectors asynchronously, ensuring optimal performance. It also automatically handles coordinate system transformations automatically based on the coordinate system value passed to the constructor.

Currently, BarcodeDecoder is supported as a detector in EntityTrackerAnalyzer, which uses the barcode-localizer model. For more information and to download the model, refer to Barcode Localizer Model.

Use Cases:

  1. Real-Time Entity Tracking - Develop applications that require real-time entity tracking using camera feeds.
  2. Augmented Reality Experiences - Leverage entity tracking for dynamic augmented reality (AR) experiences that respond to entity movements and interactions.
  3. Security and Surveillance Systems - Integrate into security systems that monitor and track entities in real-time.

Notes:

  • Extensibility - The analyzer is designed to be extended with additional detectors and consumer logic. However, currently only BarcodeDecoder is supported as a detector. Only a single instance of each detector type should be passed in to the constructor.
  • Lifecycle handling - Developers are responsible for managing the session throughout the Android application lifecycle. The application must manage the session, since EntityTrackerAnalyzer does not handle this internally.
  • Threading - The processing of image frames is efficiently managed using threads within the EntityTrackerAnalyzer.

Developer Guide

This section provides a step-by-step guide to setting up a barcode detection system using the Barcode Decoder and EntityTrackerAnalyzer.

  1. Initialize AI Data Capture SDK - Initialize the AI Data Capture SDK to enable Zebra-specific features in the application. This initialization should be performed in MainActivity. Ensure this initialization step is completed before making any other calls to the AI Data Capture SDK. If the initialization fails, it indicates a critical problem, and the application should not proceed further.

    if (!AIVisionSDK.getInstance(this.getApplicationContext()).init()) {
    throw new IllegalStateException("Critical component initialization failed");
}

  2. Setup Barcode Decoder - Configure the barcode decoder settings and obtain an instance:

    BarcodeDecoder.Settings settings = new BarcodeDecoder.Settings();
decorderExecutor = Executors.newSingleThreadExecutor();
BarcodeDecoder barcodeDecoder = BarcodeDecoder.getBarcodeDecoder(settings, decoderExecutor).get();

  3. Configure Camera Controller - Set up the camera controller to use the EntityTrackerAnalyzer as the ImageAnalysis.Analyzer for processing image frames:

    resultsExecutor = Executors.newSingleThreadExecutor();

cameraController.setImageAnalysisAnalyzer(
    resultsExecutor,
    new EntityTrackerAnalyzer(
        Collections.singletonList(barcodeDecoder),
        ImageAnalysis.COORDINATE_SYSTEM_VIEW_REFERENCED,
        resultsExecutor,
        new Consumer<EntityTrackerAnalyzer.Result>() {
            @Override
            public void accept(EntityTrackerAnalyzer.Result result) {
                List<? extends Entity> entities = result.getValue(barcodeDecoder);
                for (Entity entity : entities) {
                    if (entity instanceof BarcodeEntity) {
                        BarcodeEntity barcodeEntity = (BarcodeEntity) entity;
                        Log.d(TAG, "value: " + barcodeEntity.getValue());
                    }
                }
            }
        }
    )
);

  4. Configure the Camera Controller Lifecycle - Integrate the camera controller with the activity’s lifecycle.

    cameraController.bindToLifecycle(this); 
previewView.setController(cameraController);

  5. Resource Cleanup After Usage - Properly dispose of resources after use to prevent memory leaks and ensure that all resources are released after use:

    protected void dispose() {


    // Shutdown executors to release threads
if (decorderExecutor!= null &amp;&amp; ! resultsExecutor.isShutdown()) {
    decorderExecutor.shutdown();
    decorderExecutor = null;
}

if (resultsExecutor != null &amp;&amp; !resultsExecutor.isShutdown()) {
    resultsExecutor.shutdown();
    ResultsExecutor = null;
}

// Close any clients or scanners to free up resources
if (barcodeDecoder != null) {
    barcodeDecoder.dispose();
    barcodeDecoder = null;
}

    

}

Methods

Constructor

    public EntityTrackerAnalyzer(
            @NonNull List<Detector<? extends List<? extends Entity>>> detectors,
            int targetCoordinateSystem,
            @NonNull Executor executor,
            @NonNull Consumer<EntityTrackerAnalyzer.Result> consumer
    )

Constructs an EntityTrackerAnalyzer using specified entity detectors, target coordinate system, an executor, and an Entity consumer. It initializes components for entity tracking, including a buffer manager and an entity tracker, using specified detectors. The analyzer configures entity processing and sets the coordinate system for analysis. The executor is used to execute the consumer.

Currently, only BarcodeDecoder is supported as a detector. Only one instance of BarcodeDecoder detector is accepted. Providing multiple detectors of the same type results in undefined behavior.

The EntityTrackerAnalyzer calculates the output bounding box coordinates based on the targetCoordinateSystem set by the user:

  • COORDINATE_SYSTEM_ORIGINAL - Bounding boxes are delivered in the Analyzer's coordinate system.
  • COORDINATE_SYSTEM_SENSOR (default) - Bounding boxes are delivered in the Sensor's coordinate system.
  • COORDINATE_SYSTEM_VIEW_REFERENCE - Bounding boxes are delivered in the PreviewView coordinate system.

During transformations, the Analyzer manages image rotation. All transformations then take the image rotations from the ImageInfo.getRotationDegrees().ImageAnalysis.Analyzer interface.

analyze(ImageProxy image)

    public void analyze(@NonNull ImageProxy image)

Analyzes the image with the Detectors. This method delivers the image to the detector. Typically, this method is called by ImageAnalysis when a new frame becomes available.

Parameters:

  • image - An ImageProxy object representing the image to be analyzed.

getDefaultTargetResolution()

    public Size getDefaultTargetResolution()

Returns the default target resolution for image analysis in the EntityTrackerAnalyzer.

Return Value: A Size object representing the default resolution for target images.

getTargetCoordinateSystem

    public int getTargetCoordinateSystem()

Returns the current coordinate system used inside the Analyzer. This is passed to the analyzer when it is created via the constructor.

Returns: An integer representing the coordinate system type.

updateTransform(Matrix matrix)

    public final void updateTransform(@Nullable Matrix matrix)

Receive a transformation matrix, which converts coordinates from the camera sensor’s system to the target system which was passed as targetCoordinateSystem in the constructor.

Parameters:

  • matrix - A Matrix object for transformation adjustments.

setCropRect(RectF cropRect)

    public void setCropRect(RectF cropRect)

Sets the crop rectangle. The crop rectangle specifies the region of the image that the analyzer process for entities.

Parameters:

  • cropRect - A RectF object with crop rectangle.

Detectors

The BarcodeDecoder and TextOCR classes implement a CameraX Detector interface allowing users to use CameraX as the frame source and build custom analyzers to detect and decode barcodes or recognize text in the frame delivered to the analyze() interface: This is particularly useful for applications that require to combining multiple detectors within the CameraX analyzer.

Available detectors:

  • BarcodeDecoder – Processes ImageData and generates a list of BarcodeEntity objects. It uses the process() method from BarcodeDecoder:

    process (ImageData imageData)

  • TextOCR - Processes ImageData and generates a list of ParagraphEntity objects. It uses the process() method from TextOCR:

    process (ImageData imageData)

Developer Guide

Follow the steps below to build a custom CameraX Analyzer using the BarcodeDecoder's Detector interface:

Step 1: Create a Custom Analyzer

  1. Implement ImageAnalysis.Analyzer - Implement the ImageAnalysis.Analyzer interface and override the analyze() method.

Step 2: Set Up Decoder and Localizer Settings

  1. Create BarcodeDecoder Settings: Instantiate a Settings object for the BarcodeDecoder and enable the desired symbologies.
  2. Initialize BarcodeDecoder: Obtain a BarcodeDecoder instance asynchronously using CompletableFuture.

Step 3: Process Frames

  1. Override analyze() - CameraX continuously feeds frames to the Analyzers bound to it. Override the analyzer() interface for their specific functionalities.
  2. Convert ImageProxy to ImageData - The process() interface requires ImageData. Use the fromImageProxy() method to convert from ImageProxy.
  3. Call process() - Invoke the process() method on the ImageData object to initiate barcode detection.
  4. Success Handling - Upon successful detection, addOnSuccessListener is activated, retrieving the barcode value and symbology type.
  5. Failure Handling - If detection fails, addOnFailureListener logs the error and closes the image.
  6. Exception Handling - Handle exceptions such as InvalidInputException and AIVisionSDKException by logging issues during processing and properly closing the image.

Sample Code:

    //import necessary packages

    public class CompositeAnalyzer implements ImageAnalysis.Analyzer {
        @Override
        public void analyze(@NonNull ImageProxy image) {
            try {
                barcodeDecoder.process(ImageData.fromImageProxy(image)).addOnSuccessListener(result ->{

                    for(Entity<?> bb:result){
                        // Access bounding box and confidence
                        BoundingBox boundingBox = bb.getBoundingBox();
                        float confidence = bb.accuracy();

                        // Type cast Entity to BarcodeEntity for specific operations
                        BarcodeEntity barcodeEntity = (BarcodeEntity) bb;

                        // Access barcode-specific data
                        String barcodeValue = barcodeEntity.getValue();
                        String barcodeType = barcodeEntity.getSymbology();
                    }
                    image.close();
                })

                .addOnFailureListener(e ->{
                    image.close();
                });
            } 
            catch (Exception e) {
                Log.e(TAG, "Analysis failed: " + e.getMessage());
            }
        }
    }

EntityViewfinder

EntityViewfinder is a built-in viewfinder designed for seamless integration with CameraX as a preview component. It enhances the default PreviewView by offering advanced features, such as rendering and interacting with Entities, making it a suitable choice for developers seeking to deliver a high-performance, intuitive user experience.

This viewfinder is utilized in applications requiring a visual interface for camera operations and entity detection, such as augmented reality applications, barcode scanners, and image recognition systems. It offers user interaction capabilities with camera controls, including zoom levels, flash settings, and draggable viewfinder functionality.

image

Sample EntityViewfinder Screen

Key Features:

  • Entity Rendering - The viewfinder is optimized to display Entities directly within the camera preview, eliminating the need for additional layers or complex integrations.
  • Simplified Styling - Developers can leverage prebuilt StylePens, making it easy to associate visual styles with generated Entities.
  • Integrated Camera Controls - The EntityViewfinder includes built-in camera controls such as zoom and flash, minimizing the need for manual implementation of these features.
  • Adjustable and Movable Viewfinder - The EntityViewfinder can be expanded to full-screen mode and reverted to its original dimensions using onscreen buttons. Additionlly, users can reposition the viewfinder by dragging it anywhere within the screen boundaries.

The EntityViewfinder mainly consists of three parts:

  • EntityView - A visual interface that allows for customization of the viewfinder’s appearance and behavior.
  • EntityViewController - A functional interface that manages and implements the operational features of the viewfinder.
  • StylePen - A visual representation of the entities of the EntityViewfinder.

Developer Guide

This guide illustrates the integration and configuration of EntityViewfinder with a camera system using CameraX in an Android application. It covers setting up a viewfinder for camera preview and rendering entities with customizable styles. The guide also demonstrates how to manage user interactions and dynamically update the viewfinder with detected entities.

  1. Integrate EntityView in XML Layout - Define EntityView in your XML layout file, specifying attributes to configure its initial behavior and appearance. 

    //  Integrate EntityView in XML Layout 
<com.zebra.ai.vision.viewfinder.EntityView 
    android:id="@+id/entity_view" 
    android:layout_width="match_parent" 
    android:layout_height="match_parent" 
    app:zoom="x2" 
    app:draggable="false" />

  2. Initialize EntityView and EntityViewController - Use findViewById() to obtain a reference to the EntityView, and instantiate an EntityViewController with this EntityView instance and the current context as a lifecycleOwner. The controller manages the viewfinder’s operations and interactions.

  3. Setup Camera - Use ProcessCameraProvider to manage the camera lifecycle and CameraSelector to select the back-facing camera.

  4. Create a Preview Case - Create a Preview use case and set its surface provider using the EntityViewController.

  5. Set Up Image Analysis - Define an ImageAnalysis use case to analyze the camera feed. Set the resolution and necessary configuration and assign an EntityTrackerAnalyzer to process the images.

  6. Bind Camera to Lifecycle - Unbind any previous use cases before binding the camera to the lifecycle with the configured preview and image analysis use cases. Set the camera controller in the EntityViewController.

  7. Register Event Listeners - Register listeners to handle user interactions, such as entity click events, and to manage dynanmic layout changes, like viewfinder resize events. The resize listener allows users to establish field of view and entity coordinates between the preview and analyzer.

    Sample code for steps 2 to 6: 

    // Initialize EntityView 
EntityView entityView = findViewById(R.id.entity_view); 

// Initialize the EntityViewController 
EntityViewController entityViewController = new EntityViewController(entityView, this); 

// Setup Camera 
ProcessCameraProvider cameraProvider = ProcessCameraProvider.getInstance(this).get(); 
CameraSelector cameraSelector = new CameraSelector.Builder() 
    .requireLensFacing(CameraSelector.LENS_FACING_BACK) 
    .build(); 

// Setup a Preview usecase 
Preview preview = new Preview.Builder().build(); 

// Set SurfaceProvider to preview usecase by retrieving it from the entityViewController 
preview.setSurfaceProvider(entityViewController.getSurfaceProvider()); 

// Set up image analysis use case 
ImageAnalysis imageAnalysis = new ImageAnalysis.Builder().setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST).build(); 
imageAnalysis.setAnalyzer(cameraExecutor, entityTrackerAnalyzer); 

// Unbind previous use cases before rebinding 
cameraProvider.unbindAll(); 

// Bind camera to lifecycle and get session camera instance 
camera = cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageAnalysis); 

// Configure entity view controller with camera
entityViewController.setCameraController(camera); 

// Register listener for entity click events 
entityViewController.registerEntityClickListener(entity -> { 
    // Handle entity click events 
}); 

// Register Listener for Viewfinder Resize Events 
entityViewController.registerViewfinderResizeListener(specs -> { 
    // Handle viewfinder size changes 
});

  8. Initialize StylePens for Entity Visualization - Create various StylePen instances for visualizing different entities as required. Configure these pens with desired styles, such as stroke width and icon type. 

    // Initialize various style pens as per the use case 

private void initializeStylePens() { 
    StylePen qrCodeBoundingBoxPen = BoundingBoxDrawPen.builder() 
        .setStroke(8f, Color.BLUE) 
        .build(); 


    StylePen upcaIconPen = IconDrawPen.builder() 
    .setIconStyle(IconStyle.SUCCESSFUL_DECODE) 
    .setClickable(true) 
    .build(); 

    

}

  9. Apply StylePens and Render Entities - In the analyzer results consumer method, clear previous entities and retrieve detected entities. Apply the appropriate StylePen to each entity based on your application use case and add them to the EntityViewController. Finally, render all entities on the viewfinder at once for optimized performance. 

    // Analyzer results consumer method
private void handleEntities(EntityTrackerAnalyzer.Result result) { 


    // Clear previous entities 
entityViewController.removeAll(); 

// Get detected entities 
List&lt;? extends Entity&gt; entities = result.getValue(barcodeDecoder); 

if (entities != null) { 
    Log.i(TAG, "Handle entities: " + entities.size()); 
    // Process each entity 
    for (Entity entity : entities) { 
        if (!(entity instanceof BarcodeEntity)) { 
            continue; 
        } 

        BarcodeEntity barcodeEntity = (BarcodeEntity) entity;       

        int symbology = barcodeEntity.getSymbology(); 

        // QR Code symbology 
        if (isQRCodeSymbology(symbology)) { 
            // Associate style pen initialized for QR codes 
            entityViewController.add(barcodeEntity, qrCodeBoundingBoxPen); 
        } 

        // UPCA symbology 
        if (isUPCASymbology(symbology)) { 
            // Associate style pen initialized for UPCA barcodes 
            entityViewController.add(barcodeEntity, upcaIconPen); 
        }             
    } 
} 

// Render all entities at once 
entityViewController.render(); 

    

}

image

Workflow for integrating EntityViewfinder with EntityTrackerAnalyzer

EntityView

The EntityView is a custom view designed to render the visual interface of a viewfinder. It is the UI representation of the EntityViewfinder, displaying the camera preview along with interactive controls. It includes camera controls such as zoom and flash, as well as viewfinder-related features like a full-screen toggle button and a reset position button, providing essential functionality for managing the viewfinder and camera operations.

This view is meant to be integrated into your XML layout file to define the display area for the viewfinder interface. The EntityView should be used in conjunction with the EntityViewController to ensure proper operational management and seamless integration within your application.

Currently the programmatic instantiation of the viewfinder is not supported. It is recommended to integrate it in the application using XML-based UI layouts.

Configurations

Zoom Level

The zoom attribute sets the default zoom level for the viewfinder. It supports the following enum values:

Attribute Description
zoom="x1" Sets the zoom level to 1x (default level).
zoom="x2" Sets the zoom level to 2x.
zoom="x4" Sets the zoom level to 4x.
zoom="x8" Sets the zoom level to 8x.
Flash State

The flash attribute specifies whether the flash is enabled or disabled by default.

Attribute Description
flash="true" Flash is enabled.
flash="false" Flash is disabled (default).
Visibility of Buttons

These attributes allows controls the visibility of specific UI buttons on the viewfinder.

Attribute Description Value Default
hide_reset_position_button Shows or hides the reset position button. true or false false
hide_flash_button Shows or hides the flash button. true or false false
hide_zoom_button Shows or hides the zoom button. true or false false
hide_resize_button Shows or hides the resize button. true or false false
Draggable Viewfinder

The draggable attribute determines whether the viewfinder can be dragged or repositioned by the user.

Attribute Description
draggable="true" Enables dragging functionality.
draggable="false" Disables dragging functionality (default).
Note: This automatically hides the reset position button and resize button by default.
Viewfinder Size

The Viewfinder is designed to integrate seamlessly into Android applications, delivering consistent and reliable performance across supported devices. This widget adheres to specific minimum size constraints to ensure optimal functionality and display fidelity. Developers can integrate the Viewfinder into their layouts using standard Android XML attributes.

Examples:

  • Match Parent:

    <com.zebra.ai.vision.viewfinder.EntityView
    android:id="@+id/entity_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />

  • Using Constraint Percentage (%):

    <com.zebra.ai.vision.viewfinder.EntityView
    android:id="@+id/entity_view"
    app:layout_constraintWidth_percent="1.0"
    app:layout_constraintHeight_percent="0.5"
    app:layout_constraintStart_toStartOf="parent"
    app:layout_constraintEnd_toEndOf="parent"
    app:layout_constraintTop_toBottomOf="@id/otherWidget" />

  • Fixed Dimensions:

    <com.zebra.ai.vision.viewfinder.EntityView
    android:id="@+id/entity_view"
    android:layout_width="600dp"
    android:layout_height="400dp" />
    • Note: The Viewfinder cannot be resized smaller than the specified dimensions due to enforced minimum size constraints based on the device model. If the layout specifies a size below the minimum, the Viewfinder automatically defaults to its minimum size.
Device Model Minimum Viewfinder Size (Width x Height in Pixels)
EM45 1052 x 542
TC53/TC58, TC73/TC78, TC53e/TC58e, TC22/TC27, HC50 1076 x 554
ET6X, L10ax, ET5X 10-inch, ET4X 10-inch 600 x 309

EntityViewController

EntityViewController is a key component of the EntityViewfinder, providing a high-level interface for managing the camera preview, rendering detected entities, handling user interactions, and controlling viewfinder behavior.

Methods

EntityViewController(EntityView entityView, LifecycleOwner lifecycleOwner)
    EntityViewController(EntityView entityView, LifecycleOwner lifecycleOwner)

Constructor: Creates and initializes a new EntityViewController.

Parameters:

  • entityView - The EntityView container for the viewfinder.
  • lifecycleOwner - The LifecycleOwner responsible for automatic resource management.

Exceptions:

  • InvalidInputException - Thrown if entityView or lifecycleOwner is null.
  • AIVisionSDKException - Thrown if internal initialization fails.
setCameraController(Camera sessionCamera)
    void setCameraController(Camera sessionCamera)

Associates a Camera instance with the viewfinder after successfully invoking the bindToLifecycle API. This method ensures proper configuration of the viewfinder to display preview frames and enables effective use of camera controls. It is intended for CameraProvider-based CameraX applications.

Parameters:

  • sessionCamera - The Camera instance.

Exceptions:

  • InvalidInputException - Thrown if input session camera is null.
  • AIVisionSDKException - Thrown if the camera cannot be set.
show()
    void show()

Makes the viewfinder visible.

Exceptions:

  • AIVisionSDKException - Thrown if the viewfinder cannot be shown.
hide()
    void hide()

Hides the viewfinder from the display while maintaining its internal state. The camera stream continues running even if the viewfinder is hidden.

Exceptions:

  • AIVisionSDKException - Thrown if the viewfinder cannot be hidden.
getSurfaceProvider()
    public Preview.SurfaceProvider getSurfaceProvider()

Provides the SurfaceProvider required for integrating the CameraX Preview use case.

Return Value: Returns a Preview.SurfaceProvider for camera preview integration.

Exceptions:

  • AIVisionSDKException - Thrown if the surface provider cannot be retrieved.
getWidth()
    int getWidth()

Retrieves the current width of the viewfinder display area.

Return Value: Returns the current width of the viewfinder in pixels.

Exceptions:

  • AIVisionSDKException - Thrown if the width cannot be retrieved.
getHeight()
    int getHeight()

Retrieves the current height of the viewfinder display area.

Return Value: Returns the current height of the viewfinder in pixels.

Exceptions:

  • AIVisionSDKException - Thrown if the height cannot be retrieved.
getSensorToViewTransform()
    public Matrix getSensorToViewTransform()

Retrieves the transformation matrix used to map sensor coordinates to view coordinates.

Return Value: Returns a Matrix object representing the transformation from sensor coordinates to view coordinates.

Exceptions:

  • AIVisionSDKException- Thrown if the transformation cannot be retrieved.
add(Entity entity, StylePen stylePen)
    void add(Entity entity, StylePen stylePen)

Adds an entity to be rendered on the viewfinder.

Parameters:

  • entity - The Entity to add.
  • stylePen - The StylePen used for rendering the entity.

Exceptions:

  • InvalidInputException - Thrown if entity or stylePen is null.
  • AIVisionSDKException - Thrown if adding the entity fails.
remove(Entity entity)
    void remove(Entity entity)

Removes a specific entity from the viewfinder.

Parameters:

  • entity - The Entity to remove.

Exceptions:

  • InvalidInputException - Thrown if the Entity is null.
  • AIVisionSDKException - Thrown if removing the entity fails.
removeAll()
    void removeAll()

Removes all entities from the viewfinder.

Exceptions:

  • AIVisionSDKException - Thrown if removing all entities fails.
render()
    void render()

Renders all added entities onto the viewfinder.

Exceptions:

  • AIVisionSDKException - Thrown if rendering fails.
registerEntityClickListener(EntityClickListener entityClickListener)
    void registerEntityClickListener(EntityClickListener entityClickListener)

Registers a listener to handle entity click events.

Parameters:

  • entityClickListener - The listener to register for entity click events.

Exceptions:

  • AIVisionSDKException - Thrown if the click listener registration fails.
registerViewfinderResizeListener(EntityViewResizeListener listener)
    void registerViewfinderResizeListener(EntityViewResizeListener listener)

Registers a listener to handle viewfinder resize events.

Parameters:

  • listener - The listener to register for resize events.

Exceptions:

  • AIVisionSDKException - Thrown if the listener registration fails.

Listeners

EntityClickListener

The EntityClickListener interface offers a callback mechanism for handling user interactions with entities rendered in a viewfinder. By implementing this interface, developers can define custom behaviors that are triggered when an entity is clicked, enabling applications to respond effectively to user inputs.

Implementation:

  1. Implement the EntityClickListener Interface - Define an instance of EntityClickListener by implementing its onEntityClicked() method. This method is invoked whenever a user clicks on an entity, allowing specific actions or logic to be defined in response to the click event.

    EntityClickListener entityClickListener = new EntityClickListener() {
    @Override
    public void onEntityClicked(Entity entity) {
        // Define actions to perform on the clicked entity
    }
};

  2. Register the listener - When a user clicks on an entity rendered by the viewfinder, the onEntityClicked(Entity entity) method is called with the associated entityRegister, the listener with the EntityViewController.  Use the registerEntityClickListener() method from EntityViewController to register the EntityClickListener instance. This connects the listener to the viewfinder, enabling it to receive click event notifications for entities displayed within the viewfinder.

    entityViewController.registerEntityClickListener( entityClickListener);
registerEntityClickListener(EntityClickListener listener):
    void registerEntityClickListener(EntityClickListener listener):

This method takes the listener as a parameter and ensures that it will be notified of any click events on entities within the viewfinder.

Parameters:

  • listener - The EntityClickListener instance to register.
onEntityClicked(Entity entity)
    void onEntityClicked(Entity entity)

Callback method invoked when a user clicks on an entity.

Parameters:

  • entity - The Entity that was clicked.
EntityViewResizeListener

The EntityViewResizeListener interface provides a callback mechanism to manage changes in the size and transformations of the viewfinder. It plays a crucial role in maintaining synchronization between the camera’s sensor coordinates and display coordinates, ensuring that detected entities are accurately positioned and rendered. By implementing this listener, developers can dynamically adapt their applications to changes in the viewfinder’s dimensions, ensuring proper layout adjustments and entity synchronization.

Implementing the EntityViewResizeListener Interface:

  1. Implement the EntityViewResizeListener Interface - Define an instance of EntityViewResizeListener by implementing its onViewfinderResized() method. This method is called whenever the viewfinder undergoes a resize or transformation, allowing to define specific actions or adjustments that should be made in response. 

    EntityViewResizeListener resizeListener = new EntityViewResizeListener() { 
    @Override 
    public void onViewfinderResized(EntityViewResizeSpecs specs) { 
        // Handle changes in viewfinder size or transformation 
        // This is where you can update layouts, adjust UI elements, or synchronize entities 
    } 
};

  2. Register the Listener with the EntityViewController - Use the registerViewfinderResizeListener() method from EntityViewController to register the EntityViewResizeListener instance. This connects the listener to the viewfinder, enabling it to receive notifications of any resize events. 

    // Register the listener with the EntityViewController 
entityViewController.registerViewfinderResizeListener(resizeListener);
onViewfinderResized(EntityViewResizeSpecs specs)
    void onViewfinderResized(EntityViewResizeSpecs specs)

This method is invoked whenever there are changes in the viewfinder’s dimensions or transformations. It can be used to adjust layouts, update UI elements, or synchronize entities to match the new viewfinder size or transformation. It provides the necessary information to maintain synchronization between the camera’s detection systems and the Viewfinder.

Parameters:

  • specs - An EntityViewResizeSpecs object that contains comprehensive details of the new state and transformations of the viewfinder.
registerViewfinderResizeListener(EntityViewResizeListener listener)
    void registerViewfinderResizeListener(EntityViewResizeListener listener)

This method takes the listener as a parameter and ensures that it will be notified of any changes in the viewfinder’s dimensions or transformations.

Parameters:

  • specs - An EntityViewResizeListener object that contains the listener.
EntityViewResizeSpecs

The EntityViewResizeSpecs class is an immutable data container that provides detailed specifications for viewfinder resize events, including the viewfinder’s dimensions, a transformation matrix from sensor to view coordinates, and the calculated field-of-view (FOV) crop region in sensor space.

EntityViewResizeSpecs is primarily used in conjunction with the EntityTrackerAnalyzer to maintain accurate coordinate synchronization when the viewfinder undergoes size or transformation changes. It provides essential data to update entity detection systems, ensuring that the field of view and entity positioning remain consistent with the viewfinder’s current state.

Methods for EntityViewResizeSpecs are provided below.

getHeight()
    int getHeight()

Return Value: Returns the height of the viewfinder in pixels.

getWidth()
    int getWidth()

Return Value: Returns the width of the viewfinder in pixels.

getSensorToViewMatrix()
    Matrix getSensorToViewMatrix()

Returns a copy of the sensor-to-view coordinate transformation matrix.

Return Value: A Matrix representing the transformation from sensor to view coordinates.

getViewfinderFOVCropRegion()
    RectF getViewfinderFOVCropRegion()

Calculates and returns the viewfinder’s field-of-view crop region in sensor coordinates.

Return Value: A RectF representing the viewfinder FOV in sensor coordinate space, or null if the calculation fails.

StylePen

StylePen is an interface that visually represents entities on the EntityViewfinder. It provides predefined styling options that allow developers to customize the appearance of entities.

The following built-in StylePen’s are supported:

The methods and properties available for each are provided in the subsections below.

Developer Guide

Follow these steps utilize a StylePen:

  1. Create and customize a StylePen - These pens allow for customization of stroke, fill, icon styles, and other visual attributes. Create a BoundingBoxDrawPen or IconDrawPen based on your use case.
  2. Associate Entities with the StylePen - Use the configured BoundingBoxDrawPen or IconDrawPen to add multiple entities to the controller. This applies the defined style to each entity.
  3. Render Entities - Call the render() method on the controller to draw all the entities on the viewfinder. This method processes the added entities and applies the styles, displaying them with the configured visual attributes.

Sample Code:

    //Create Stylepens 
    StylePen boundingBoxPen = BoundingBoxDrawPen.builder() 
        .setStroke(3f, Color.RED) 
        .setFill(Color.YELLOW) 
        .setClickable(true) 
        .build(); 

    IconDrawPen iconPen = IconDrawPen.builder() 
        .setIconStyle(IconStyle.ACTION) 
        .setclickable(true) 
        .setIconSizes(50, 100) 
        .build(); 

    // Associate the stylepens with Entities 
    entityViewController.add(detectedEntity1, boundingBoxPen); 
    entityViewController.add(detectedEntity2, iconPen); 
    entityViewController.add(detectedEntity3, iconPen); 

    // Render Entities 
    entityViewController.render();

BoundingBoxDrawPen

The BoundingBoxDrawPen class enables the drawing of bounding boxes around detected entities on a viewfinder, implementing the StylePen interface. It allows the customization of the bounding box appearance through a builder pattern, offering features such as adjustable stroke width and color, fill options, and clickability.

BoundingBoxDrawPen methods and properties are provided in the following subsections.

Builder builder()
    Builder builder()

Creates a new Builder instance for constructing a BoundingBoxDrawPen.

Return Value: A new Builder instance.

Exceptions:

  • AIVisionSDKException - Thrown if the BoundingBoxDrawPen cannot be created.
setStroke(float width, @ColorInt int color)
    setStroke(float width, @ColorInt int color)

Sets the stroke width and color for the bounding box.

Parameters:

  • width - The stroke width.
  • color - The stroke color.

Return Value: The Builder instance.

Exceptions:

  • IllegalArgumentException - Thrown if the width is less than or equal to 0.
setFill(@ColorInt int color)
    setFill(@ColorInt int color)

Sets the fill color for the bounding box.

Parameters:

  • color - The fill color.

Return Value: The Builder instance.

setClickable(boolean clickable)
    setClickable(boolean clickable)

Sets whether the bounding box is clickable.

Parameters: 

  • clickable - Set to true if clickable, otherwise set to false.

Return Value: The Builder instance.

StylePen build()
    StylePen build()

Builds and returns a BoundingBoxDrawPen instance.

Exceptions:

  • AIVisionSDKException - Thrown if the BoundingBoxDrawPen cannot be created due to internal errors.

Return Value: A BoundingBoxDrawPen instance.

IconDrawPen

The IconDrawPen class offers the ability to draw built-in icons on a viewfinder for the detected entities, implementing the StylePen interface. It uses a builder pattern to create instances with predefined icon styles, supporting configuration options such as icon size constraints and clickability.

IconDrawPen methods and properties are provided in the following subsections.

Builder builder()
    Builder builder()

Creates a new Builder instance for constructing an IconDrawPen.

Return Value: A new Builder instance.

setClickable(boolean clickable)
    setClickable(boolean clickable)

Sets whether the icon is clickable.

Parameters:

  • clickable - Set to true if clickable, otherwise set to false.

Return Value: The Builder instance.

setIconStyle(@NonNull IconStyle style)
    setIconStyle(@NonNull IconStyle style)

Sets the icon style.

Parameters:

  • style - Specify the IconStyle to use. The IconStyle enum defines a set of built-in visual styles for icons rendered on a viewfinder. The IconStyle enum is used in conjunction with the IconDrawPen to specify the visual representation of icons on a viewfinder. Each style indicates a different state or action related to an entity,

IconStyle Enum Description:

Enum Description
NO_ACTION Represents a neutral state with no specific action associated.
ACTION_OPERATE Indicates an actionable item, typically for entities that can be interacted with.
ACTION_RECALL Used for items that can be recalled or reverted.
ACTION_TAKEN Indicates that an action has been taken, showing a completed state.
DETECTION_ONLY Represents a detected entity without any specific action.
SUCCESSFUL_DECODE Indicates a successful decode or operation.

Return Value: Returns the Builder instance for chaining.

Exceptions:

  • AIVisionSDKException - Thrown if the style is null or invalid.
setIconSizes(int minSize, int maxSize)
    setIconSizes(int minSize, int maxSize)

Sets the size constraints for dynamic scaling.

Parameters:

  • minSize - The minimum size in pixels.
  • maxSize - The maximum size in pixels.

Return Value: The Builder instance.

Exceptions:

  • AIVisionSDKException - Thrown if the size constraints are invalid.
StylePen build()
    StylePen build()

Builds and returns an IconDrawPen instance.

Return Value: A StylePen instance configured with the specified properties.

Exceptions:

  • AIVisionSDKException: Thrown if the pen cannot be created due to internal errors.

Sample Apps

Refer to the following resources:

  • Start building your first product and shelf recognizer application with the QuickStart Sample application source.
  • Consult the Java/Kotlin snippets, which demonstrate the SDK's capabilities and can be easily integrated into your applications.
  • Access advanced use case and technology-based demos through the Showcase Application, including he AI DataCapture demo, which outlines how users can enroll and recognize products in real-time.