Enabling Natural Language and Visual Image Search for an Intelligent Video Surveillance Platform

Enabling Natural Language and Visual Image Search for an Intelligent Video Surveillance Platform

Client Overview

  • $1.3B

    Annual Revenue

  • 330%

    YoY Net Income Growth

  • 36+

    Years in Operation

INDUSTRY

  • Intelligent Video Surveillance
  • AI-Driven Security & Analytics

TECH STACK

  • NLP Engine
    • Rule-Based Processing
    • POS Tagging
    • Custom NER Model
  • Vector Search
    • OpenSearch Vector Store
    • Image Embeddings
    • Cosine Similarity Matching
  • Query & Spatial Layer
    • AWS OpenSearch
    • Bounding Box / 2D Coordinate Storage
  • AI & ML Infrastructure
    • AWS Bedrock
    • Python
    • Custom Retraining Pipeline

Executive Summary

An intelligent video surveillance platform operating thousands of cameras globally needed to make its analytics accessible to non-technical security and operations teams. Traditional filter-based search required technical knowledge most operators did not have. Inferenz built two search modalities on top of the existing OpenSearch foundation: a rule-based NLP engine that parses plain-English queries into structured searches, and a vector-based visual similarity search that allows users to upload a reference image and find all matching appearances across every connected camera. A custom-trained Named Entity Recognition model handles inconsistent naming conventions across deployments and is retrained every 10 to 15 days as the platform scales.

Challenges

01

Users Could Not Express What They Needed to Find

Security and operations teams were the primary platform users but were not technical. Complex filter syntax created a steep learning curve and meant many searches returned incomplete results. Users needed to describe what they were looking for in plain English and receive accurate results immediately.

02

Keyword Filters Could Not Handle Natural Language

The existing search layer accepted structured queries only. A user wanting to find a person in a red hat near Gate A yesterday evening had no way to express that as a filter combination. The gap between what users wanted to ask and what the system could accept was too wide to bridge manually.

03

Metadata Search Could Not Solve Image-Based Investigations

Security investigations often start with a reference image of a person or vehicle. Text-based metadata search cannot match visual appearance. Operators needed to upload an image and find all matching appearances across all cameras without knowing any attribute labels in advance.

04

Camera and Location Naming Was Not Standardised

Different deployments used different naming conventions for cameras, zones, and locations. The same physical location might be called Gate A, gate-a-cam, or entrance camera 1 depending on the site. The search engine had to understand all of these as equivalent without requiring standardisation across deployments.

Our Solution

Inferenz built two search modalities on top of the existing OpenSearch foundation, both accessible to non-technical users without any filter configuration or query syntax. The NLP engine and visual search layer work independently and in combination, covering both description-based and appearance-based investigation workflows.

Rule-Based NLP Engine With POS Tagging

A rule-based NLP engine parses free-text queries into structured OpenSearch searches. Part-of-speech tagging identifies grammatical roles within a sentence. Combination rules then extract objects, attributes, locations, and time references from the input. Parsed intent maps directly to OpenSearch query fields — returning filtered event results without the user ever seeing a query interface.

Custom-Trained Named Entity Recognition Model

A Named Entity Recognition model was trained on the client's own camera names, zone labels, and location identifiers for each deployment. The model correctly resolves informal references even when naming is inconsistent across sites, and is retrained every 10 to 15 days as new deployments and cameras are added, keeping recognition accurate as the platform scales.

Vector Store for Visual Image Similarity Search

For cameras capable of generating image embeddings, a vector store was built inside OpenSearch. Each detected frame produces a vector representation. Users upload a reference image and cosine similarity matching surfaces all visually similar detections across all connected cameras, with timestamps and camera locations returned alongside each match.

Bounding Box and Spatial Query Support

Object-level bounding box coordinates (2D Cartesian X/Y) were stored alongside event data, enabling spatial queries where operators draw a region on a camera view and retrieve only events from within that defined area. This gave investigation teams precise spatial filtering without any technical configuration required.

Impact Delivered

2

Search Modalities in Production

Natural language + visual image search

10–15

Days Retraining Cycle

NER model retrained as new cameras deploy

0

Filter Configuration Required

Non-technical users search from day one

100%

Search Coverage

Queries span every connected camera simultaneously

Let’s create something truly remarkable & intelligent!

Whether you’re starting with data modernization or exploring AI copilots, we’re here to help.

Contact Us