$1.3B
Annual Revenue
330%
YoY Net Income Growth
36+
Years in Operation
A next-generation intelligent video surveillance startup needed to build its entire data platform from scratch, before a single camera was live in production. With thousands of cameras set to generate continuous high-volume JSON event streams, the architecture had to guarantee zero packet loss, millisecond query latency, and real-time enrichment at scale. Inferenz designed and built the full IoT streaming pipeline: from camera event through EMQX and Kafka, into a Logstash enrichment layer, across DynamoDB, Aurora, and OpenSearch, with Redis caching for high-frequency lookups. The result was 100% packet integrity, no pipeline failures, and millisecond query response on terabytes of data.
The platform was being built from scratch with no existing data model, no streaming pipeline, and no search infrastructure. Every architectural choice had to be made for long-term scale before a single camera was connected. Getting these decisions wrong would mean expensive rewrites at exactly the moment the business needed to be shipping product.
Thousands of cameras continuously generated high-volume JSON event streams at hundreds of events per second. Any dropped packet was a permanent gap in the analytics record. The pipeline had to absorb peak loads without degradation, even during concurrent spikes from cameras across multiple time zones firing simultaneously.
Security and operations teams needed query responses in milliseconds. A relational database design could not simultaneously meet the latency, availability, and scalability requirements of a platform serving thousands of cameras. The access patterns demanded a purpose-built NoSQL design engineered specifically for this workload.
Raw events arriving from cameras carried only device identifiers. Enriching each event with organisation, location, and device metadata before it reached the search layer required a reliable join at ingestion time. Doing the join at query time would have added latency to every search and created a brittle dependency under load.
Inferenz designed and built the full data architecture from the ground up, covering ingestion, enrichment, storage, caching, and query performance across all connected cameras. Every layer was engineered from first principles for the platform's specific access patterns, not adapted from a general-purpose template.
The data model was designed from the ground up following NoSQL principles, not adapted from a relational schema. Partition strategies and access patterns were engineered to achieve millisecond query latency on terabytes of data while meeting high availability and horizontal scalability requirements.
Cameras publish events to EMQX, which routes them to AWS Managed Kafka. Consumer groups and partition management absorb peak throughput. The pipeline delivered 100% packet integrity throughout the entire engagement with zero event loss, zero pipeline failures, and zero data gaps in the analytics record.
Logstash picks up each event from Kafka, enriches it by joining device ID, organisation ID, and location data from DynamoDB, and pushes the fully composed record to OpenSearch. Every record in OpenSearch carries its full context from the moment it was written — no joins needed at query time.
OpenSearch was deployed as the purpose-built event search and analytics store, with index mappings designed for the platform's specific query patterns and sharding strategies for large time-range searches. Redis Cache was layered on DynamoDB to eliminate repeated round trips for high-frequency lookups.
Packet Integrity
Zero event loss across all cameras
Pipeline Failures
High availability, no outages recorded
Query Latency
On terabytes of device event data
Data Stores
Working in concert via one pipeline
Whether you’re starting with data modernization or exploring AI copilots, we’re here to help.
Contact Us
