Total.js Platform and NVIDIA DeepStream: Automated AI Video Anonymization

Here we describe a solution we delivered to one of our major enterprise customers: automatic video anonymization running on real-world footage. This project is also part of our broader experience with NVIDIA DeepStream. Over the years, we have developed several solutions using NVIDIA DeepStream across different industries and use cases, and one architectural pattern consistently emerges: DeepStream handles the computationally intensive video processing and AI workloads on the GPU, while Total.js brings everything together into a complete, integrated solution.

What the system does

Before videos enter the anonymization process, they go through an initial preprocessing stage orchestrated by Total.js Flow. The original Full HD recordings are automatically converted to HD resolution using FFmpeg, reducing file sizes and optimizing the videos for subsequent AI processing.

Every day, the system processes over 20 GB of video recordings, with individual videos typically ranging from 3 to 7 hours in length. Total.js Flow manages this conversion pipeline automatically, ensuring that incoming recordings are prepared and ready for the next stage.

One particularly interesting aspect is the remarkably low memory footprint of Total.js Flow. Even while orchestrating FFmpeg video conversions, the Flow application consumes approximately 20 MB of RAM. The actual video processing is handled by FFmpeg as a separate process, allowing Total.js Flow to remain extremely lightweight while coordinating resource-intensive operations.

Once converted, the videos are placed in a queue, where they wait for an explicit anonymization request.

When anonymization is triggered, NVIDIA DeepStream takes over the computationally intensive work. The system detects people in the footage and automatically blurs their heads and faces using GPU acceleration.

However, not everyone necessarily needs to be anonymized. An operator can review the detected individuals and select those who should remain visible. The system then applies anonymization to everyone else, preserving the selected individuals in the final video.

Once processing is complete, the anonymized video is written to the configured output destination. From there, Total.js Flow continues orchestrating the workflow — sending notifications, moving files, storing results, or triggering additional services.

This approach combines GPU-accelerated AI processing with flexible workflow automation, allowing the entire pipeline to handle large volumes of long-duration video recordings with minimal manual intervention.

Two frames of a video anonymized by the system. Everyone's face is hidden automatically, including the people in the background.

The technology we use

• NVIDIA Jetson Thor with DeepStream 9 and a local Triton inference server, running in Docker.
• Application server running on Red Hat Enterprise Linux (RHEL), providing a stable and reliable environment for the backend services and overall system orchestration.
• PostgreSQL as the primary relational database for storing application data, processing metadata, and managing workflow-related information.
• A head and face detection model, with a small custom parser written in C++.
• GPU blur and H.264 encoding in the same pipeline.
• Total.js Flow to connect everything together, automate video processing workflows, and coordinate communication between system components.
• Model training: when an off-the-shelf model does not fit, we also train our own models using datasets prepared for the specific use case and package them for DeepStream.
• Total.js v5 with a plugin-based architecture, powering the main application server and user interface.

All video processing happens within the customer's infrastructure. The video footage never leaves the customer's environment, ensuring that sensitive recordings remain under their control throughout the entire process.

Where Total.js fits

NVIDIA DeepStream handles the computationally intensive video processing and AI workloads. Total.js Flow connects everything together, orchestrating the entire process across multiple locations.

Distributed Flow architecture

For this customer, we operate multiple Total.js Flow instances distributed across different locations throughout the country. Each location runs its own Flow instance with an identical implementation, allowing video processing and file transfers to be managed locally.

One of the key advantages of this architecture is automatic synchronization between Flow instances. All distributed instances are synchronized with a central master Flow. Whenever we make a change to the main Flow implementation, the update is automatically propagated to every connected location within seconds.

This means we can maintain and improve the entire distributed system from a single place, without manually updating individual installations. Whether we modify an existing workflow, fix an issue, or introduce new functionality, all locations receive the changes almost immediately.

Total.js Flow orchestration:

Reliable video transfers over WebSocket

Video recordings are transferred from geographically distributed Flow instances to the central infrastructure using the WebSocket protocol.

To make these transfers reliable, we developed two custom Total.js Flow components:

• File uploader WS — sends video files from remote locations to the central infrastructure.
• File receiver WS — receives incoming files and manages the transfer process on the receiving side.

Both components support resumable file transfers, which is particularly important when working with large video recordings across multiple locations.

Files are transmitted in 1 MB chunks. If a network connection drops during an upload, the transfer does not need to start from the beginning. Once the WebSocket connection is re-established, the components automatically resume the upload from where it was interrupted.

This significantly reduces unnecessary network traffic and makes the system more resilient to temporary connectivity issues, especially when transferring recordings that can be several hours long.

AI-powered person detection and anonymization

We also developed two dedicated Flow components that integrate directly with NVIDIA DeepStream:

Person detector receives a video, runs AI-powered detection, and returns an image of each detected person together with the corresponding timestamp in the recording. These images are then displayed in the central management interface, where an operator can review the detected individuals.

Anonymizer queue manages the GPU-based anonymization process and produces the final MP4 video. It supports processing up to four videos in a single batch.

The anonymization workflow follows a two-stage approach:

1. Initial anonymization: All detected individuals are anonymized by default, ensuring that heads and faces are blurred throughout the recording.
2. Selective de-anonymization: After reviewing the detected individuals in the central management interface, an authorized operator can select specific people who should remain visible. The system then generates the appropriate video output, keeping the selected individuals recognizable while everyone else remains anonymized.

This approach ensures that privacy is the default, while still allowing controlled exceptions when required by the customer's workflow.

Workflow orchestration and reliability

Beyond video processing and file transfers, Total.js Flow manages the operational aspects that make the entire system reliable:

• Videos wait in a queue instead of overloading the GPU.
• Processing progress is displayed in real time through Flow Activities.
• Errors are routed through dedicated outputs, allowing custom error-handling workflows.
• Unresponsive processing jobs are automatically detected and restarted.
• Distributed Flow instances stay synchronized with the central implementation.
• Interrupted file transfers automatically resume after reconnection.

The individual components remain relatively small and focused. They initiate operations, monitor progress, handle communication, and pass data to the next step in the workflow.

The real strength of Total.js Flow is how easily these components can be combined into a complete distributed system. Adding a new step, such as an email notification, external storage integration, approval process, or another AI service, often requires nothing more than connecting another component.

With this architecture, we can centrally manage a geographically distributed video-processing infrastructure while keeping individual locations synchronized, transfers reliable, and GPU workloads under control.

Other experiences with DeepStream

Anonymization is only one of the things DeepStream can do. It reads many video sources at the same time and runs AI models on them in real time, on a device that fits next to the camera. Together with Total.js, that opens a long list of ideas for any company with cameras. We have worked on several of them, and we know the rest well enough to help you judge what is realistic:

• Detecting and tracking people, counting them, measuring queues and occupancy of spaces.
• Recognizing situations, such as a person who has fallen, a crowded area or someone entering where they should not.
• Traffic and parking: counting vehicles, classifying them, measuring flow and free spaces. We have deployed this one at urban intersections and parking areas, with DeepStream on Jetson devices and the results processed in Flow.
• Detecting objects: products on a line, missing safety equipment, packages, tools, damage.
• Quality control in production, with a camera and a model trained for your parts.
• Privacy: blurring faces, people or license plates before video is stored or shared.
• Alerts and evidence: a possible event goes to the right person with the video frame attached.

In each case the pattern is the one above. DeepStream detects, Total.js Flow connects the result to your process, your data and your people, and the interface is built with Total.js too. It is a complete solution, not only a model.

Why it matters

This is the pattern we like about Total.js, and the one we have used again and again. A powerful tool, written in C++ or Python or anything else, can sit behind a Flow component. Your team gets a visual process they can read and change. The specialized technology stays where it is strong.

With DeepStream, that covers many video and AI needs. If you have one in mind, get in touch. We are happy to look at it with you.