Building an Edge AI People Counter: The Parts After Detection

A working people counter produces an entry or exit event, not a box drawn around a person. The detector is only one part of the job. A camera frame still has to arrive on time, detections have to remain attached to the same person, and the application has to decide whether that person actually crossed an entrance.

That distinction gets lost in a lot of demos. They show a model finding five people in a frame, print an FPS number in the corner, and stop there. It looks good, but it cannot yet tell a store how many visitors came in.

Start With the Whole Pipeline

A small edge counter normally has five stages:

1. Capture a frame.
2. Resize and normalize it for the model.
3. Run person detection.
4. Track detections between frames.
5. Convert track movement into entry and exit events.

Each stage can lose information. Capture may deliver old frames from a buffer, resizing can make distant people too small, detection can miss an occluded person, and a tracker can assign a new ID halfway across the doorway. The final counter inherits all of those errors.

Frame capture is a sensible place to begin because it is easy to test without the AI model. Read frames, attach monotonic timestamps, and measure how often the capture loop stalls. If the camera claims 30 FPS but the application receives frames in uneven bursts, the detector will see jumps rather than continuous movement.

OpenCV’s VideoCapture class is a convenient first interface for USB cameras, files, and network streams. But it can sit on top of different backends, so the same code may behave differently with V4L2, GStreamer, or FFmpeg. Check which backend is active, set the buffer deliberately when the backend allows it, and log failures instead of treating an empty frame as an ordinary miss.

Detection Needs Memory

A detector gives you coordinates, a class, and a confidence score for the current frame. It does not know that the person near the door is the same person detected 40 milliseconds earlier. That memory comes from a tracker.

The tracker does not need to identify anyone. It only needs a temporary ID that remains stable while an object moves through the scene. Simple centroid matching can work in a quiet, top-down view. Busier entrances usually need motion prediction and appearance or overlap cues, because paths cross and people briefly hide one another.

Line crossing then becomes a state problem. For every active track, keep the previous side of the virtual line and the current side. Emit an event only when the side changes in the expected region, then mark that crossing so a person hovering on the line does not add five visits.

The core logic can stay pleasantly small:

for track in active_tracks:
    previous = track.previous_side(counting_line)
    current = track.current_side(counting_line)

    if previous != current and not track.crossing_recorded:
        direction = "in" if current == "inside" else "out"
        publish_event(track.id, direction, frame_timestamp)
        track.crossing_recorded = True

Enter fullscreen mode Exit fullscreen mode

Real code also needs hysteresis, a valid crossing zone, minimum track age, and a way to clear the recorded flag after the person moves away. Without that, a detection jittering by a few pixels can look like repeated movement through the door.

Counting and Occupancy Are Different

Entry and exit counts are events. Occupancy is state, usually calculated as the previous occupancy plus entries minus exits. If one event is lost, occupancy stays wrong until the system is corrected. That makes health monitoring and reconciliation more important for occupancy than for a daily footfall trend.

The choice of sensor matters too. A narrow doorway with orderly traffic might be handled by a simple beam or depth sensor. A wide entrance with groups, carts, and people changing direction benefits from richer vision and tracking. Camera placement often changes accuracy more than swapping one object detector for another.

An RK3588-class device is useful when capture, preprocessing, neural inference, and event logic need to run in the same small box. Its NPU can run the detector while the CPU handles tracking and networking. But the split only helps after the model has been converted successfully and the surrounding pipeline avoids unnecessary copies.

Make Failures Visible

The counter should report more than counts. Useful health data includes the last frame timestamp, capture FPS, inference time, number of active tracks, event queue depth, device temperature, and model version. If a camera is covered or a process has stopped, the dashboard should say the count is stale rather than quietly display zero visitors.

And test the finished event stream, not just detector precision. Record short periods of representative traffic and compare manual entry and exit totals with the output. Include groups walking side by side, people turning around, staff standing near the line, changing sunlight, and a temporary network failure.

Privacy is easier when raw frames never leave the device, though local processing is not a magic switch. Debug recordings, live preview endpoints, logs, and retained snapshots can still expose images. Keep them disabled by default, restrict access during commissioning, and publish anonymous events rather than track histories.

A reliable edge AI people counter is mostly careful state management around a neural network. The model finds people; the rest of the application decides whether a movement became a real visit, whether the result can be trusted, and what happens when one part of the pipeline fails. That is less impressive in a demo than colorful bounding boxes, but it is the part a deployed system actually needs.