Designing a Three Reviewer Consensus Platform for Digital Harm Reporting

Agency Work for the South African Government

I was freelancing through an agency and ended up working on a platform for the South African government. Real411 is where citizens report digital harms. Disinformation, hate speech, online harassment. The kind of content that poisons public discourse.

The responsibility was heavy. Every report needed to be handled carefully because getting it wrong meant either letting harmful content stay up or silencing legitimate speech. There was no room for error.

Why Three Reviewers

A single reviewer could be biased. Two reviewers could disagree and deadlock. Three gave us a system where consensus was possible without being impossible to reach.

Each reviewer comes from a different category. Different sectors, different expertise. A legal expert, a technical expert, a community representative. They evaluate the same report independently and cannot see each other’s decisions until all three have submitted.

More importantly, reviewers from different categories cannot communicate outside the platform with each other. They do not know who the other reviewers are. They do not work for the same organization. This isolation prevents collusion and ensures each decision is truly independent.

The AI Problem

We wanted to use AI to help reviewers by providing summaries and overviews of reports. But when you are dealing with sensitive content, AI introduces risks. What if the summary misses a crucial detail? What if the AI hallucinates something that influences the reviewer’s decision?

We had to be extremely careful about where we placed AI in the workflow. It could help triage and organize, but it could never make or suggest a decision. The human reviewers remained the sole authority on every case.

How the Workflow Works

A report comes in and gets assigned to three reviewers from different categories. Each reviews independently. If at least two of three agree on the outcome, the decision is applied automatically. If there is no consensus, it escalates to a senior reviewer.

Timeouts ensure no report gets stuck. If a reviewer does not respond within twenty four hours, the report is reassigned. The entire process is tracked and auditable.

What I Built

React frontend for the reviewer dashboard. Node.js API layer. PostgreSQL with row level security to ensure reviewers can only see their own assignments. AWS Step Functions to orchestrate the workflow. SQS and SNS for event driven processing between components.