SignalShield

• 

  Landing screen

• 

  Trust and safety dashboard

• 

  A detailed review of a suspicious account

• 

  Human moderation decision

• 

  A look into a network of suspicious accounts

SignalShield

SignalShield is a human-in-the-loop Trust and Safety tool that detects and explains harmful automated amplification and coordinated activity on social media.

Inspiration

Social media platforms face two related but distinct problems: harmful content and coordinated automated amplification.

A highly automated account is not necessarily malicious. Weather alerts, news feeds, customer-support systems, and other legitimate services may behave very much like bots. At the same time, a human-operated account can still spread harmful content.

This led to the central idea behind SignalShield: instead of simply asking "Is this account a bot?", a Trust and Safety system should ask a more useful question:

Is suspicious automation being used to amplify harmful content, and what evidence does a human investigator need to make an informed decision?

SignalShield was built as a human-in-the-loop Trust and Safety prototype that separates automation, harmful-content, and coordination signals before bringing them together for investigation.

For privacy and safety, the prototype uses an entirely synthetic social-media dataset. It does not scrape or analyse live social-media users.

What it does

SignalShield analyses social-media-style account and post data across three dimensions:

1. Automation likelihood - identifies behavioural patterns associated with automated activity.
2. Harmful-content signals - analyses synthetic posts for potentially harmful identity-targeted content.
3. Coordination signals - looks for suspicious relationships between accounts based on similar content, shared URLs, and closely timed posting.

These signals are combined into an overall prioritisation score:

$$ Risk = 0.35(Automation) + 0.40(HarmfulContent) + 0.25(Coordination) $$

A key safeguard is that automation alone does not make an account high-risk. For example, the demonstration includes a legitimate automated weather account with a very high automation score but almost no harmful-content signal. SignalShield therefore keeps it at low moderation risk.

Accounts requiring attention are displayed in a prioritised Trust and Safety review queue.

Investigators can open an account to inspect behavioural indicators, score explanations, flagged synthetic posts, and coordination evidence before making a decision.

SignalShield also provides a network view where relationships between accounts are generated dynamically from the detected coordination signals.

Human-in-the-loop moderation

SignalShield is designed to assist moderators, not replace them.

The system performs:

Detection → Prioritisation → Explanation → Network Discovery

The human investigator then performs:

Review → Decision

Available recommendations include:

• No violation
• Monitor account
• Reduce distribution
• Remove violating content
• Escalate for investigation
• Recommend suspension

Moderators can also record a rationale for their decision.

SignalShield itself does not automatically suspend or remove accounts.

How I built it

SignalShield uses a React and Vite frontend with a Node.js and Express backend.

The backend ingests bundled synthetic account and post data and dynamically calculates the three principal risk signals.

Automation detection

Automation likelihood is calculated from behavioural indicators including:

• posting frequency
• daily activity span
• content repetition
• median posting interval
• account age
• URL-sharing behaviour

This creates an explainable behavioural score rather than an unexplained bot label.

Harmful-content analysis

Synthetic posts pass through a local, explainable demonstration classifier that looks for combinations of identity-related and hostile language.

Flagged posts are surfaced directly to the moderator along with their category and classifier score.

The current classifier is intentionally a prototype heuristic rather than being presented as a production-ready machine-learning moderation model. A production implementation could replace this component with a validated moderation model aligned with the platform's policies.

Coordination detection

One of the most important improvements during development was moving away from predetermined network relationships.

SignalShield now compares posts across accounts using:

• token-based text similarity
• shared URLs
• posting-time proximity

Text similarity is calculated using a Jaccard-style comparison:

$$ J(A,B) = \frac{|A \cap B|}{|A \cup B|} $$

Similar posts published within a short time window provide stronger evidence of possible coordination. Shared URLs contribute an additional signal.

These factors are combined into relationship strengths, and relationships exceeding the detection threshold are used to dynamically generate the network visualisation.

An investigator can then click a node in the network and move directly into the corresponding account review.

Privacy-conscious demonstration

A major design decision was not to build the hackathon demonstration around scraping real social-media users.

Instead, SignalShield uses synthetic accounts, posts, behavioural patterns, harmful-content scenarios, and coordinated activity.

The interface clearly identifies the data as:

Synthetic demonstration dataset - no live user data

This allows the complete detection and moderation workflow to be demonstrated without collecting personal information from real users.

The architecture is designed to be data-source agnostic. In a production environment, an authorised social-media platform could replace the synthetic ingestion layer with its internal Trust and Safety data or an approved API feed.

Challenges I faced

Distinguishing automation from harm

One of the biggest conceptual challenges was avoiding the assumption that automated behaviour is inherently suspicious.

A weather service can behave more like a bot than an abusive human account.

I therefore separated automation and harmful-content detection and introduced a safeguard preventing accounts with negligible harmful-content signals from becoming high enforcement priorities purely because they are automated.

Making coordination explainable

It would have been easy to create a visually impressive but predetermined bot-network diagram.

Instead, I wanted the relationships shown in SignalShield to have an analytical basis.

The coordination engine therefore derives relationships from similarities between synthetic posts, shared links, and posting times. The interface also exposes the evidence contributing to each connection.

Avoiding automated enforcement

Another challenge was deciding what the system should actually do after identifying a suspicious account.

Because automated moderation can produce false positives and contextual mistakes, SignalShield deliberately stops before enforcement.

The system recommends and prioritises. The human investigator decides.

Demonstrating safely

Using real social-media data would have introduced unnecessary privacy, ethical, and platform-access issues for a hackathon prototype.

Building a synthetic dataset allowed me to demonstrate realistic scenarios while keeping the prototype safe and reproducible.

What I learned

Building SignalShield reinforced that Trust and Safety systems are not simply classification problems.

A useful moderation tool needs to consider behaviour, content, networks, context, explainability, and human judgement together.

I also learned how seemingly independent signals become much more useful when combined. High automation alone may be harmless. Harmful content from a single account may require content review. But high automation, harmful content, and strong cross-account coordination together can reveal a much more important investigation priority.

Technically, the project gave me experience building an end-to-end application with React, Vite, Express, dynamic scoring, text-similarity analysis, network generation, API integration, human-review workflows, GitHub version control, and cloud deployment.

What's next for SignalShield

A production version could extend the prototype with:

• validated multilingual harmful-content models
• platform-specific moderation policies
• larger-scale graph and community detection
• more sophisticated temporal coordination analysis
• model confidence and uncertainty estimates
• adversarial and evasion detection
• moderator feedback loops
• fairness and bias evaluation
• role-based investigator access
• audit and appeal workflows
• model monitoring
• secure integration with authorised platform data

The long-term goal is not to build a system that automatically decides who should be removed from a platform.

It is to give Trust and Safety investigators better evidence for finding coordinated harmful amplification while keeping consequential moderation decisions under meaningful human oversight.

Built With

• express.js
• github
• human-in-the-loop
• jaccard
• javascript
• node.js
• railway
• react
• restapi
• vercel
• vite