RiskGraph · Our internal research platform
A threat model should show how harm could happen, what supports that account and what could change our minds. We use RiskGraph internally to structure that reasoning, examine evidence and guide our research. This walkthrough shows how we work.
As a method
Define events, trace dependencies and compare explanations. Keep assumptions separate from evidence.
As a tool
Collect judgments, preserve disagreement and explore what a model implies under stated assumptions.
As a platform
Keep model revisions, evidence, study rounds and analysis provenance connected across a research programme.
How we work
Our internal workflow follows five steps: model, elicit judgments, pool them, simulate the probabilistic model and read the results. Our research loop continues by reviewing evidence and revising the model.
01 / Define a claim
A node needs a clear statement, a time horizon and resolution criteria. Conditional questions make dependencies explicit. Definitions are frozen within study rounds so changed wording cannot silently inherit old answers.
Evidence dossiers connect a claim to sources and explain their relevance and limitations. A source can inform a model without resolving an event or establishing its probability.
02 / Challenge the assumptions
Collect probability judgments with reasons, counterarguments and “cannot assess” responses. Human contributors and AI model families remain distinct groups; explicit weights govern how their judgments are combined.
A pooled distribution can retain competing views. An argument map makes the reasons behind those views available for review.
03 / Compare model implications
Compare a baseline with an observation or a modeled intervention. These answer different questions. RiskGraph separates them and requires a causal review before enabling intervention analysis.
What follows from this model and its current inputs?
What changes if we learn that an event occurred? Other connected events can become more or less plausible.
What follows if we force an event and sever its incoming causal links? This depends on the validity of the causal model.
RiskGraph propagates uncertainty through an event graph; it does not run agents. Whether a system obeys a stop signal is tested in a separate executing environment.
04 / Read, test and revise
Inspect the model revision, study-round provenance and simulation settings behind a result. Read the uncertainty alongside the number, then identify which assumption would most benefit from stronger evidence.
Evaluation findings, literature and counterevidence can motivate a reviewed revision. Updating a model is a reviewed scientific judgment.
A platform for cumulative research
Within COAI, RiskGraph’s project library and institute map connect individual models to broader research questions. Source links, adoption records and revision signals make it possible to inspect where an assessment came from and when it needs attention during internal review.
Coverage gaps, disagreement and uncertainty help identify where further work may be useful. They are signals for research judgment, not a single score of scientific quality.
Cross-project links do not automatically preserve correlations or make separate models valid to combine. Composition needs its own review.
From method to evidence
Our STPA analysis of human oversight of a swarm of agents is set up as a RiskGraph model. Our monitoring pilot supplies evidence for one of its nodes: whether a violation is noticed in time.