eDIF: A European Deep Inference Fabric for Remote Interpretability of LLMs
Interpretability Research Needs GPUs Most Labs Do Not Have
Preprint on arXiv, August 2025.
Mechanistic interpretability has an access problem that rarely gets stated plainly. Inspecting what happens inside a large model means running it and capturing activations layer by layer — and for anything at frontier scale, that requires hardware most European research groups simply do not have sitting in a basement. The result is a field where the questions you are able to ask depend on the cluster your institution happens to own.
eDIF is our attempt at removing that constraint: a European Deep Inference Fabric, NDIF-compatible, built so that researchers can inspect large models remotely instead of needing to host them.
This paper is a feasibility study — we built it, opened it to researchers across Europe, and reported what happened.
What It Is
eDIF is a GPU cluster hosted at Ansbach University of Applied Sciences, interconnected with partner institutions, exposing remote model inspection through the NNsight API.
The architecture is deliberately layered, and the diagram is worth reading top to bottom because each layer removes a different obstacle:
- Client interface — the open-source NNsight library and a FastAPI client, installable with
pip. Researchers work in ordinary Python, locally. - API layer — a FastAPI server handling validation, routing and responses, with asynchronous processing and token-based authentication.
- Orchestration — a Ray cluster with FIFO queuing, load balancing across GPU nodes and fault-tolerant distributed execution.
- Execution & data — model containers, MinIO object storage for activation data, the NNsight backend, and a monitoring stack (Loki, Prometheus, InfluxDB, Grafana).
- Runtime & infrastructure — Docker on Ubuntu 22.04 with CUDA, running on 8× NVIDIA RTX A6000 with 48 GB VRAM each.
The point of that stack is that a researcher never touches any of it. They write NNsight code as though the model were local, and the fabric handles scheduling, execution and the return trip.
What 16 Researchers Actually Did With It
We ran a structured pilot study with 16 researchers from across Europe, evaluating technical performance, usability and scientific utility.
They did real interpretability work: activation patching, causal tracing, and representation analysis, on models ranging from GPT-2 up to DeepSeek-R1-70B. That range matters — the small end is where you develop a method, and the large end is where you find out whether it survives contact with a frontier-scale model. Very few groups can do both on their own hardware.
Usage grew gradually rather than spiking and dying, which is the pattern you want from a pilot — it indicates people came back. Platform performance stayed stable throughout, and the remote experimentation capability was received positively. Just as importantly, the study became the starting point for a user community around the platform, which is the part that does not happen automatically when you stand up a cluster.
What Did Not Work
Two limitations showed up clearly enough to name.
Downloading activation data took too long. Activations from a 70B model are large, and moving them from the fabric to a researcher’s machine became the slow step in the loop. When the bottleneck is transfer rather than compute, iteration speed suffers in a way that shapes which experiments feel worth running.
Execution was interrupted intermittently. Not constantly, but often enough to be worth reporting rather than smoothing over.
Both are on the development roadmap. We would rather publish them than present a pilot as friction-free.
Why This Matters
The argument for eDIF is not that it is novel infrastructure — NDIF exists, and compatibility with it is deliberate. The argument is about who gets to do this work.
If interpretability research on frontier-scale models is only possible for groups with their own large GPU clusters, then the set of people scrutinising these systems stays small and concentrated. Shared remote infrastructure changes the economics: a researcher with an ordinary laptop and a good question can run activation patching on a 70B model.
For a field whose entire purpose is making AI systems legible and accountable, keeping the tools of scrutiny narrowly held is a poor arrangement. This is a step toward widespread accessibility of LLM interpretability infrastructure in Europe, and groundwork for broader deployment, expanded tooling and sustained community collaboration.
Paper
Heithoff, I., Guggenberger, M., Kalogiannis, S., Mayer, S., Maag, F., Schacht, S., Lanquillon, C. (2025). eDIF: A European Deep Inference Fabric for Remote Interpretability of LLM. arXiv:2508.10553.
Feasibility study of an NDIF-compatible GPU fabric at Ansbach University of Applied Sciences, evaluated with 16 researchers running activation patching, causal tracing and representation analysis on models from GPT-2 to DeepSeek-R1-70B.