The gateway that weaves your test bench together
BenchWeave connects instruments and devices under test through reusable plugins and typed profiles — coordinating bounded test procedures with explicit ownership, measurement evidence, and verified safe endings.
Wouldn't it be cool to have AI help test hardware?
Where it came from
BenchWeave started as a conversation between two friends. The question stuck, and it sharpened into a harder one: what would it take for an AI to drive a real test bench safely — instruments, a device under test, and the guard rails written down as contracts rather than carried in someone's head?
The plan is deliberately small: get a working proof of concept, keep all of it open source, and open it to the community. Beyond that, the project is a living thought experiment — the architecture, the standards and the code are the working-out, published as they happen, to see where the idea leads. Everything on this site says what has been built and what hasn't.
Talk bench with us
Questions, plugin builds, standards arguments and works in progress — BenchWeave has a Discord, and the invite is permanent.
What surprised us
A device plugin doesn't need BenchWeave
A plugin is a descriptor plus an adapter against structural interfaces, with no runtime dependency on the SDK or the gateway. The DPS-150 plugin builds and tests unchanged outside this checkout and has driven the real supply. So a standalone MCP server that implements the host side can control a device in its own right — the plugin is a standard way of saying what a device is and how it talks. What that path doesn't give you is the bench safety model: ownership, protection deadlines and verified endings live in the gateway.
Greenfield hardware can start from the firmware
For your own board, the same workflow starts one step earlier: Build my own device firmware gives you a shared session instruction and stage prompts to get from board and intended behaviour to firmware, descriptor and adapter that agree, with the firmware kept in the plugin's own firmware/ beside its toolchain and compatibility record. The prompts authorise software work only; flashing and energising stay with you.
Build your instrument and workbench with AI
Five-step paths for device plugins and custom firmware — development, testing, review, and release prompts, checked against the contracts by the AI device reviewer before anything ships.
The plan
Four commitments from the architecture, not four feature bullets. The first PoC is simulator-first; the planned hardware MVP targets the FNIRSI DPS-150, with ESP32 as the provisional controller family.
Connect
Instruments and devices under test through reusable plugins and typed device profiles.
Share
Profiles and implementations through a central registry, with local admission and offline execution.
Coordinate
Bounded test procedures with explicit ownership, measurement evidence, and verified safe endings.
Expose
A consistent REST and MCP interface for applications and coding agents.
Where it stands
Working now. The simulator-first PoC is through its acceptance gate: durable run and lease state with kill-recovery, procedure admission and protection, 20 REST routes and 17 MCP tools on one core-operations seam, and an operator CLI. The SDK scaffolds an independent plugin project and release CI builds one outside the checkout and drives it through the gateway on mock transport. The DPS-150 plugin — hand-written, zero runtime dependencies — has driven the real supply; it is read-only and claims no PSU profile yet.
Next. Gateway and devices talking through the registry: the gateway side of the registry contract is done (signed catalogue, admission with a content-addressed cache, activation, an unsigned dev loop) but the registry service and a device-install command do not exist, and the OTDP bridge carries identify and scalar read/write only. Then a basic operator UI on the component library that exists today. Hardware qualification and unattended bench operation remain separate, commissioned work.
Standards
The versioned contracts everything else is built against — treated as agreements, not informal notes. Each is independently versioned, digest-frozen, and validated in CI; the SDK vendors the same bytes.
OTDP
v0.1.0Open test-device protocol — how the gateway talks to instrument and DUT plugins: device descriptors, measurement model, device classes, extension contract.
Read the specification →Registry
v0.1.0Plugin and device-profile registry — admission, sharing, release manifests and offline execution.
Read the specification →Execution
v0.1.0Bounded test-procedure execution — bench, commissioning, procedure and run-record schemas; ownership, evidence and safe endings.
Read the contract →Interface
v0.1.0The REST and MCP surface exposed to applications and coding agents — OpenAPI document, MCP tool catalogue, operation catalog and review scenarios.
Read the contract →Plugin UI
v0.1.0Plugin presentation contract — bindings, presets and envelopes for the workbench UI.
Validation report →Plugin UI preview
v0.1.1Preview wire documents and fixtures for simulation-only plugin UI rendering.
Schemas on GitHub →Governance, the standards manifest and the digest-locked corpus manifest live at standards/GOVERNANCE; the rendered standards index lists every prose companion and validation report.
Documentation
Built with Great Docs from the repository's own guides, architecture record and standards. Unversioned until the first tagged release; every page links back here.
Operator guide
Install, run and care for a gateway — every command verified against the shipped CLI.
Device developer guide
Device creation, gateway hosting and shared packages, with the AI task template and human review checklist.
Develop your device with AI
Five-step paths for device plugins and custom firmware, with reusable prompts.
AI device reviewer
Role and checklist for assessing candidate integrations against the contracts.
Development and CI
uv setup, local checks and the GitHub workflows.
Architecture v1.5
The admitted architecture baseline.
Decisions
The decision record behind the baseline.
Compatibility
Compatibility record and matrix — what is reachable today, and what is disclosed as out of scope.
PoC/MVP PRD
Product requirements for the simulator-first PoC and hardware MVP.
Delivery plan
Work packages and sequencing.
CLI reference
The benchweave command tree — setup, serve, status, verify, demo, report, backup, restore, evidence — generated from --help.
Standards
Every prose companion and validation report, rendered beside its machine corpus.
Changelog
Generated by git-cliff from conventional commits.
FNIRSI DPS-150 protocol
Protocol evidence for the planned hardware MVP target.
DPS-150 compatibility record
Tested models, firmware and limitations.
ESP32 selection
Why ESP32 is the provisional controller family.
Agent skill
The benchweave skill under .well-known/, for coding agents working on the gateway.
llms.txt
Plain-text index of the docs; llms-full.txt carries every page.
SDK
The plugin developer SDK lives in its own repository with its own site, CI and release cycle. It is mounted here at packages/sdk as a git submodule.
benchweave-sdk
Build an external device plugin without importing gateway internals. Python 3.13+ · SDK 0.0.2 · OTDP 0.1.0 baseline. Offline authoring and conformance tools: scaffold, validate, mock, preview.
pip install benchweave-sdk
# scaffold an independent plugin project
benchweave-sdk new plugins/acme/model100 --package benchweave_acme_model100
Built with AI
BenchWeave is built by AI coding agents under human direction. For anyone curious how that works in practice, this page is the honest record: which model did which job, and the open tooling around them. The stack is deliberately mixed — no single vendor wrote this project.
Architecture brainstorming
GPT AstraThe origin conversations — turning “wouldn't it be cool to have AI help test hardware?” into a first architecture worth committing to.
The code
GLM-5.3The working model behind the gateway, the SDK, the plugins and the tests. Spec-driven and test-first: every slice landed as a failing test made green.
Z.ai →This website
Anthropic FableDesigned and built on free credits — which have since expired.
Anthropic →MuninnDB
A self-hosted long-term memory server. This project keeps its own local vault, so hard-won context survives the session that earned it.
Gortex
A local code-intelligence daemon. The codebase is graph-indexed, and reads and edits route through it — nothing is touched blind.
Mixed models, one rule: nothing lands on trust. Every change, AI-written or not, passes review and the CI gates — pytest, ruff, mypy strict — before it reaches main.