The plugin framework, described honestly.
MARK-17 can be extended with additional evaluation domains, and the machinery for doing that safely is built and shipping. What does not exist yet is a marketplace or a community around it, so this page tells you where things actually stand.
Where this actually stands today
The framework works. Beyond the 11 built-in domains, there is sample and example material rather than a catalogue of production plugins, and there is no marketplace or community directory. We would rather say that plainly than hint at something we have not built.
If extending MARK-17 for your own industry or compliance requirements matters to a purchasing decision, talk to us before you buy. How plugin installation is packaged against a license has not been announced.
Extending the built-in domains.
MARK-17 ships with 11 built-in test domains covering reasoning, coding, safety, chat, multimodal, agentic workflows, and more. The plugin system exists so a domain can be added when those do not cover what you need to evaluate.
What a plugin can add:
- ✓ New evaluation domains with test cases specific to your industry or application
- ✓ Domain-specific scoring criteria and pass/fail thresholds
- ✓ Full integration with the existing benchmark wizard, queue, and reporting
- ✓ A custom suite you built inside MARK-17, exported as a distributable package
Nothing runs because it claims to be a plugin.
A plugin is code that produces evaluation results you may end up defending to someone else. The validation around it is the part we care most about.
Package validation
Plugin packages are validated before they are accepted, with a strict whitelist of what the archive may contain.
Signature verification
Release builds verify an Ed25519 signature on the package.
No silent overwrites
A plugin cannot quietly replace a built-in domain or a plugin you already have.
No version downgrades
Installing an older version over a newer one is blocked, and version history is kept.
Enable and disable
Installed domains can be turned on and off without removing them.
It shows up in the audit trail
Plugin activity is recorded like everything else MARK-17 does.
Need a domain that does not exist yet?
Tell us what you need to evaluate and why the built-in domains do not cover it. That is genuinely useful to us, and it is the fastest way to find out whether it is something you could build or something we should.
New here? Start with MARK-17 — the plugin framework only matters once you are using it.