Test Suites

Test Suites and Domains

998 scoring dimensions across 254 tests and 11 domains. 22 pre-built evaluation suites.

22 Pre-Built Suites

22 Ready-to-Run Evaluation Suites

Run curated suites or build fully custom test combinations directly inside MARK-17.

Production

7 suites

Agent Readiness

Validates tool calling, multi-step reasoning, and autonomous decision-making for agentic deployments.

~12 min

Enterprise Safety

Tests prompt injection resistance, PII handling, content policy adherence, and adversarial robustness.

~10 min

API Reliability

Measures structured output compliance, error recovery, and consistent behavior under load.

~6 min

Customer-Facing Chat

Evaluates tone, helpfulness, refusal handling, and conversation quality for user-facing applications.

~15 min

Enterprise Quality Gate

Combined safety, reliability, and capability gate for production deployment sign-off.

~18 min

Context MRCR

Multi-round context retention — tests whether models maintain accuracy across long conversations.

~8 min

Multi-Turn Safety

Adversarial safety testing across multi-turn conversations where context can be manipulated.

~10 min

Role-Specific

4 suites

Coding Assistant

Code generation, debugging, refactoring, and explanation across multiple languages.

~12 min

Content & Writing

Creative writing, summarization, tone adaptation, and editorial quality.

~10 min

Reasoning & Analysis

Logic, math, data interpretation, and multi-step analytical reasoning.

~12 min

Multimodal

Image understanding, chart reading, OCR, and visual reasoning capabilities.

~10 min

Comparison

3 suites

Quick Compare

Fast sweep across all domains — ideal for initial model screening.

~4 min

Regression Check

Detects capability regressions between model versions or configuration changes.

~9 min

Full Benchmark

Complete evaluation across all 11 domains with deployment tiering and forensic analysis.

~60 min

App-Specific

8 suites

OpenClaw Readiness

Evaluates tool calling, agentic behavior, safety, and context retention for OpenClaw integration.

~15 min

n8n Workflow Ready

Tests tool calling accuracy, sequential execution, error recovery, and structured output for n8n automation.

~10 min

RAG Pipeline

Retrieval-augmented generation accuracy, citation fidelity, and context grounding.

~12 min

Local Copilot Ready

Benchmarks local models for code completion, inline suggestions, and IDE copilot use cases.

~15 min

Creative & Content Studio

Long-form generation, style consistency, and creative task performance.

~8 min

Document Analyst

Document comprehension, extraction accuracy, and structured summarization.

~10 min

Roleplay & Character

Character consistency, persona adherence, and conversational immersion.

~8 min

MCP Tool Use Ready

Model Context Protocol tool calling compliance, schema adherence, and multi-tool orchestration.

~12 min

11 Domains

Every dimension of model capability.

Each domain tests a different aspect of model performance. Together, they paint the complete picture.

Reasoning

30

Logic, math, probability, constraint satisfaction, meta-reasoning

Coding

35

Algorithms, data structures, concurrency, systems programming, edge cases

Chat

25

Conversation quality, format compliance, creativity, empathy, bias detection

Multimodal

30

Image understanding, OCR, charts, diagrams, medical imagery, satellite

Deployment Risk

28

Safety refusals, prompt injection defense, PII handling, jailbreak resistance

Adversarial Safety

30

Role-play bypass, authority injection, sycophancy, obfuscated attacks, instruction conflicts

Tool Calling

33

Function calling accuracy, parallel/sequential, error recovery, fault tolerance

Agentic

27

Goal decomposition, multi-agent coordination, state management, autonomous troubleshooting

Multi-Turn Adversarial

8

Gradual escalation, persona persistence, language switching across turns

Agentic Email

1

Real-world email inbox management task

Context Retention

7

Needle-in-haystack from 8K to 1M tokens

254 Tests Across 11 domains
Test Catalog

Domain breakdown.

Domain Tests Difficulty Range What It Measures
Reasoning 30 Easy → Edge Case Logic, math, probability, constraint satisfaction, meta-reasoning
Coding 35 Easy → Edge Case Algorithms, data structures, concurrency, systems programming, edge cases
Chat 25 Easy → Edge Case Conversation quality, format compliance, creativity, empathy, bias detection
Multimodal 30 Easy → Edge Case Image understanding, OCR, charts, diagrams, medical imagery, satellite
Deployment Risk 28 Easy → Edge Case Safety refusals, prompt injection defense, PII handling, jailbreak resistance
Adversarial Safety 30 Easy → Edge Case Role-play bypass, authority injection, sycophancy, obfuscated attacks, instruction conflicts
Tool Calling 33 Easy → Edge Case Function calling accuracy, parallel/sequential, error recovery, fault tolerance
Agentic 27 Easy → Edge Case Goal decomposition, multi-agent coordination, state management, autonomous troubleshooting
Multi-Turn Adversarial 8 Medium → Hard Gradual escalation, persona persistence, language switching across turns
Agentic Email 1 Specialized Real-world email inbox management task
Context Retention 7 Medium → Hard Needle-in-haystack from 8K to 1M tokens
Total 254 254 tests across 11 domains

Tests range from Easy to Edge Case difficulty, designed to expose hidden model weaknesses across every domain.

Suite System

Forensic-grade AI evaluation.

Hard-Fail Rules

Critical tests that must pass. One failure flags the entire suite — regardless of overall score.

Deterministic Evaluator

Same input, same score, every time. No variance, no randomness in evaluation.

Requirement Proof Logging

Every pass/fail decision is logged with the evidence that produced it. Full traceability.

Confidence Scoring

Run count validation ensures your results are statistically significant, not a fluke.

Deployment Tiering

Automatically classify models into deployment tiers based on safety, reliability, and capability.

Custom Tests & Suites

Build custom evaluations by mixing tests across domains and save them as reusable suites.

Custom Test Builder

Create new evaluations on the fly, combine tests across domains, and save them as reusable test suites.

Evaluate AI models against the work that matters to you.

All 254 tests across 11 domains, run on models you choose — local, cloud, or both.

How MARK-17 Works Read the Documentation