Tool compatibility needs two layers of evidence: structural validation proves that arguments and results match declared schemas, while semantic cases prove that both sides still assign the same meaning to accepted values.
Pin the declared contract and its version
Capture the tool name, description, input schema, output schema, server version, client version, and any capability flags used during the accepted run. The Model Context Protocol tool specification defines inputSchema as JSON Schema and permits an outputSchema for structured results. That declaration gives the test harness a concrete boundary, but only if the saved contract is the contract the client actually received at runtime.
Build fixtures for the smallest valid request, a representative full request, each required field missing, wrong primitive types, unknown properties, boundary values, and deprecated fields. Validate responses against the declared output schema and preserve the raw protocol error when validation fails. A client should not silently coerce an incompatible result into plausible text, because that hides the exact contract break the schema was meant to expose.
Add semantic cases beyond structural validation
JSON Schema checks structure, constraints, and types. It cannot prove that a customer identifier belongs to the authenticated account, that a date range is permitted, or that preview mode avoids a side effect. Add executable semantic assertions for those rules. Each case should name the precondition, request, expected tool selection, expected authorization result, expected effect or non-effect, and response fields the next workflow step consumes.
Compatibility also includes error behavior. Test a denied request, unavailable dependency, exhausted quota, stale version, and malformed downstream result. The client needs a stable distinction between correctable input, denied permission, transient failure, and permanent incompatibility so the runtime can choose correction, review, retry, or hold without asking a model to infer the category from prose.
Run old and new pairs before release
Create a small matrix covering the current client with the current server, the proposed client with the current server, the current client with the proposed server, and the proposed pair. Run the same fixtures against all four when both components change. A schema change that passes only with the new pair can still strand a deployed client, and an unchanged schema can still conceal a semantic behavior change.
Agent Runtime Reliability prepares these tests through Reality Contact, LLC. The buyer identifies supported versions, deprecation policy, permission owners, and production release order. The compatibility report states which pairs and cases passed at a named revision; it does not certify every possible argument, client, server, or downstream integration.
Where the service stops
Reality Contact, LLC implements and verifies bounded runtime changes but does not certify security or availability, approve credentials, authorize consequential actions, or operate the service indefinitely. The buyer approves boundaries and scenarios, controls every production credential and release, and moves workflows only after reviewing the evidence. This is software implementation and technical verification; it does not replace the buyer's security, privacy, legal, compliance, or production-readiness review. We do not promise continuous availability, error-free execution, complete incident causation, safe behavior outside the accepted scenarios, or recovery from every possible failure.
Sources: Model Context Protocol tool specification; JSON Schema documentation for validation contracts.