Armature

Armature is an analytics and evaluation platform for products accessed through AI agents. Its SDK captures activity at the MCP server or app backend, then reconstructs fragmented tool calls as complete sessions with user intent, agent reasoning, outcomes, and failure points. Teams can understand usage that conventional web analytics cannot observe and convert real behavior into repeatable tests.

Highlights

• Groups sessions into recurring use cases and ranks them by volume and success rate
• Detects loops, dead ends, unsupported requests, and other issues even when APIs return successful responses
• Replays complete sessions with requests, reasoning traces, tool calls, and outcome scores
• Runs end-to-end evals across major models and agent environments using configurable pass criteria
• Redacts personal information and secrets before captured session data is stored

Armature suits product managers and engineers responsible for MCP servers, ChatGPT apps, Claude connectors, and command-line agents. Analytics and evals can be used independently or combined into a feedback loop for finding problems and preventing regressions. A permanent free allowance covers limited monthly usage, while additional credits are usage-based and custom enterprise arrangements are available.