How we measure

In the app, every number comes from your threads' own reports and from what AgentMingo itself did: nothing is estimated, and nothing is called a saving. Home and Insights show them for your team's threads; Sessions, a thread's page and a task's page show the rest.

• Review results sent back. Review bot comments and failed CI checks on a pull request, handed to the thread working on its branch (or the thread running its workflow). Failures main has too are counted apart, as “not yours”, and aren't sent as fixes. Each bot comment counts; a check counts once however many commits it failed on.
• Told before merge. Merge conflicts on a thread's pull request, and main changing files a thread had edited, told to the thread before merge. A main change is information, not a problem found: most are harmless. A ticket that changed, or a teammate on the same ticket, is told too, but isn't counted here.
• Waits told. Times a thread waited on a permission prompt or a question and its person was told.
• Steps handed. Workflow steps given to threads one at a time. Pull request checklist steps aren't counted here or in review results sent back: Sessions shows them on their own.
• Result to resolved. From a thread getting a review result or a conflict to it being resolved: the bot marked it addressed, the review thread was resolved, CI passed on a later commit, or the conflict went away. A median, shown over at least 5; below that, only the count.
• Going again after a notice. From telling someone a thread needs them to the thread going on (an answer, an allowed tool, or the turn stopping). It shows timing, not that the notice caused it. A median, over at least 5.
• Agent work, clock time, side by side. Agent work adds up the time each thread spent working; clock time is how long at least one was. Side by side is the difference, counted for each person on their own: it's your parallel work with Claude Code, not something AgentMingo did.
• Waited on people. Time threads sat on a permission prompt or a question, at most 2 hours a wait (a prompt left open overnight isn't a measure of anything).
• Tokens and cost. Counted from Claude Code's transcripts on each connected computer, from when it connected, and priced at API list prices: a subscription isn't billed per token. What was read from cache is Claude Code's own prompt cache, with or without AgentMingo.

Our public pages show no usage numbers today. Before any number added up across the beta appears there, it has to pass these gates, in a weekly snapshot, never live:

• At least 10 organizations, 20 people and 500 threads that worked, over the last 4 weeks.
• Each count at least 100, from at least 5 organizations; each median over at least 50.
• No one organization making up more than 30% of a number. A number that fails its gate is left out, never shown as a small one.
• Every gate passing in 2 weekly snapshots in a row before anything first shows.

Some organizations are left out of everything we publish and of fair tests, along with any organization that shares a member with one of them, and our own. We never publish a count of what was left out, one team's numbers, or a test's running counts.

We'll say a habit saves tokens or time only once a fair test shows it: a random share of new threads without it, a measure fixed in advance, a sample size set in advance, an interval, and a date. Review results, notices about a thread's work, workflow steps and your team's own rules are never held back to test something.

The first: the git habits. Do threads given the six git habits print less git output, and use fewer tokens in all, per hour of agent work? It hasn't started. When it does, half of new threads in organizations set to Measuring get the habits and half don't, picked at random when each thread starts.

• Measured: git output per hour of agent work first, then all tokens per hour (the habits' own note included).
• Counted: finished threads with at least 10 minutes of agent work, in organizations not left out.
• Size, set now: 300 threads in each group, from at least 15 people (none over 25% of a group) and at least 5 organizations (none over 30% of a group).
• Analysis: each group's total over its total hours, and a 95% interval from resampling people 2000 times.
• If the gates haven't passed after 10 weeks, it ends “inconclusive”, saying which gate didn't.

We'll publish the result either way, here, with its date and numbers: a difference, no difference, or inconclusive. Until then, no running counts.

• Short notes, only when something changed, with the thread's next message or at its start. Each is given once, and again after /clear or compaction.
• The git habits note is about 738 characters, given at a thread's start while the habits are on.
• Its lookups (a thread's ticket, its next step) give up after 1 second, and the thread carries on. We haven't measured the hooks' own time across everyone's computers yet; when we have, it goes here with its date.

• The time someone would have spent copying review results, tickets or conflicts into a thread by hand.
• Whether a thread would have found a conflict or a main change by itself.
• Whether “not yours” saved a thread from chasing main's failure.
• Whether a thread would otherwise have gone looking for its ticket or its review results, and what that would have cost.
• Anything from before a computer connected.