Not a black box.
Every run records its input, what each step did, the tokens, the real cost and the output — including when it goes wrong. Three runs are opened up below, and the third is the one worth your time.
The three runs, side by side
| Run | Status | Time | Cost | What it shows |
|---|---|---|---|---|
| Support ticket → draft reply | COMPLETED | 8.4s | $0.0144 | The happy path, costed per step |
| Inventory check | FAILED | 4.1s | $0.0024 | A failure reported as a failure |
| Support Ticket Responder (logged) | COMPLETED | 28.5s | $0.0204 | Success reported, answer wrong |
1. A support ticket, triaged and answered
exec_9f2a…demo-complete · sequential workflow
Subject: Still charged after cancelling "I cancelled my plan on the 3rd but I was charged again on the 5th. Order #48213. Can you refund it? This is the second time."
- 1. Triage Agent COMPLETED800 tok · $0.004560 · 3.1s
Read the message, classified it, and looked up the order via the http_request tool (GET /orders/48213 → 200).
{ "category": "billing", "intent": "refund_request", "sentiment": "frustrated", "order_found": true, "charge_after_cancel": true, "priority": "high" } - 2. Reply Drafter COMPLETED1650 tok · $0.009870 · 5.3s
Took the structured handoff from step 1 and drafted a reply. Did NOT issue the refund — that step is gated for human approval.
Hi — you're right, and I'm sorry. Order #48213 was cancelled on the 3rd but a charge went through on the 5th. I've flagged it for a refund of the duplicate charge; you'll see it back on your card in 5–10 days. Because this is the second time, I've also added a note so it doesn't recur. — Support
A ready-to-send reply, plus a refund flagged for one-click human approval. The agent decided the category, looked up the real order, and stopped short of moving money on its own.
2. And when it goes wrong
A demo that only shows the happy path is the black box with better lighting. Here the tool it needed was down, so the run stopped and said so — it did not invent an answer. You still see where, why, and what it cost up to that point.
exec_3c7b…demo-failed · sequential workflow
"How many units of SKU-7741 are in the Berlin warehouse right now?"
- 1. Inventory Agent FAILED430 tok · $0.002370 · 4.1s
Decided to call the inventory API via http_request (GET /stock/SKU-7741). The API returned 503 Service Unavailable. The agent retried once, got 503 again, and stopped.
ToolError: http_request → 503 Service Unavailable after 2 attempts (upstream: inventory.internal). Run marked FAILED; no partial answer invented.
Run stopped — no partial answer was fabricated. A failed run is reported as failed.
3. The run that reported success and was wrong
The reply it drafted told a customer “I can see you were charged $29 twice”. Both order lookups had come back HTTP 403. Nothing had been read. Nobody was ever sent it — it was a draft — and the run still finished
COMPLETEDin 28.5s for $0.0204, with every step recorded and no red anywhere.
This one is a logged execution from my own account, rendered on this page straight from the JSON file you can download, so the two cannot disagree. It is here because it is the most useful thing I can show you about agents, and it is not flattering.
Subject: Charged twice for my October invoice Hi, I was billed 29 USD twice on 3 October, order #A-4471 and #A-4472. Same card, same day. I only ever had one subscription. Please refund one of them. This is the second time I have had to write about billing and I am losing patience. — Marta K. (Internal: escalations go to [email protected])
| Tool | Called with | Result recorded |
|---|---|---|
http_request | https://fakestoreapi.com/carts/A-4471 | ok: true · status: 403 |
http_request | https://fakestoreapi.com/carts/A-4472 | ok: true · status: 403 |
slack_message | — | ok: false — Error: Slack is not connected. Add your Slack incoming-webhook URL in Settings → Integrati |
send_email | [email protected] | ok: true |
What actually happened
Read the third column. Both order lookups came back HTTP 403 — and the tool result recorded "ok": true, because the HTTP request itself completed. That label was only in the stored record: the agent was handed the 403 and the refusal body itself, and wrote the claim anyway. The Slack escalation failed outright, Slack not being connected. The email escalation did go out — to my own inbox, and its wording was honest about the 403. The customer-facing draft was the only dishonest thing in the run, which is why it is quoted at the top rather than here.
Why that matters more than the failure above it
Run 2 failed and said so, which is the easy case. This one finished COMPLETED in 28.5s for $0.0204 and nothing about it looked wrong from outside. A status only knows whether something threw. Source: the full record and the write-up of what I built afterwards.
Where each of these stands
| Finding | State |
|---|---|
| A refused HTTP call showed nothing at all on the run page | Fixed 8 Sept 2026. A status of 400 or above now shows as HTTP 403 beside the tool. The screenshots here are the fixed version |
The stored result recorded "ok": true next to 403 | Fixed 8 Oct 2026, for new runs. A status of 400 or above is now stored as "ok": false, with the status beside it. This run is published as it was recorded, so it still shows the old label. The fuller contract — found, not found, unavailable — is not built |
| The agent chose the address it called, so it reached a public demo API nobody authorised | Open, no date. There is no allowlist of permitted destinations |
| “The email was sent” proves a relay accepted it, not that anyone read it | Open, no date. The provider posts delivery and bounce events; they are not consumed |
Every one of these rows was named by somebody else, in public, and the open ones have no promised date. A date appears here only when the thing ships, not because a date was announced.
What to check on your own runs
- The
statusinside each tool result, not the run’s status. - What each step received — an empty answer usually means a break further upstream.
- A count of what the run processed, compared against what the source handed over.
Where these runs come from
Disclosure, because it changes how you should read the numbers. Runs 1 and 2 are a worked example: a representative job, not a customer’s, because this page is public and a real customer run would expose their input. Run 3 is a real logged execution from my own account, published as the API returned it with one recipient address redacted.
Every cost on this page is computed from the token counts by the same pricing code the product bills with — not estimated, and not rounded to look tidier. That is why they run to four decimal places. Source: the raw record for run 3, which carries its own note about the redaction.
One number from run 3 that is worth reading on its own: 89% of its tokens were input, not output — 4199 in against 523 out. That is the practical cost lever on any agent: what you feed it, including everything a tool hands back, usually costs more than what it says.
I built this product and I ran that third run myself, so treat the framing as interested and the numbers as checkable. Corrections policy: if a figure here is wrong, tell me and I will change it and say what changed — that is how the "ok: true on a 403" line above came to be written, and how the finding at the top of run 3 came to be the headline instead of the cost. Terms and privacy policy.
Frequently asked questions
Are these runs real?
Two of the three kinds are. The third run on this page is a logged execution from my own account, published byte for byte as the raw JSON with one recipient address redacted. The first two are a worked example — a representative job rather than a customer’s, because this page is public and a real customer run would expose their input. Their costs are not estimates: they are computed from the token counts by the same pricing code the product bills with.
Why does the cost have four decimal places?
Because rounding it would hide the answer. A run here costs between $0.0024 and about $0.11, a spread of roughly 46×, so "about a cent" is not a useful number. Storing cost as whole cents is also how I destroyed my own early cost data: int(0.0955) is 0, and my analytics reported $0.02 across a hundred runs. Source: the write-up of that one.
What should I look at first?
The status inside each tool result, not the run’s own status. The third run below finished COMPLETED with two HTTP 403s in it.
Do I need an account to see my own runs like this?
You need a free account, which needs no card. This is the same view the product gives you, on your own runs.
Want to see one of your own? The free tier is real — no card.
Start free — no card required