Ask a business question in plain English. Get a chart you can check, or an honest “I can’t answer that”.

One real run · recorded 2026-10-27 · staging

One question, from words to a published answer

Everything below is taken from the recorded run, not retyped. The same run plays in the replay.

1. 01

   Understand

   The model reads the question and returns a typed intent, not SQL.

   Question

   Eligible item sales value by category in January 2024

   Answer kind and period, as published

   Eligible item sales value · by category
   Requested 2024-01-01 to 2024-02-01 (UTC, end exclusive)

2. 02

   Compile

   The pipeline writes the one query for that intent. The month is a bound parameter, never pasted into the SQL.

   Metric query

   SELECT COALESCE(p.category, '(unmatched)') AS grp, COUNTIF(oi.status IN ('Complete', 'Shipped')) AS eligible_items, COUNTIF(oi.status IN ('Complete', 'Shipped') AND NOT oi.sale_price IS NULL) AS priced_items, SUM(IF(oi.status IN ('Complete', 'Shipped'), oi.sale_price, NULL)) AS value_sum FROM order_items AS oi LEFT JOIN products AS p ON p.id = oi.product_id WHERE oi.created_at >= @window_start AND oi.created_at < @window_end GROUP BY grp

3. 03

   Guard and run

   Every query passes a SQL allowlist, then runs in BigQuery as a read-only account under a byte cap. This answer took 6 queries.

   • integrity:order_items1 row
   • integrity:products1 row
   • categories3 rows
   • status_profile9 rows
   • analyst_metric4 rows
   • reconciliation_by_group4 rows
4. 04

   Check

   Integrity, status and reconciliation checks decide whether the numbers may be published. Here 8 of 8 passed.

   • sql_guardpassed
   • order_items.id not nullpassed
   • order_items.id uniquepassed
   • products.id not nullpassed
   • products.id uniquepassed
   • status_exclusionpassed
   • reconciliationpassed
   • reconciliationpassed
5. 05

   Publish, with its caveats

   The answer carries a note for every row the rules affected, the data's coverage, and the limits that apply.

   Category	Value (USD)	Note
   Jeans	90.00	items with no status were excluded (1)
   Outerwear & Coats	110.00	—
   Socks	9.50	items without a price were left out (1)
   (unmatched)	12.00	includes items whose product is unknown (1)

   Source

   bigquery · bigquery-fixture · context dev_v1.3 · data as of 2024-02-01T06:00:00Z · covers 2023-12-01 to 2024-02-01 (end exclusive)

Recorded on staging, 2026-10-27 · phase02-candidate-2 · took 10.6 s · USD 0.0059

Why you can trust it

The model proposes. Code and checks decide.

Measured results

A sealed test it failed, then one it passed

Each sealed set was written before the tuning it judged and run once. The bar, set in advance: at least 35 of 36 correct, no unsafe answers, guard rails passing and every run within its caps.

• Custody was one person: an AI model drafted the sealed sets from an agreed plan, and one person reviewed, sealed and ran them. That is weaker independence than a separate custodian.
• A seal proves the files were not changed, not that the expected answers are right.
• Both sealed sets are now used up; the next independent result needs a new one.
• The result measures question interpretation and a checked analytics path on synthetic data, not general SQL writing.

The design

Five decisions, two minutes each

Each page answers one question: what was decided, what it costs, and what was shown.

1. 01The model plans; the pipeline writes the SQLWhy doesn't the AI write the query?
2. 02Checks decide what gets publishedHow does a query that runs still get rejected?
3. 03Spend is reserved before a run startsHow can a chat tool have a hard budget?
4. 04The backend authors the screenWhy can a reader trust what the page shows?
5. 05Evaluation is sealed, and a failure countsHow do you know it's accurate, and how accurate?

What isn't proven

Tested offline, or not at all

The claims on this site are tagged by how they were shown. These are the weakest.

• Tested offlineA failed integrity, status or reconciliation check means no answer, with the failed check named. A deliberately wrong join that runs cleanly is caught.
• Tested offlineA run cut off by a restart is closed as interrupted and keeps its full reservation.
• Tested offlineTimeout and internal-error handling were tested offline but not shown on staging, where they cannot be triggered on demand.
• Not provenBigQuery job cancellation was not re-shown on staging, where queries finish in about 1 s; it was shown in the live guard-rail suite.
• Not provenNo screen-reader pass has been done; keyboard use and labels were checked.
• Not provenThe pilot itself targets desktop Chrome; these public pages work on phones, the pilot does not.
• Not provenThe data is a 17-row synthetic sample; nothing here measures warehouse-scale cost or speed.