diagnose AI Workflow Diagnostics
中文
01 · Skill / Workflow diagnostics

It runs, so you ship it?
This Skill catches the flaws
that only surface after launch.

An inspection for AI workflows you built yourself. Hand over your prompt text, tool inventory, or agent config, and you get a 1 to 5 score on five dimensions, an overall score out of 25, and the fixes ranked by severity.

Prompts, Skills, multi-agent pipelines: once they run end to end, most people call them done. The structural problems that break them in production were already there the moment the first run succeeded.

diagnose is a diagnostic Skill. It scores Prompt Quality, Context Efficiency, Tool Health, Architecture Fitness, and Safety & Reliability from 1 to 5 each, then returns a scorecard out of 25 with the critical findings and the fixes ordered by severity.

What comes back is not a generic checklist. A score has to land on a specific layer, and every finding has to cite its evidence.

106 lines / 0 dependencies Official repo 39,089 ★ 418 Skills in the same repo License MIT Install needs GitHub CLI 2.90.0+
Workflow Diagnostic CASE 01 / REAL RUN
Prompt QualitySTRUCTURE, OUTPUT SCHEMA, EDGE CASES
4/5
Context EfficiencyBUDGET, ORDERING, MEMORY
3/5
Tool HealthCOUNT, CONTRACTS, ERROR PATHS
2/5
Architecture FitnessTOPOLOGY, BOUNDARIES, COST
4/5
Safety & ReliabilityVALIDATION, LIMITS, RECOVERY
4/5
Overall 17/25
Critical Finding
The flow states that six reusable scripts ship with the package. Searching every one of the six filenames returns zero hits.
02 · Before / After

Three shifts: how you judge, what comes back, what you can audit

In each pair, the left side is the state before you use it and the right side is what you hold after a diagnosis.

01

How you judge it

Before

Once an AI workflow is built, there is usually one signal available for judging it: it ran without errors. Whether the output format is defined, whether it recovers from failure, whether cost has a ceiling, you reconstruct all of that from memory afterward.

After

Five dimensions, each scored 1 to 5, each one stating where points were lost. Anything under 3 is flagged, so you can see which layer the problem sits in.

From “looks fine to me” to “I know which dimension scored 2, and why.”

Real scorecard · CASE 01 · Single-file prompt Skill17 / 25
Prompt QualityRole and deliverable are explicit; Steps 0-4 are cleanly partitioned; the report keeps a fixed 5-item structure4
Context EfficiencyEverything loads at once; the opening 40 lines are machine-specific setup sitting in the most expensive attention slot3
Tool HealthThe reusable scripts the document declares do not exist; paths are hard-coded to one machine2
ArchitectureSingle responsibility, single agent; retrying a hung command is explicitly forbidden4
Safety & ReliabilityThe 7 false-positive traps are false-alarm suppression, each with a cause and the correct test4
Scores and per-dimension reasons come from this session's runs and can be checked line by line in research/diagnostic-runs.md. Dimensions below 3 are marked in amber.
02

From a paragraph to a report

Before

The problems are scattered across prompts, scripts, config, and memory. Reviewing the workflow means rereading every file, and with no fixed format to compare against, you finish with an impression rather than a list.

After

A report out of 25: five dimension bars with an overall score, three critical findings ranked by severity, and the fix for each one.

From “I think this part might be a problem” to “these three are what to fix now.”

Diagnostic report · Key findings and fixes · CASE 01Ranked by severity
HIGH
Bundled dependencies are missing, and it reproduces
Step 1 states that six scripts live in audit/ and can simply be copied. A full-disk search for those six filenames returns zero hits, and the directory does not exist. Following the document stops right there.
Either ship the six scripts, or rewrite the step so it no longer claims they are bundled.
MED
Attention order runs against value
The most valuable part of the document, the seven false-positive traps, sits at lines 87 to 109. What leads instead is 40 lines of environment setup that only holds on one machine.
Move the machine-specific setup later or collapse it, and let the general steps lead.
MED
No cost ceiling
The Step 2 measurement matrix is viewports × throttle profiles × routes, so browser operations can run into the dozens. The document sets no ceiling.
State a default ceiling, and make raising it an explicit action.
Findings are taken from this session's runs in research/diagnostic-runs.md. The recommended fixes follow from each scoring dimension; they are not verified conclusions.
03

What size you can audit

Before

A prompt you can still eyeball from experience. Once the scripts multiply and the layers deepen, there is no general way to check it, and the more files there are, the more you miss.

After

One scoring scale covers three orders of magnitude: a single-file prompt, a multi-stage gated pipeline, and a system with scripts and external side effects.

From “only prompts are auditable” to “1 file or 52 files, same scale.”

Three-case comparison · one ruler, three scalesReal runs
DimensionCASE 01CASE 02CASE 03
SubjectSingle-file prompt SkillMulti-stage gated pipelineExecution system with real side effects
Files11052
Scripts0037
Size (bytes)7,53539,9921,921,632
Prompt Quality454
Context Efficiency334
Tool Health222
Architecture443
Safety & Reliability453
Overall17 / 2519 / 2516 / 25
Most damaging findingA declared script does not existA tool capability the flow requires was never declaredUndeclared webhook and command-execution channels
Scroll the table sideways to see all three cases
All three score 2 on Tool Health, for three different reasons: a missing dependency, a blank tool contract, and tool overload. Every number can be checked in research/diagnostic-runs.md.
03 · Pain Point / Use Case

What breaks an AI workflow is rarely that it fails to run

All three situations below come from real workflows. None of them is hypothetical.

Situation 01 / Author of a single-file Skill

The dependencies are in the document, not in the package

A prompt Skill at 123 lines / 7,535 bytes. The logic is clear, the steps are complete, every path and version number is spelled out. It reads like a finished product you can use as is.

But it points to six scripts described as already included and ready to copy, and those six files were never in the package. The author had them on their own machine. The reader does not.

Nothing here throws an error. It leaves the reader stuck at step two, assuming the problem is their own environment.
Situation 02 / Owner of a multi-stage pipeline

The flow needs the network but never declares it

A gated copy pipeline whose output contract, banned-word list, and self-check are written more carefully than most. Its anti-fabrication rules run four levels deep on evidence.

Yet it puts searching official pages to verify links into the required flow, whenever the network happens to be available, while the documentation never declares a single tool capability.

On a host with network access it searches. On a host without it, it stalls. The same workflow changes behavior with the environment, and the user is never told.
Situation 03 / Owner of a system that moves real money

The code can do more than the documentation says

An execution system across five exchanges with 37 scripts. Credentials live only in environment variables, every order is confirmed by the user first, and no pre-trade check is missing.

But one script marked optional can POST data to any URL, and can execute an external command named by an environment variable. Neither channel appears in its capability declaration.

The capability surface a reader sees is smaller than the one the code actually has, and a single functional test never reaches that gap.

None of these three is a bug

They share one property: they run clean and they pass the tests. They only surface in a particular environment, on a particular call path. So no check for “does it work” will ever stop them.

And the tools people reach for by default do not operate on this layer. Prompt-writing articles hand you a generic checklist that never looks at your workflow. Code checkers understand syntax but cannot tell whether a prompt defines its output format. Rereading the files yourself mostly surfaces what you meant when you wrote them, not the gaps in the structure.

What you get is delayed exposure. The problem does not appear while you are building. It appears when other people start using it, after launch, once it starts moving real money.

04 · Transition

What is missing is not more experience. It is one review run against fixed dimensions. Structural problems only come out under direct questioning: does this section define its output? Do the declared tools match actual behavior? What happens after a failure? Is there a ceiling on cost?

diagnose is that questioning, made repeatable. It splits the review into five fixed dimensions and asks the same questions every time, so two diagnoses land on the same scale and a revision shows up as better or worse.

05 · Solution

You bring the workflow. It scores each dimension and shows its reasoning.

Three steps from input to output, and nothing in between needs configuring.

Input

Any one of these four is enough to start. The more specific you are, the sharper the findings.

  • Workflow description
  • Prompt text
  • Tool inventory
  • Agent config
Review / five dimensions
D1

Prompt Quality

1 to 5

Whether role, context, instructions, and output region each sit where they should, whether the output format is written down or left implied, whether edge cases are covered, and whether obvious anti-patterns are present.

D2

Context Efficiency

1 to 5

Whether the budget was planned, whether key information sits at the start or the end, whether state and memory are managed explicitly or left to defaults, and whether length matches the task.

D3

Tool Health

1 to 5

Whether the tool count sits in a workable range, whether descriptions are specific, whether error handling exists, whether input, output, and error are all defined, and whether repeat calls are safe. It also separates the tools a project configures from the tools an agent brings with it, and only counts the first group.

D4

Architecture Fitness

1 to 5

Whether a single agent is enough, whether multiple agents are genuinely needed, whether boundaries overlap, whether handoffs have a protocol, whether decisions leave a trace, and whether cost has a budget.

D5

Safety & Reliability

1 to 5

Input validation, output filtering, cost ceilings, failure recovery, evaluation strategy. Risk is judged against how far the data actually travels, not against a blanket maximum.

Output
Always these three
  1. A scorecard out of 25, one bar per dimension plus the overall score
  2. Three critical findings ordered by severity
  3. The fix for each finding
06 · Real Cases / Outputs

One scale, three orders of magnitude

All three cases are real workflows scored against its five dimensions in this session. Every score and every finding can be checked line by line.

CASE 01

Single-file prompt Skill

1 file · 123 lines · 7,535 bytes
Input

A single-file prompt Skill that audits websites with a real headless browser. Its own rule: every conclusion has to trace back to a number, a screenshot, or a raw response.

Output

17 / 25. The most damaging finding is a missing packaged dependency: the flow states that six reusable scripts live in audit/ and can simply be copied, while a full search for all six filenames returns 0 hits and the directory does not exist. The other two environment claims in the same document, the Chromium binary path and the playwright-core install location, both checked out as true.

Why it matters

True and false claims sit in the same stretch of text. Trusting the whole block misses the break, and doubting the whole block wrongly discards two correct claims. Only checking line by line works, and that is exactly what a manual reread skips.

Diagnostic · 17/25
Prompt Quality4
Context Efficiency3
Tool Health2
Architecture4
Safety & Reliability4
Overall17/25
▼ MISSING DEPENDENCY
CASE 02

Multi-stage gated pipeline

10 files · 719 lines · 39,992 bytes
Input

A copy-production pipeline with a fixed nine-section output contract, three human gates, four evidence tiers, and 11 final self-checks. It explicitly bans fabricated screenshots, data, and links.

Output

19 / 25, the highest of the three. The typical gap is an empty tool contract: the flow requires searching official pages to verify links whenever the network is available, while the documentation declares no tool capability at all, so the same workflow behaves differently on different hosts. Each invocation also carries a fixed 35,363-byte preload that cannot be trimmed on demand.

Why it matters

Its anti-fabrication design is the strongest of the three, and its score is in fact the highest. Scoring is not fault-finding. It also registers what was done right.

Diagnostic · 19/25
Prompt Quality5
Context Efficiency3
Tool Health2
Architecture4
Safety & Reliability5
Overall19/25
▼ EMPTY TOOL CONTRACT
CASE 03

Execution system with real side effects

52 files · 37 scripts · 1.92 MB
Input

An algorithmic execution system across five exchanges that can create, modify, and cancel real orders. Credentials stay in environment variables, and every write requires user confirmation.

Output

16 / 25, the lowest of the three. The core finding is a declared surface smaller than the actual behavior: one script marked optional can POST data to any URL and execute an external command named by an environment variable, and neither channel appears in the capability declaration. On top of that, all 37 scripts sit flat in one directory, which makes choosing between them costly.

Why it matters

The more complex the workflow, the more likely it carries an undeclared capability surface, and a single functional test never reaches that gap. It will not turn any test red.

Diagnostic · 16/25
Prompt Quality4
Context Efficiency4
Tool Health2
Architecture3
Safety & Reliability3
Overall16/25
▼ UNDECLARED SURFACE
Before publishing

The scores, file counts, line counts, and search results in all three cases come from this session's runs against the five dimensions, and can be verified one by one in diagnostic-runs.md. The scorecard, the key findings, and the comparison table in section 02 are rendered from that same record: they visualise the data, they are not screenshots of an interface. If real screenshots become available, they can replace them.

07 · Core Advantages

Four differences that reach the user

01

Scores land on a layer, not on a feeling

Because the five dimensions are scored separately, a 2 points to a specific layer. Tool Health scored 2 in all three cases for three different reasons: a missing dependency, an empty tool contract, and tool overload.

A blanket “too many tools” would misjudge two of them. Layered scoring does not.

02

Every finding has to cite its evidence

Its rules require concrete findings for every dimension, not just a number. The zero hits for six scripts, the fixed 35,363-byte preload, and the undeclared POST channel in the three cases all trace back to something you can check.

Whoever holds the report can verify it themselves, without first trusting the reviewer.

03

One scale, so you can retest

The dimensions, the score range, and the meaning of the five bands are fixed. Score again after a change and both numbers land on the same scale, so you can see whether it improved or regressed instead of re-deciding from scratch.

That is what a generic checklist cannot give you, because a checklist has no score.

04

It only bills you for tools you control

Tool Health carries a specific rule: separate the tools a project configures from the tools an agent ships with or installs globally, and deduct only for the first group. Otherwise a report charges your host environment’s tool count to you, and it reads as a long list of problems you cannot fix.

Every finding left in the report is one you can act on.

08 · Why It Matters

What diagnose changes is not the result of one review. It is where in the timeline the discovery happens.

Put a fixed check between “it runs” and “you ship it”, and problems move from the moment someone hits them to before it goes out. Because the dimensions are fixed, the check is reusable: score again after editing the prompt, after adding a script, after switching hosts, and every score lands on the same scale.

One diagnosis leaves you with more than three fixes. It also shows which dimension of your own workflow is thinnest. Once you know that, the next workflow you build starts with that layer reinforced.

09 · CTA

Diagnose

AI workflow diagnostic Skill · five dimensions · a report out of 25

gh skills install github/awesome-copilot diagnose Requires GitHub CLI v2.90.0+

Official source: github/awesome-copilot · skills/diagnose

You provide

A workflow description, prompt text, a tool inventory, or an agent config. Any one of the four is enough to start.

You get

A 1 to 5 score on five dimensions, a scorecard out of 25, and the critical findings with fixes ordered by severity.