Reading Markdown Reports

Evidence: Report shape and exit behavior were reproduced on 2026-07-14 with cargo-teaql 2.0.8 and TeaQL Maven plugin 1.1.0 against the current TeaQL evaluation service. Both clients returned the same evaluator report body for the same model; their client-specific framing differs.

TeaQL reports are ordered diagnostic documents, not decorative console output. Read the first model-level finding before reacting to a later generator, compiler, or stack-trace line.

Report Anatomy

The current evaluator report body contains:

1. the selected model input and evaluation endpoint;
2. KSML Evaluation Report;
3. a severity summary;
4. an error-pattern summary when errors exist;
5. detailed errors, warnings, and suggestions;
6. specialized findings, such as privacy findings, when applicable;
7. solids: checks that passed.

Cargo prints the model input and selected endpoint before the report. Maven prints its effective configuration summary and endpoint before the report, then wraps the result in Maven's build summary. The plugin masks the API key in the observed output, but retained reports should still be reviewed for local paths, account metadata, endpoints, and other environment details before they are shared.

The summary uses four categories:

Category	Meaning	Immediate action
Errors	Blocking model violations.	Fix before generation.
Warnings	Risk or quality findings that do not necessarily block evaluation.	Review explicitly; do not hide them behind a successful exit code.
Suggestions	Recommended improvements.	Decide whether to accept or document why they do not apply.
Solids	Rules the submitted model passed.	Retain as useful evidence, not as proof that every unlisted rule passed.

Read Errors by Pattern, Then by Target

When errors repeat, the report adds an Error Pattern Summary with:

• count;
• pattern and stable rule identifier where available;
• explanation;
• recommended action.

Fix the largest repeated source pattern first. Then use each detailed row's Rule ID, Target, Error Message, and Action to find the smallest source change. A blank target means the rule applies to the model or upload as a whole, not that the diagnostic can be ignored.

A current deliberately invalid probe produced this structure:

Errors: 1
Warnings: 2
Suggestions: 0
Solids: 5

Rule ID: KSML-BUSINESS-003
Target: <model-level>
Error: the model has no business object other than its root candidate
Action: consult the modeling error guidance, apply the smallest fix, rerun

Both clients exited with status 1. After adding the missing business object, the same model produced zero errors, three warnings, one privacy suggestion, and six solids, then exited with status 0 in both clients.

Exit Status Is Necessary, Not Sufficient

Result	Cargo	Maven plugin	Interpretation
One or more errors	Exit 1	Exit 1, BUILD FAILURE, and KSML model evaluation failed	Evaluation failed; do not generate.
No errors, with warnings/suggestions	Exit 0	Exit 0 and BUILD SUCCESS with default teaql.eval.failOnWarning=false	Evaluation completed, but the findings still require review.

Automation should fail on a nonzero exit status and preserve the full report. Whether warnings also block a release is a project or client configuration decision; do not infer that policy from exit status alone. Maven plugin 1.1.0 exposes teaql.eval.failOnWarning when a stricter build is required.

Recovery Loop

1. Preserve the complete command, report, and exit status.
2. Record client, generator, model revision, and selected service/target.
3. Start with the first blocking rule or largest repeated error pattern.
4. Locate the reported target in the source KSML or client configuration.
5. Apply the smallest source fix; do not patch generated output.
6. Evaluate again.
7. Generate only after errors are cleared and remaining findings are reviewed.
8. Build/test the generated result and retain that evidence separately.

For example, an empty _audit_mask_fields value in the current probe did not count as masking. The evaluator returned a privacy suggestion naming the sensitive field and recommending a concrete value. The credible fix is to put that field in _audit_mask_fields or remove the ineffective attribute while the model decision is reviewed; it is not to suppress the report.

Distinguish Evaluation from Later Failures

Failing stage	Primary evidence	Typical fix surface
Evaluation	Rule ID, target, message, and action in the Markdown report.	KSML or evaluator configuration.
Generation	Target name, service version, Markdown error, and output path.	Source model, client options, or generator template.
Compilation	First compiler error plus generated signature/dependencies.	Version alignment, source model, generator defect, or handwritten caller.
Runtime	First exception, configuration, provider/database version, query/mutation intent.	Application/runtime/provider configuration or handwritten extension.

Do not report a compiler consequence as the model root cause when evaluation already identified an earlier blocking rule.

Generation Failure Shapes

Generation failures do not all use the same report shape. Current probes found three distinct layers:

Layer	Observed output	Exit	First fix surface
Local client input validation	Maven reports TeaQL generation failed: input does not exist: <resolved-path> without calling the generation endpoint.	1	Correct the input path. With mvn -f, relative input is resolved from the Maven project base directory, not necessarily the shell directory.
Model evaluation during generation	Cargo generation returns the full KSML Evaluation Report, including summary, rule IDs, targets, and actions.	1	Fix the source KSML, evaluate explicitly, then generate again.
Server target/template failure	Maven returns a Markdown Bad Request (400) section, a fenced server message, server-build information, and a final Mojo failure.	1	Preserve the report and dynamic target; report the server/template defect or select a verified target.

The live java-app-console probe returned:

Bad Request (400)
No such group file: generator/java-app-console/application.properties.stg
TeaQL generation failed: Generation failed due to server validation errors.

The response also included server build information. Retain it when escalating because it identifies the server state that produced the failure, but do not reinterpret commit timestamps as artifact release dates or support promises.

An advertised target proves discovery only. A target becomes usable evidence after generation and its generated build/test path both succeed.

Escalation Record

For an unknown or reproducible generator defect, attach:

• complete Markdown report and exit status;
• exact command and working directory;
• smallest non-sensitive model that reproduces it;
• client, generator, runtime, toolchain, framework, and provider versions;
• dynamic target/service name;
• first compiler or runtime error, if evaluation passed;
• relevant generated signatures or manifest entries;
• result after a clean regeneration.

Use the Diagnostics Catalog for known patterns, the Troubleshooting Index for the next failing stage, and Debug Generated Code Safely when a generated file contains the visible failure.