Validation and quality
Read the candidate's checks correctly and separate learning success from product readiness.
Internalization has two distinct outcomes: whether a candidate passed its learning checks, and whether that candidate became active. A job can pass validation without activation. It can also finish execution with a rejection, leaving the existing subject unchanged.
Returned validation
An internalization result may contain:
{
"validation": {
"passed": true,
"passed_count": 7,
"total_count": 8
}
}These illustrative values mean seven of eight checks passed for that candidate. They do not mean 87.5% accuracy across all future requests. The service chooses the validation threshold and check count through its deployed recipe; clients should inspect the response rather than hard-code a denominator.
The subject listing uses a compact form, validation: { passed, total }, where both fields are counts. That differs from the job result, where passed is a boolean and the counts have explicit _count names. See the subject schema.
What the worker checks
The curriculum separates training questions from held-out questions. Source evidence must match the submitted content. The worker validates teacher answers before using them and evaluates candidate answers without including the source in the candidate prompt.
This detects some malformed learning examples and some failures to acquire the supplied facts. It does not establish that the model will preserve every prior behavior, refuse every unsupported claim, or correctly apply a policy under every possible wording.
The public API does not accept custom retention-check arrays or a validation threshold. If your application needs additional tests, run them through inference and retain their results in your own evaluation system.
Interpret each outcome
| Outcome | Meaning | Application action |
|---|---|---|
ready, activated | A saved candidate passed and became active | Test important behavior using the same subject |
ready, not activated | A candidate passed but is not the current route | Inspect manual activation or a concurrency conflict |
rejected | The candidate did not pass the required gate | Review the passage and failed behavior before new learning |
failed | A known execution error stopped the operation | Read the job error and settlement |
reconciliation_required | Outcome or usage is not yet established | Preserve the job and request investigation |
Build an independent evaluation
Write questions before training. Include direct recall, paraphrases, multi-fact applications, exceptions, and facts the source does not contain. Keep the source out of inference messages. Record the exact adapter version on each evaluation job so a later activation cannot confuse the results.
For sequential updates, include questions from earlier knowledge as well as the new passage. Training from parent weights gives continuity, but it does not prove that every earlier fact survives. Test corrections explicitly: ask the old wording, the new wording, and a scenario where the difference matters.
Use the evaluation guide for a concrete test matrix. Validation results are a release input. Your application's acceptance criteria decide whether the resulting behavior is suitable for its users.