Review Function Prediction

Use this skill to review computational function predictions against allowed evidence and write schema-compatible PredictionReview YAML.

the predictions are in YAML alongside the main curation review YAMLs:

genes/<TAXON>/<GENE>/<GENE>-<TOOL>-predictions-review.yaml

id: <UniProt accession>
gene_symbol: <gene symbol>
locus_tag: <locus tag if available>
taxon:
  id: NCBITaxon:<taxon id>
  label: <organism>
status: COMPLETE
description: >-
  Brief summary of the prediction review outcome, based only on the allowed
  evidence for this review mode.
source_documents:
  - <allowed file path used>
predictions:
  - source_method: <do not change>
    source_version: <do not change>
    source_reference_id: <do not change if present>
    predicted_term:
      id: <do not change>
      label: <do not change>
    predicted_term_type: <EC | GO_MF | GO_BP | GO_CC>
    review:
      assessment: <COR | CNN | LSP | UNC | PLI | NPI | REP>
      confidence_score: <0 | 1 | 2>
      error_type: <omit unless applicable>
      summary: >-
        Explain the decision in 3-8 sentences. State what allowed evidence was
        used, whether the prediction is already present in curated annotations,
        whether it is more or less specific than existing annotations, and why
        the chosen category is justified. If using UNC, explicitly say what
        evidence is missing and avoid speculation.

Selected records with no GO/EC predictions

When a selected benchmark target has no emitted GO/EC predictions in the frozen source, still write its PredictionReview sidecar with an explicit predictions: []. Use description for the summary evaluation: identify the method and snapshot, distinguish absent GO/EC output from any emitted name, location, or function text, and assess whether well-supported functions were omitted. Cite the frozen output and decisive biological sources through source_documents and references, with supporting excerpts in reference findings where applicable. status: COMPLETE can describe a completed assessment of the empty output; it does not imply the full gene review is complete.

Use exact-accession and isoform evidence before calling an omission an obvious miss. Unresolved substrate specificity or absent catalytic architecture can make withholding a term reasonable. Do not insert expected terms into predictions, assign a COR/CNN/LSP/UNC/PLI/NPI/REP score to an absent prediction, or infer recall without a defined expected annotation set. Report zero-output records separately from emitted-claim assessment counts. A missing sidecar means unrecorded review, whereas predictions: [] plus a completed summary records an assessed absence.

If the user requests a cohort of genes with GO predictions, enforce that selection criterion against the actual source. Function text, protein names, and subcellular-location identifiers do not satisfy it.

Evidence and reasoning

Separate biological correctness, agreement with existing annotations, and the quality of the evidence supporting the judgment. A plausible conclusion can have an inadequate rationale; repairing the rationale need not change the conclusion.

• Do not use ARBA assertions to bolster correctness. An ARBA-derived GO term, UniProt keyword, or functional description is an association-based prediction, not independent biological validation. It may be recorded as annotation provenance or as a comparison result. Agreement with ProtNLM does not establish correctness. This is not necessarily circular; the problem is treating an unreliable predicted assertion as validating evidence. If the assertion leads to useful underlying evidence, inspect and cite that evidence instead.
• Assess AI-assisted sources by their evidential content. A generated gene description or accepted/core-function label adds no support by repeating a conclusion; use it to locate the underlying evidence. An OpenScientist investigation that integrates sequence, structural, evolutionary, and literature evidence is a substantive analytical synthesis and can carry substantial weight in adjudication. Cite the report directly in references and supported_by, identify its decisive findings and reasoning, and inspect the underlying sources or analysis artifacts for consequential claims where available. Distinguish reported computations from independently reproduced results and experimental validation. Evaluate limitations and disagreements claim by claim; neither accept a bare verdict as proof nor dismiss an integrated investigation because AI produced it. Do not count the report and its underlying sources as independent replications. A source_documents entry alone does not explain evidential weight.
• Anchor decisive claims with source excerpts. Include concise, verbatim supporting_text for sequence features, domain assignments, experiments, and analytical findings used in the rationale. Use explicit ellipses between noncontiguous excerpts. A report's recommendation is not a substitute for the findings used to reach or qualify the assessment.
• Inspect provenance within database records. "UniProt says" is insufficient when the relevant text or keyword was generated by ARBA or another predictor. Distinguish experimental findings, curator-assessed inferences, sequence/domain observations, and automated functional assertions. Do not count several restatements of the same inference as independent lines of evidence. This does not equate PAINT/IBA's curated phylogenetic judgments with ARBA associations; assess an IBA through its ancestral assertion and experimental grounding.
• Family-based inference is legitimate when the transfer is justified. Identify the target's supported family/subfamily placement, cite the characterized relatives and their relevant findings, and explain why the property is expected to be conserved. State that this is an inference rather than an observation on the target. Consider relevant divergence, paralog differences, sequence features, and taxonomic context without inventing exceptions or requiring a new experiment on every target. Strong, well-grounded transfer can support correctness; a generic domain label or an unsupported family stereotype cannot.
• Match evidence to the exact claim. Evidence for extracellular localization does not by itself establish extracellular-matrix residence, matrix structural activity, or matrix organization. Evaluate each prediction's specificity and mechanism separately. For the OLFML2A example, a defensible rationale would identify verified family/sequence evidence and relevant localization findings in characterized relatives, then explain transfer to the target with its limits. Quoting an AI description of a "matricellular regulator" supplies none of those missing steps. This example does not pre-adjudicate either ECM prediction.
• Distinguish activity from participation. Loss of a catalytic domain can refute an intrinsic enzyme activity without refuting participation in the corresponding biological process through a complex or regulatory role. A scaffold need not catalyze the reaction to participate in that process.
• Separate limited usefulness from biological error. A broad but true term is not incorrect merely because a more informative annotation is available. Use LSP only when a supported, more specific annotation actually exists; functions in different GO aspects are not automatically parent/child terms. Annotation overlap alone does not establish training-data contamination, and a common predicted term alone does not establish frequency bias.

Use the evidence scope requested for the evaluation. In an unrestricted review, follow citations to publications and reproducible analyses. In curated-annotation-only mode, do not silently broaden the evidence scope, and do not substitute ARBA or AI prose when permitted evidence is insufficient. State what remains unverified and use UNC when the exact prediction cannot be supported or refuted. Never invent supporting quotes or recast a paraphrase as a quotation.

Auditing existing prediction reviews

For each prediction, trace the rationale's decisive claims to their underlying sources. Check for ARBA used as validation, AI descriptions used as authority, repeated inferences presented as independent support, and family-level evidence stretched to a more specific function. Keyword searches can find candidates but cannot adjudicate them; an ARBA mention may simply document provenance.

Replace weak reasoning with a source-grounded explanation where possible. Retain the assessment if that evidence supports it; use UNC if support remains insufficient, and an incorrect category only when justified by contrary evidence. Do not automatically downgrade a reasonable family inference or mark a prediction incorrect just because its original rationale was poor. Synchronize the file's overall description with its final per-prediction judgments. Preserve the model's original prediction and source metadata.

Descriptions, summaries, explanations, and other substantive YAML fields must state the current best assessment and its evidence, not the sequence of revisions. Do not write "previously", "reclassified", "on re-review", or accounts of changed minds into those fields. Put the old/new assessments, reasons for changing them, and audit provenance in the separate append-only history mechanism described in docs/history.md, using just new-history to scaffold records. Notes fields may carry review-process context where appropriate.

Validation

Validate sidecars with just validate-predictions <file> [<file> ...]. This checks the PredictionReview schema, score/category agreement, source paths, publication titles, and supporting excerpts, fetching missing PMID caches. CI runs the same check for changed sidecars and publishes prediction-evidence-validation results. The recipe requires excerpts for every supporting-source entry, and local-file excerpt segments must follow source order without reusing the same passage. Missing source text cannot be certified as verified. The check tests citation accuracy; biological relevance and inference quality still require review.

Assessment Categories

Use these categories for review.assessment.

COR: correct novel prediction. The predicted function is supported by allowed evidence and is not already present in UniProt/GOA as an equivalent annotation. Be conservative. In curated-annotation-only mode, COR requires strong non-literature curated support such as a specific reviewed UniProt functional statement, diagnostic domain/subfamily annotation, or local pathway context. Do not call COR merely because the prediction sounds plausible.

CNN: correct but not novel. The prediction is biologically supported and the term is already present in UniProt/GOA, or is an obvious synonym/equivalent of an existing curated annotation. Mere overlap with an ARBA or other automated assertion establishes annotation overlap, not correctness. If biological support remains insufficient, use UNC and describe the overlap separately.

LSP: less precise. The prediction is broadly correct but less specific than an existing curated annotation. Example: predicting generic oxidoreductase activity when the curated annotation specifies the exact substrate or EC reaction.

UNC: uncertain. The allowed evidence cannot validate or refute the prediction. This is the default for plausible predictions absent from curated annotations when no allowed evidence establishes them. In curated-annotation-only mode, UNC should be common and is not a failure mode.

PLI: paralog incorrect. The prediction appears to come from over-propagation within a homologous family or paralog group, and allowed evidence indicates this gene belongs to a different paralog/subfamily/function than the predicted one. Set error_type: PARALOG_OVERANNOTATION.

NPI: nonparalog incorrect. The prediction is refuted for reasons other than paralog confusion: pathway absent in the organism, predicted process incompatible with taxon or compartment, curated function clearly different, activity belongs to an unrelated gene, or the prediction contradicts allowed evidence. Choose the most specific error type, often PATHWAY_CONTEXT_IGNORED, MULTIPLE_FUNCTIONS, CURATION_MISTAKE, or another schema enum value. Other specific options: PSEUDOENZYME_OVERANNOTATION (catalytic activity predicted for a fold-retaining protein that has lost its catalytic residues), LOCALIZATION_DEFAULT (cytosol/cytoplasm assumed when no TM/signal features, mislocalizing secreted/organellar/membrane proteins), TAXON_CONSTRAINT_VIOLATION (term valid only in a different lineage/kingdom, e.g. animal terms for a plant protein), WRONG_INPUT_SEQUENCE (the predictor was fed the wrong protein — a pipeline error, not a model error).

REP: repetition / frequency bias. The prediction looks like a high-frequency/default label assigned without supporting sequence or curated evidence, especially common generic enzyme classes such as histidine kinase or PTS transporter. Set error_type: FREQUENCY_BIAS.

Confidence Score

• 2: concordant with allowed evidence. Use with COR, CNN, or LSP.
• 1: uncertain. Use with UNC.
• 0: discordant with allowed evidence. Use with PLI, NPI, or REP.

Judgment Rules

• Prefer UNC over confident correctness when the prediction is absent from curated annotations and only weakly suggested by domains.
• Once correctness is supported, prefer CNN over COR when the prediction is already in GOA/UniProt. Agreement with an unsupported existing assertion is not sufficient for either category.
• Prefer LSP over CNN when the prediction is correct but materially less specific than the curated annotation.
• Use PLI only when the error is specifically homolog/paralog/subfamily confusion.
• Use NPI for pathway absence, wrong organismal context, wrong compartment/process, or contradiction by curated function.
• Use REP only when the best explanation is a generic high-frequency label rather than a specific biological confusion.
• Do not invent identifiers, publications, or supporting quotes.