Reviewed 2026-10-02T23:49:30-04:00 · Agent inventory · Finish plan · Evidence manifest

Updated decision review for Papers 1 and 2

October 2, 2026, late-evening EDT review. This supersedes earlier coordination statements that actual neighbour-aware training had not run, faint extraction was not in main, or phase-randomization flags alone established instrumental trends. Evidence was checked in the latest task turns, run records, saved tables, inspected figures, current main commits and global Condor queue. No new science jobs or model choices were made by this review.

Recommendation: complete the removal-ledger campaign and extraction acceptance review, but repair the sample and measurement design before one further bounded neighbour-aware training test. Do not launch the final five-field fitting campaign yet. Paper 2 first needs its statistical acceptance test revalidated.

What changed

Paper 1 training result and its limits

Source: /astro/armin/koji/syndiff/dev_runs/neighbour_joint_training_20261002/README.md and its corrected _fluxfix_final results. Earlier A/B/polish/converged generations are superseded after an inherited source-flux-unit mismatch was found. Only the corrected generation is discussed here.

• Baseline A has 252 sources; B adds 75 faint nuisance sources. There are 17 bright training contributors, with one common local ePSF and fixed WCS. This is not a full-CCD A3 refit.
• Primary held-out residual-score change: −0.328%, with a conditional block 95% interval [−1.624%, +0.299%]. The interval includes no improvement.
• RMS difference from the independently trained RAW-faint reference is B/A = 1.0369. It did not get closer to that reference; the reference is not absolute PSF truth.
• Only one bright validation star has at least 80% radius-2 core coverage. Useful covered-core counts in T=8–10 / 10–12 / 12–13 are 1 / 9 / 20. Bright–faint trend closure is underpowered.
• The primary mean squared normalized residuals are about 538 for both arms. Both remain poor absolute fits under the stated noise model. Audit their large residuals and conventions before scaling the test up.
• Nominal Tmag-to-electron-rate conversion, not calibrated PS1 neighbour photometry, supplies the corrected prior centres. The nuisance penalty is not a measured catalogue uncertainty. The smaller all-target transfer gain is a separate conditional diagnostic, not the primary endpoint.

The first figure shows both learned responses remain different from the reference. The second shows the actual validation pixels, large residuals and limited intact-core support. Both were inspected for this review.

Interpretation: physically necessary source accounting should not be discarded, but we cannot claim it eliminates the magnitude-dependent ePSF difference. Nor does this small pilot prove a useful neighbour treatment cannot improve the result.

Targeted photometry is now an acceptance task

The corrected deep-Gaia cohort spans Tmag 13–19 and excludes reference-fit participants. Saved production-interface results account for all 1000 targets and two epochs; failures and low-support cases are retained in the record. Earlier Tmag-18-truncated samples are superseded. The latest output tables are ahead of the older opening text in the task README, and the owner is still completing its report.

The saved checks establish exact producer/replay consistency and excellent conditional matched-profile signal recovery. They also expose PSF sensitivity: the 5% profile-dilation stress test has median recovered injected amplitude about 0.924. WCS, ePSF and upstream sky are held fixed in these injection tests, so they are not whole-training-chain or full-sector validation. The reference catalogue matters: stack versus mean/forced photometry changes the bright-bin comparison on the same extracted targets. A catalogue discrepancy must not automatically be diagnosed as extraction bias.

Before final acceptance, obtain one owner-reviewed table of bias, scatter, nonpositive fluxes, support failures and completeness by magnitude, colour, position and crowding. Preserve signed-flux statistics; magnitude plots selected by S/N alone can bias the faint-end view. State the useful precision range instead of treating ability to process T=19 as equal precision to brighter stars. These are provisional F1 input-generation results, not final five-field acceptance.

Paper 2 needs a corrected acceptance test

The saved null_stationarity_validation/summary.json tests 128 synthetic realizations per case. For both independent slow sinusoids and independent linear trends, the legacy phase method rejects all 128 even though no common instrumental signal was added. A whole-history sign method behaves much better in these particular synthetic distributions, but assumes sign symmetry and does not yet simulate all shared calibration covariance, selection or real stellar populations.

I therefore retract the earlier coordination shorthand that the phase-based group counts by themselves prove instrumental failures. Physical residual trends and signal attenuation remain separate evidence; existing light curves are not thereby accepted as clean. Validate false-positive rate and detection power with the actual cadence, gaps, extraction/calibration covariance and realistic variability before using a revised statistic to accept or reject corrections. Keep injected-signal preservation independent of the trend test.

Recommended next actions

1. Sample and baseline audit before any training launch. Use completed C4 ledger cells to select fully contained, intact-core bright validation stars with useful magnitude/color coverage. Quantify sensitivity from actual sample variance; do not choose another tiny crop first and discover inadequate coverage afterwards. Establish consistent image/flux/noise units, normalization and baseline parity; locate the large residuals shown above.
2. One stronger A/B comparison. Same initialization, original contributor roles, pixels/noise, WCS policy, regularization and schedule. Evaluate on the same neighbour-complete held-out scenes. Calibrate neighbour flux/prior assumptions using real independent photometry, and retain a weak/free-prior sensitivity control. Record source completeness, partial survival and training/reference overlap. Do not simultaneously introduce new shape terms or retune the background in the main A/B comparison.
3. Accept two handoffs in parallel. Let C4 ledger accounting/transport complete under its existing owner. Let the faint-photometry task finalize its corrected-cohort report, physical pixel/flux controls and reference-catalogue conventions. Neither requires a new open-ended research branch.
4. Make the recipe decision from the improved pilot. If neighbour treatment gives robust useful held-out improvement without flux/WCS regression, adopt it. If not, retain only justified source accounting, state the unresolved magnitude dependence and decide whether those limits permit the intended paper claims. A background alternative is a subsequent targeted test if material bias remains.
5. Only then run final five-field products and K=5 checks. Existing verified templates/catalogues remain reusable when pixels/geometry are unchanged. Final extraction must follow the same accepted generation. Methods writing, limitations and figure structure can proceed now; neither Paper 2 closure nor complete detector-physics attribution blocks Paper 1.

Choices for the user

Conclusion: the project has moved from missing implementation to inconclusive training evidence and measurement validation. The best next move is one stronger controlled experiment and acceptance of the completed extraction work, rather than another wholesale rerun or a new physical correction.