> **2026-10-03 follow-up:** the staged lightly masked reference expansion is completed in [LIGHT_REFERENCE_EXPANSION.md](LIGHT_REFERENCE_EXPANSION.md); its signal gate fails and no new reference arm is adopted.

# Population-informed static PSF prior: signal stability without a photometric cure

**2026-10-03 conclusion:** a static shape prior learned from fully observed
other stars does not improve the missing-light photometry in the tested
five-mode model. The prior-only prediction preserves injected signals well,
but has larger temporal errors than the original population PSF. Allowing
the retained-pixel posterior fit leaves nearly the same hidden-core failures
as the previous weak-prior fit. Neither arm is adopted. The broader temporal
ePSF/WCS, reference-population and photon-noise goals remain active and unmet.

Run: `/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/`.
Worktree: `/home/kshukawa/syndiff_pipeline/dev/.worktrees/temporal-raw-epsf`;
branch `codex/temporal-raw-epsf`, code local only. Previous turn was progress:
the retained-pixel static correction was measured and rejected with matched
hidden-pixel, signal and residual-grid validation.

## Actual source and epoch exclusions

The five zero-sum latent modes, linear per-exposure flux/sky fit and fixed
WCS/blur/focus transport are defined in `PARTIAL_STATIC_PSF.md`. This experiment
changes the shape prior rather than the observations or error floor.

For a target in source fold 0, full-support prior trainers are the 89 eligible
stars in fold 1; for fold 1, they are the 76 eligible stars in fold 0. Every
trainer uses the **same target-fold population template and state convention**
as the evaluated target. Reusing that trainer's own reverse-fold model could
have imported the target's hidden pixels through its population reference.
Each prior is learned only on the opposite orbit from evaluation.

The base WCS plus target-fold RAW correction replays existing target positions
within 3.61e-13 pixel. Target baseline log photometry replays exactly. The
catalogue source IDs actually used in each prior are stored and checked,
not inferred from a nominal fold label. Upstream initial templates, global
WCS and response models are still fixed inherited inputs, so the exclusions
prove independence of these new learned layers, not the complete pipeline.

## Prior construction and declared alternatives

Full-support training fits provide dimensionless static parameters $u_r$.
The prior mean at a target is

$$\mu_s=h_s\,\Beta,\qquad
\Sigma=0.8S+0.2\operatorname{diag}(S)+10^{-6}I.$$

Here $r$ indexes a prior-training star and $s$ a target. Each five-component
$u$ is the corresponding physical shape coefficient divided by its fixed
scale: 0.05 pixel for the two first-difference modes and 0.05 pixel squared
for the three second-difference modes. The row $h_s$ contains an intercept
and training-standardized detector x/y, Gaia BP-RP, and a catalogue brightness
proxy derived from TESS magnitude. $\Beta$ is the fitted five-output catalogue
regression coefficient matrix. Ridge strength is 0.001 times the number of
training stars, with no penalty on the intercept. $S$ is the residual
covariance of the training $u$ vectors after that regression, with the
regression degrees of freedom removed from its denominator. $\Sigma$ is a
dimensionless parameter covariance; $I$ is the five-dimensional identity.
The diagonal shrinkage and covariance floor were fixed before evaluation.
They are **not added to photometric uncertainty**.

The prior-only arm uses $u_s=\mu_s$ without fitting the target's shape. The
posterior arm fits its retained training pixels with the same profiled
flux/sky objective and adds

$$\chi^2_{\rm prior}=(u_s-\mu_s)^\mathsf{T}\Sigma^{-1}(u_s-\mu_s).$$

All quantities in this prior term are dimensionless and defined above.
Pixel-error inputs, mode bounds and source/epoch support are unchanged.
Catalogue standardization, regression and covariance estimation index out
target coefficient rows before any calculation; changing excluded rows to
arbitrary values leaves the learned prior identical.

Both arms retain all 172 controls, 165 eligible sources, the 36 preselected
FFIs and masks 2/4. The joint flux estimator covers all 165 eligible sources;
the ring-aperture estimator retains its existing unsupported row per mask,
giving 164/165. No source is removed to improve performance.

## Numerical and signal validation

Five unit tests pass, including a correlated nonzero-prior Jacobian and
prior-source exclusion. All 660 full-support prior fits and 2,064 retained-
pixel target fits converge. No prior fit hits a bound; twelve target fits
do. Sixteen real target-pixel/held-epoch perturbation checks are invariant.
Independent starts on four catalogue-spaced sources agree to 4.45e-16
fraction in objective and 1.18e-8 in scaled parameters. These are numerical
checks, not scientific acceptance.

There are three datasets: nominal observations; known 1-ppt measured-profile
signals added to all sources; and the same variable prior-training stars
with the evaluated target's own pixels unchanged. Both prior and posterior
learning are repeated. The last dataset measures variability imported through
shape-prior learning, separately from the target's signal. Signal profiles
are different from the fitted five-mode family, but still derive from the
observed full-support profile; global photon means/WCS/response remain fixed.

![Actual known signals and prior-variable contamination](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_hierarchical_psf/assessment/signal_histories.png)

| Joint estimator | Mask 2 worst primary amplitude error, all 172 [%] | Mask 4 worst error [%] |
|---|---:|---:|
| Prior only | 0.04443 | 0.02402 |
| Retained-pixel posterior | 0.53494 | 0.07169 |

The posterior heavier-mask case still fails the existing 0.5% amplitude
criterion. Prior-only imported variability has median/p95 0.0603/0.1927 ppm
for mask 2 and 0.0184/0.0741 ppm for mask 4 under this injection. The much
smaller posterior values approach numerical-sensitivity scales and are not
claimed as a physical noise floor or a resolved astrophysical effect.

## Matched photometric outcome

| Joint temporal error, median/p95 [ppt] | Mask 2 | Mask 4 |
|---|---:|---:|
| Original population PSF | 0.6343 / 2.5751 | 0.2609 / 0.8764 |
| Previous weak static fit | 0.8482 / 5.6636 | 0.2732 / 1.0325 |
| Other-star prior only | 0.7743 / 2.6538 | 0.2685 / 1.0384 |
| Retained-pixel posterior | 0.8433 / 5.6674 | 0.2732 / 1.0325 |

These compare with the same observed complete-r3 control measurement, using
one whole-series mean and no orbit-by-orbit normalization. They are not the
full-cadence candidate's light-curve RMS or independent astrophysical truth.
The underlying histories show that the posterior scarcely changes the earlier
problem, while the prior-only histories remain close to the population baseline
with additional source-dependent differences.

![Matched photometry and all-source signal check](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_hierarchical_psf/assessment/photometry_and_signal.png)

![Actual held-out source histories](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_hierarchical_psf/assessment/heldout_histories.png)

The hidden-core image failure remains in the posterior. Residuals and model
differences below share one scale; the missing pixels were never available
to the target shape fit and are shown solely as clean-control evaluation.

![Held-out pixel residuals and model differences](/astro/armin/koji/syndiff/dev_runs/temporal_raw_epsf_20261001/results/partial_hierarchical_psf/assessment/heldout_pixels.png)

## Required grids and reproducibility

Four new model grids, each absolute and cell-relative, are generated and
inspected for prior-only/posterior under both masks. Their full clean-source
evaluation support, four snapshots and rate normalization match the previous
population baselines. There are 165 inputs and 159 in the standard
magnitude/position range. Sparse bins and automatic per-image display scales
remain explicit; a relative grid subtracts the cell-wide residual stack.
The main canvas embeds all eight images. The existing shared residual gallery
contains the new `raw_partial_prior_m{2,4}_{prior_only,posterior}_20261003`
labels alongside their matched `raw_partial_static_m{2,4}_population_20261003`
baselines. Earlier entries are preserved and copied image hashes verified.
The unavailable external Artifact mirror is not republished.

Frozen local jobs `partial_hierarchical_psf` and
`partial_hierarchical_assessment` both exit 0. Training used four processes
with a 6,000-MB local resource hint; assessment used two CPUs/4,000 MB.
These hints are not enforced local memory limits. Exact source snapshots,
commands and execution metadata are under `condor/`; outputs, prior-training
IDs, covariance matrices, all fit records, arrays, scores and figures are
under `results/partial_hierarchical_psf/`. The source files are
`partial_population_prior.py`, `partial_hierarchical_psf.py`,
`partial_hierarchical_assessment.py`, and the extended tested
`partial_static_model.py`. Original default solver behavior is retained.

## What this changes next

This rejects the tested static five-mode prior as a cure; it does not show
that independent stars cannot constrain a better PSF model. Simply tuning
this prior to flatten the target histories is not justified. More expressive
temporal shape or model-discrepancy treatment remains a separate hypothesis.
Earlier pixel-level CRM-corrected reference runs already exist, so they must
be read before proposing that operation as a new experiment.

The next controlled expansion can start with lightly masked references,
whose measured transfer errors were much smaller in
`PARTIAL_REFERENCE_SUPPORT.md`, using the better-performing original population
model. Select candidates by model/geometry support rather than light-curve
flatness; validate actual neighbour variability, full-cadence support, count
restoration, reference response and original science histories before adoption.
The older complete-r2 test is not this partial-r3 observable and must not be
repeated unchanged. Heavily masked and crowded sources remain in the full goal.

**Conclusion:** the learned prior has sound exclusion and signal checks but
does not improve the required photometry. Neither new arm is promoted, no
uncertainty floor is inflated, and instrumental-trend removal remains unfinished.