# Paper 1 §5 figure set, rebuilt from the near-final products (2026-10-07)

**Question.** Build the §5 figures of Paper 1 ("Final de-crowded image and measurements") from scratch on the newest
products, showing the method's best version, with Hotpants and a colour-independent ePSF kernel as the only baselines.
No earlier figure is reused.

**Plan:** `FIGURE_PLAN.md` (claims, data, metrics, images per figure; written before the figures).
**Code:** `/home/kshukawa/syndiff_pipeline/dev/paper/paper1_single_ffi/figures/s5_20261007/` (dev repo HEAD a3bbc9a when
run; scripts not committed — the orchestrator owns commits). Read-only imports of the chain code (`forward_model/chain/kernels.py`, identical on main; the run used the
`perband-v3` worktree, now removed — code at tag `archive/perband-v3`) are used only by `kernel_recon.py` (Operator / render_grid of the kernels stage; worktree HEAD
98d84b1 at run time, chain products were built at 498af2d / perband-fft 9a430d9; the check reproduces the stored eps-rule
`rms_best` bit-for-bit, so the operator is unchanged).

**Inputs (all read-only):** `perband_v3_20261005/<F>/` for F ∈ {F1 = S24 C2K2, F2 = S20 C3K3, S22 = S22 C3K3,
C1 = S24 C1K2, C4 = S23 C4K2}: `perband/band_templates`, `kernels/`, `final/<stem>/{band_w,achrom_w}`,
`hotpants/<stem>`, `bootstrap/diff/hp_d` (preliminary image), `score/<stem>/weighted`, `fit`, `refit`, `compare`;
frozen G4 scorer star lists (`perband_3field_20260929/scorer`, `perband_v3_20261005/g4_scorer_v12/scorer`);
XP synthetic TESS magnitudes `heldout_harness_20260929/xp/{F1,F2,S22}_xp_tess.csv`; sky levels from
`bkg_offset_20261006/README.md`. Every figure's `.json` lists its input paths with sha256 and all quoted numbers.

**Commands** (`mamba activate syndiff`, in the scripts dir):
```
python region_choice.py            # catalogue-only region rule, BEFORE any residual image was opened
python fig_kernels.py              # fig_s5_kernels, fig_s5_kernel_checks
OMP_NUM_THREADS=16 python kernel_recon.py F1 ; OMP_NUM_THREADS=16 python kernel_recon.py --mix F1
# (run 10-07 with PYTHONPATH=.claude/worktrees/perband-v3; that worktree is removed, its code is at tag archive/perband-v3;
#  kernels.py is identical on main, so the editable install / PYTHONPATH=/home/kshukawa/syndiff_pipeline suffices)
python fig_region.py ; python fig_templates.py
python compute_colour.py ; python fig_colour.py ; python table_g4.py
python fig_refit.py ; python fig_photometry.py
```

**Held-out status.** This product set has **no unseen frame**: every image is the ePSF/WCS training frame of its
field. The scored stars (set H, $13\le T<16$) are held out by star (the ePSF trains on $T\le13$), and the stars and
pixel sets were frozen before any of these images existed. The bright-star photometry (Fig 5.5) is in-sample.

## Figures (all viewed)

### Fig 5.1 (supporting) `fig_s5_templates.png` — per-band templates on the fitted WCS
Zoom of Fig 5.3a at template sampling: $\sum_b T_b$, the template colour
$$c_T = c_{\rm ref} + \frac{\sum_b \delta'_b T_b}{\sum_b T_b}$$
($T_b$: band-$b$ template, e⁻ s⁻¹ per subcell; $\delta'_b$: band colour labels with the adopted band weights, BP−RP
units, $(-5.37,-0.81,+3.03,+6.27)$ for F1, identical to those logged by the chain's final stage; $c_{\rm ref}=0.954$:
the ePSF colour reference) and the four band shares. At the six H stars in the zoom, $c_T$ follows Gaia BP−RP to
0.02–0.08 mag (e.g. 1.78 → 1.76, 0.89 → 0.90). $\sum_b T_b$ equals the combined template to $5\times10^{-8}$
(`sum_check.json`).

### Fig 5.2 `fig_s5_kernels.png` — response and matching-kernel bases (F1)
$$E_b = P_0 + \delta'_b P_1,\qquad K_b = K_0 + \delta'_b K_1$$
$E_b$: effective response of band $b$ at a node (OS4 grid weights, pixel-integrated); $P_0$, $P_1$: least-squares
intercept and colour slope (per mag) through the chromatic ePSF at 25 colour quantiles of the training stars;
$K_0, K_1$: Wiener-Fourier matching kernels of $P_0, P_1$ against the template's PS1 pre-blur $\mathcal G$
($\sigma\approx0.47$–$0.56$ TESS px) and the native pixel; the map is linear, so $K_b$ is exact.
Three nodes: nearest the optical axis (0, 2048; 75 px from the axis), centre (819, 819), far corner (2048, 0; 2972 px).
- $P_1$ is a dipole along the axis direction at the centre and far corner (redder light moves away from the axis, the
  sense of the residual dipole in Fig 5.3c), and a sharpening (positive core, negative ring) near the axis.
  $|P_1|_{\max}$ = 11–13 % of the $P_0$ peak per mag.
- The per-band responses are extrapolations: $|\delta'_b|$ up to 6.3 mag, so $E_r$ peaks at 0.43 and $E_y$ at 1.67 of
  the $P_0$ peak near the axis; only share-weighted mixtures ($x\in[-0.4,+0.9]$ for the training colour range) are physical PSFs.
- Kernels ring (Wiener deconvolution of a 0.5 px blur). At the far corner $K_0$ has negative lobes up to 68 % of its
  peak; the ringing sits at spatial frequencies the pre-blurred template does not carry, which the native reconstruction
  check (Fig 5.2b c) bounds.

### Fig 5.2b `fig_s5_kernel_checks.png` — normalisation, linearisation, reconstruction (5 fields / F1)
- (a) $\sum K_b - 1$ over 36 nodes (adopted-weight labels): S24 C2K2 −0.57…+0.53 %, S22 C3K3 ±0.84 %, S24 C1K2
  −0.61…+0.69 %, S23 C4K2 −2.0…+2.2 %, **S20 C3K3 −6.6…+5.9 %** at one node (row 3, col 2; (819, 1229)) where the eps
  rule chose $\varepsilon=10^{-6}$ and the kernel exceeds the 127-subcell support ($\sum K_1 = 0.0106$ instead of 0);
  for a real star ($x\approx-0.4$…$+0.9$) that node's kernel sums to 0.987–1.001. Single kernel within ±0.07 % except
  S20 C3K3 (−0.86 %).
- (b) linearisation error vs colour (fraction of the node's $E(c_{\rm ref})$ peak): node median 0.08–0.17 %, worst
  node 0.7–1.5 % (at the colour extremes); one colour-independent response costs 1.8–1.9 % (median), up to 11–15 %.
- (c) S24 C2K2 native-pixel reconstruction (point source in the pre-blurred template → $K$ → native pixels, vs $E$
  rendered at the same phase; 23² px × 16² phases; the kernels stage's own eps-rule metric, which this code reproduces
  bit-for-bit for the stored `rms_best`): rms / native peak, node median (max): single bands $K_r$ 0.34 (0.77) %,
  $K_i$ 0.10 (0.25) %, $K_z$ 0.21 (0.58) %, $K_y$ 0.29 (0.84) %; single kernel 0.10 (0.27) %; **the mixtures real stars
  use, $K_0+xK_1$ at the 2/50/98 % training colours: 0.10 (0.25), 0.10 (0.26), 0.12 (0.35) %**. So the strong kernel
  ringing (Fig 5.2) costs ≈ 0.1 % of peak rms on native pixels, the same as the colour-independent kernel. Other fields
  not run (≈ 15 CPU-h per field).

### Fig 5.3a `fig_s5_region.png` — one real region, preliminary vs final
Region chosen by a catalogue-only rule before any residual image was opened (`region_choice.py`,
`region_choice.json`): 192² px tiles ≥ 64 px inside the CCD with no $T<7$ star or bad column (the first rule, $T<9$,
left no tile: F1 has 667 $T<9$ stars, so the fixed fallback list was used), most red H stars → tile x 1600–1792,
y 1024–1216 (7 red, 60 H stars); zoom = its 64² sub-tile with the most red H stars (x 1616–1680, y 1024–1088).
Grey = footprints of the 87 $T\le13$ stars, which are not in the template and stay at full flux in every image.
- Off-footprint pixels: robust σ 0.452 (preliminary) → 0.393 e⁻ s⁻¹ (final), Hotpants on the same template 0.455;
  |D|/σ > 5: 1.85 % → 1.17 %. Median sky −0.085 (preliminary) vs +0.001 e⁻ s⁻¹ (final): the Hotpants background
  pedestal (bkg_offset_20261006), visible as the uniform tint of (d).
- (d) preliminary − final and (e) Hotpants − final look the same (rms 0.734 vs 0.739 e⁻ s⁻¹): almost all of the change
  comes from the kernels, not from the remapped template.
- In the zoom the preliminary image shows ± pairs at faint stars; the final image is cleaner but the three red H
  stars keep a negative core (the red core deficit of Fig 5.3c).

### Fig 5.3b `fig_s5_colour_metrics.png` + Fig 5.3c `fig_s5_colour_stacks.png` — colour-resolved residuals, 5 fields
Per-star metrics from the scorer (G4 v1.1/v1.2 rules): $\chi^2_s(r\le2.5)$ with the reference noise plane (≈1.58× over,
noise-only ≈ 0.40); radial dipole $d^{\rm rad}$ (positive = residual light away from the optical axis); aperture
residual $\rho=\sum_{r\le3.5}(D-\beta)/F$ ($\beta$: annulus median, $F=10^{-0.4(T-20.44)}$ e⁻ s⁻¹). Stacks: 4×
oversampled $(D-\beta)/F$ of the same stars, shared ±1.2 % scale (Hotpants/single-kernel dipole lobes reach ±2.6 % and
saturate).
- **Dipole:** Hotpants and the single ePSF kernel rise from about −20 mpx (BP−RP ≈ 0.7) to +108…+146 mpx (BP−RP ≈ 2.4)
  in every field; per-band stays within −14…+12 mpx at all colours. Red − blue: +81…+108 (Hotpants), +75…+107 (single),
  **+3…+10 mpx (per-band)**. The stacks show the ± lobe along the away-from-axis arrow for Hotpants and single kernel
  and no lobe for per-band.
- **χ²:** per-band is lowest at BP−RP ≥ 1.2 in every field (red χ² 2.9–6.2 vs 6.8–12.3 Hotpants). At 0.8–1.2 the three
  are within ~10 % and per-band is sometimes slightly worse (F1 1.0–1.2: 2.39 vs 2.16 single kernel; C4 1.0–1.2: 3.08 vs
  2.83 Hotpants); blue (< 0.8) per-band is lowest except C4 (4.47 vs 3.94 single kernel).
- **Remaining failure (shown, not hidden):** per-band over-subtracts red stars: $\rho$ = −2.2…−2.9 % at BP−RP 1.5–2 and
  −4.8…−6.6 % at BP−RP ≈ 2.4, versus −1.4…−2.1 % for Hotpants; red − blue $\rho$ −4.0…−5.5 % (Hotpants −1.5…−2.3 %). In the
  stacks this is the symmetric negative red core (−1.6…−2.3 % per native px at the centre). Blue stars keep a positive
  core (+1…+1.5 % in $\rho$). Cause per `perband_v3_20261005/redstar_diag`: the trained ePSF makes redder stars sharper
  (~2.5 %/mag light into r ≤ 2 px), faint stars do not follow it.

### Table 5.3 `table_s5_g4.{md,csv,tex}` — five-field scores
Values from each field's `metrics.json`; the median H χ² was recomputed from the per-star CSVs and agrees to 1e-4.
G4 verdict (frozen rule): PASS for S24 C2K2, S20 C3K3, S22 C3K3, S24 C1K2; **S23 C4K2 FAIL** only on the overall-χ²
significance (−0.20, −1.4σ; needs 3σ; 983 H stars, 85 red). Red χ² 6.2 vs 10.0, dipole +8 vs +81 mpx, pixel χ² 1.43 vs 1.77
in C4 all improve.

### Fig 5.4 `fig_s5_refit.png` — refit stability
Refit on the final image minus the calibration fit on the preliminary image, same initialisation and recipe, all
scene stars ($T\le13$). Recomputed here and asserted equal to `compare/compare_fits.json`.

| field | eff. position rms / max (mpx) | WCS rms (mpx) | ΔT median (rms) (1e-3 px²) | ΔEE(r≤3) (ppt) | flux T 7–9 … 12–13 (ppt) |
|---|---|---|---|---|---|
| S24 C2K2 | 0.40 / 3.6 | 0.82 | +0.17 (0.31) | −0.35 | +0.30 … +1.08 |
| S20 C3K3 | 1.41 / 12.2 | 1.27 | +0.05 (0.34) | −0.10 | +0.10 … +0.50 |
| S22 C3K3 | 2.01 / 13.4 | 2.05 | +0.03 (0.50) | −0.13 | +0.19 … +0.55 |
| S24 C1K2 | 0.94 / 6.6 | 1.09 | +0.14 (0.25) | −0.18 | +0.07 … +0.74 |
| S23 C4K2 | 1.32 / 11.3 | 1.74 | +0.20 (0.40) | −0.30 | +0.24 … +1.04 |

Tolerances (D14 draft): position rms ≤ 2 mpx, |ΔT| ≤ 0.2e-3 px², |flux| ≤ 1 ppt.
- Position rms passes in 4/5 (S22 2.01, at the limit). The maxima (6.6–13.4 mpx) are local: panel (f) shows compact
  blobs at single ePSF nodes (ePSF centroid changes), not a global WCS change.
- ΔT medians pass (0.03–0.20); per-node rms 0.25–0.50 exceeds 0.2.
- Flux rises toward faint stars in every field and reaches the 1 ppt tolerance at $12\le T<13$ in S24 C2K2 (1.08) and
  S23 C4K2 (1.04). Panel (d): the T 12–13 change tracks the empty-sky level difference between the two images
  (0.06–0.14 e⁻ s⁻¹, from bkg_offset_20261006) — the Hotpants background pedestal in the preliminary image, absorbed by a
  fit with no background term. A simple additive $A/F$ law does not fit the T dependence (it was tried and dropped:
  the observed rise is close to linear in T). hpfix_d14_20261007 (running) rebuilds the preliminary images without the
  pedestal; its first field (S24 C1K2) gives +0.05…+0.20 ppt. **Redraw this figure when hpfix lands.**

### Fig 5.5 (bright half) `fig_s5_photometry_bright.png` — fluxes on the final image vs Gaia XP
$\Delta m = -2.5\log_{10} f + 20.44 - T_{\rm XP}$, per-field median over $9\le T_{\rm XP}<12$ removed ($f$: refit flux,
e⁻ s⁻¹; $T_{\rm XP}$: synthetic TESS magnitude from Gaia DR3 XP spectra with the TESS v2.0 response). F1/F2/S22 only
(XP TESS tables do not exist for C1/C4). In-sample.
- Robust per-star scatter 13–22 mmag at all T (not photon-limited; includes XP synthesis error).
- Bright end: +17…+19 mmag at T ≈ 8.3, falling to −3…−5 mmag at T 12.5–13 (zero point set at 9–12).
- Colour: Δm rises +32/+38/+35 mmag per mag of BP−RP (measured fainter for red stars), the same sign and size as the
  earlier held-out estimate (+34/+41/+37, MODEL_FREEZE Fig 17 entry) and as the red-sharpening diagnosis above.
- The $T>13$ half (targeted extraction) is **not built**: see Open items.

## Bugs / issues found
1. **No tile passed the planned region rule** (T<9 exclusion): F1 has 667 T<9 stars. Fixed with a pre-declared
   fallback list (still catalogue-only), recorded in `region_choice.py`.
2. **Band labels:** `band_epsf.npz` stores the production-weight labels $\delta_b$ (−5.95…+5.93), but the final image
   was built with the adopted-weight labels $\delta'_b$ (−5.37…+6.27). The first kernel figure draft used $\delta_b$;
   corrected to $\delta'_b$ (verified against every field's chain log).
3. **F2 kernel-sum outlier** (above): node (3,2), ε = 1e-6, $\sum K_r = 0.934$, $\sum K_y = 1.059$ (adopted labels). Not fixed (pipeline
   product); worth a kernel-support or ε-floor check before the paper run.
4. My first reconstruction check (Fourier, OS4 grid, without subcell integration of the pre-blur) gave 4–33 % errors —
   a wrong operator, not a kernel problem. Replaced by the chain's own native-pixel operator, which reproduces
   `rms_best`.

## Open items for the user
- **Targeted de-crowding (Fig 5.5 stamps + faint Δm):** `faint_target_photometry_20261002` was built on the
  provisional e2e_f1_20260930 mapping, band cells, kernels and image, so it cannot be reused. A rerun on v3 needs
  (i) `syndiff star prepare-band` with a manifest pointing at the v3 F1 snapshot (mapping `perband_v3_20261005/F1/mapping`,
  band cells `paper_dataset_20261001/F1/perband/band_cells`, contributions `perband_v3_20261005/F1/perband/contrib`,
  a v3 publisher list, kernels `perband_v3_20261005/F1/kernels/band_epsf.npz` with the adopted labels, store weights
  = adopted, a new mask cache) — previously 32 Condor jobs × 2 CPU / 12 GB; (ii) target and neighbour profiles rendered
  from the v3 fit (`dev/forward_epsf_wcs/faint_targets/render_profiles.py`, profile_config pointed at
  `paper_dataset_20261001/F1/fit` + scene); (iii) `syndiff star extract-band` on `final/<stem>/band_w` with its
  `match.json` and an $a(x,y)$ scale image; (iv) a second-epoch final image if two-epoch fits are wanted (v3 has only
  the training frame). Not launched (Condor campaign).
- **Refit figure** depends on hpfix_d14_20261007 (running).
- **Red-star over-subtraction** (ρ −5…−7 % at BP−RP 2.4) is the main visible weakness; the figures show it.
- **C4** fails G4 only on significance; decide whether to show it as a fifth field with that caveat (done here).

## Headline / Conclusion (2026-10-07)
On the five near-final FFIs, the per-band ePSF kernels remove the colour-dependent residual dipole that both
Hotpants and a colour-independent ePSF kernel leave on identical templates, WCS and stars (red − blue +75…+108 →
+3…+10 mpx, ≥ 30σ in F1), and lower the red-star χ² by a factor 1.6–3. The cost is a symmetric over-subtraction of
red-star cores (aperture residual −5…−7 % at BP−RP ≈ 2.4), traced to the ePSF colour model making red stars too
sharp. The refit on the final image moves positions by ≤ 2 mpx rms (one field at the limit) and ePSF size within
tolerance, but fluxes by up to 1.08 ppt at T 12–13, which follows the preliminary image's background pedestal and is
being retested (hpfix_d14_20261007). Bright-star fluxes carry a +32…+38 mmag/mag colour slope against Gaia XP.
Limits: one (training) frame per field, no unseen frame; C4 fails G4 on significance; the targeted faint-star half of
§5.5 still needs a v3 rerun.

## 2026-10-09 rerun on new fits
Same driver as §4: `run_all.py --fit-root <run root> --field F2 --stage final --out <dir>` (S5 runs in stage final only; fields assembled from every field
ready in the run). Legacy: `--set s5 --preset perband_v3 --no-...` -> `python run_all.py --set s5 --preset perband_v3 --out <dir>` reproduces the 10-07 figures
(all 10 JSONs identical). Fig 5.x(d) sky-level panel needs `P1FIG_PEDESTAL=<json {F: e-/s}>` for new runs (the old values were hard-coded for perband_v3).