# Nonlinearity and flattened stellar peaks — 2026-10-02

Status: diagnostic investigation complete; physical attribution inconclusive;
no production correction accepted. Owner: Codex nonlinearity investigation.

Question: can incorrect detector-response calibration explain brightness-dependent
peak flattening, and can an independently validated correction repair it?

Inputs: archived S24 C2 K2 raw/calibrated pairs and calibration XML under
`../temporal_raw_epsf_20261001/inputs/calibration_audit/`; existing calibrated-FFI
scene residuals under `../magwidth_20260924/raw/lanes/`.
The historical word RAW in the latter denotes calibrated, unsubtracted FFIs.

Plan: measure actual nonlinear CAL transfer after removing the candidate flat;
test response-shaped residuals against brightness-dependent broadening with
fixed pixel support, colour/spatial controls, held-out stars, and null/injection
checks. No production pixel correction until physical sign, generalization and
signal preservation are supported. Existing input files are read-only.

Code: task scripts in this directory; backup branch
`codex/nonlinearity-peak-20261002` in
`/home/kshukawa/syndiff_pipeline/dev/.claude/worktrees/codex-nonlinearity-peak`.
No shared production code or input products are modified.

Commands (activate `syndiff` first; full script pins one BLAS/OMP thread):

```bash
python /astro/armin/koji/syndiff/dev_runs/nonlinearity_peak_20261002/cal_transfer.py
python /astro/armin/koji/syndiff/dev_runs/nonlinearity_peak_20261002/cal_transfer.py --c3k3
python /astro/armin/koji/syndiff/dev_runs/nonlinearity_peak_20261002/prepare_oof.py
bash /astro/armin/koji/syndiff/dev_runs/nonlinearity_peak_20261002/run_v2.sh
```

Inputs added: one S20 C3 K3 uncalibrated counterpart and its public flat/black
maps, with download URLs and hashes in `inputs/manifest.json`. Existing K5
out-of-fold hp_d pixel dumps are from `bullseye_20261001/dumps/`; conversion
hashes and producing code pin are recorded in `OOF_*/provenance.json`.

**Only `v2/` is the accepted diagnostic implementation.** The exploratory
root-level shape outputs used a cell-mean response profile. An own-profile
injection demonstrated response/width leakage; v2 uses each star's saved
pixel model for the response and flux nuisance terms. Root-level v1 results
are retained for audit and must not be used as physical calibration estimates.
The calibration reconstruction outputs at root are independent of this change.

The XML's raw polynomial coefficients must not be used directly as a
multiplicative response. Final reconstruction uses the transformed SPOC
coefficients in the pinned independent TICA implementation, and tests the
actual raw/calibrated pixel transfer. No claim that the laboratory model is
still physically correct follows from reconstructing its application.

## Headline / Conclusion (2026-10-02)

The intended SPOC response is applied accurately in five raw/calibrated pairs
on two CCDs. A response-shaped stellar residual is measurable, but this does
not establish an incorrect physical response curve as the common cause.
The candidate removes part of the bright core/ring; other structured residuals
remain. Independent-sector transfer is inconclusive, so do not apply it globally.

Canvas: [full illustrated investigation](http://localhost:18888/nonlinearity-peak-20261002/)
and [local HTML](/astro/armin/koji/syndiff/dev_runs/nonlinearity_peak_20261002/index.html).

![Same-pixel calibration reconstruction](cal_transfer.png)
![Actual stellar pixels behind the reconstruction](cal_stamps.png)
![Correction comparison on measured residuals](comparison.png)

### Key held-out scores

These are reductions of normalized residual squared power after nuisance
shape controls, **not photometric precision or recovered flux accuracy**.
The response-only addition has one coefficient per output, trained on four
source folds and evaluated on the fifth. Intervals are conditional source
bootstraps and omit original-PSF uncertainty/spatial correlations.

| Dataset | Sources | Response improvement | 95% interval |
|---|---:|---:|---:|
| S20 C3 K3 calibrated FFI | 5497 | 0.0649% | 0.0266–0.1075% |
| S20 C2 K2 calibrated FFI | 5524 | 0.0185% | −0.0050–0.0442% |
| S20 C1 K2 calibrated FFI | 5082 | 0.0212% | 0.0016–0.0401% |
| S20 C4 K2 calibrated FFI | 4962 | 0.0207% | −0.0008–0.0453% |
| F1 hp_d / existing K5 OOF | 5144 | 0.2627% | 0.0973–0.4586% |
| F2 hp_d / existing K5 OOF | 5227 | 0.0789% | 0.0071–0.1613% |
| S22 C3 K3 hp_d | 5331 | 0.1763% | 0.0773–0.2682% |
| S53 C3 K3 hp_d | 6315 | 0.0081% | −0.0226–0.0388% |

The underlying absolute/model/residual and arm-minus-baseline stacks for
every row are in `v2/<dataset>/shared_stacks.png` and
`absolute_difference.png`, embedded in the canvas. Colour × magnitude ×
3×3 CCD grids are included for raw C3 K3 and both OOF cases. These are
post-hoc residual diagnostics, not new scene fits or official ePSF training runs.
No standard training-comparison tab was added for these projected residuals.

After adding interpolation-phase controls, transferring the fixed S20 C3 K3
response to S22 gives 0.0907% improvement (95% −0.0133–0.1670%); transfer to
S53 gives 0.0297% (−0.0062–0.0645%). Both include zero. The same-epoch OOF
hp_d comparison improves but is not independent replication.
See `v2/sensitivity.json` and the corresponding `transfer_stacks.png` images.

### Validation and interpretation limits

- Identical per-arm pixels/sources; explicit source IDs and fixed quality cut.
- Zero, response, width and joint conditional injections pass; own-profile
  injection exposed and fixed v1's response/width leakage. This remains a
  shared-profile estimator test, not independent proof of PSF correctness.
- Original raw-FFI and later-sector models are in-sample; F1/F2 hp_d reuse
  existing out-of-fold models. No new joint nonlinear scene fit was run.
- Image data/model are normalized by fitted source flux. Projecting flux
  removes an important discriminator between charge loss and redistribution.
- Nonlinearity on an asymmetric PSF need not change both axes equally. The
  historical "H2 isotropic" argument does not exclude nonlinearity.
- No temporal signal-preservation or absolute flux-truth test was completed.
  No global correction or claimed photometric accuracy follows from this work.

**Conclusion:** nonlinearity is a plausible contribution, not an established
single explanation or validated fix. The existing reference curve is applied
correctly in the tested range. Next: joint PSF/response fitting on unsubtracted
calibrated images with source and epoch holdouts, followed by temporal signal
preservation, or an independent on-orbit detector-response reference.