# Causal injection result: erased neighbours are recorded but omitted from the fitting scenes

2026-10-02. Status: completed across F1, F2, S22, C1 and C4. This identifies a real contamination mechanism and a concrete scene-construction omission. It does not assign every part of the earlier unmatched/matched population contrast to that mechanism.

## Headline / Conclusion

The near-star loss occurs in **PS1 connected-footprint removal**. Known faint sources remain detectable immediately before that step, then are erased with the bright star’s footprint. Their light would still be in the TESS science image and therefore remain in hp_d.

For catalogued neighbours, the pipeline writes removal records, but the actual five `scene_boot` bundles stop at TESS magnitude 13. They include **none** of the fainter `catalog_neighbor` sources in the audited science regions. The scene builder swaps new hp_d pixels into the old source geometry without appending those removed sources. This is the missing handoff.

Five complete, isolated one-source experiments confirm the consequence: omitting a known erased T17 neighbour shifts the bright-star fitted flux upward by 17–20 electron/s. Adding the neighbour to the same fixed-PSF fit reduces that shift to at most 0.043 electron/s, including every collateral change to the original PS1 pixels. This is controlled linear photometry, not a full ePSF/WCS retraining.

**Recommended next fix:** keep the bright training/contributor selection, but append the actually removed Gaia neighbours as nuisance sources when building scenes, with footprint/mask checks. Re-export the scenes and validate an old/new ePSF/WCS fit on held-out data before changing PSF model terms. Do not fill the template depression with an inferred flat pedestal.

## Experiment and provenance

- Run: `/astro/armin/koji/syndiff/dev_runs/epsf_stage_trace_20261002/injection/`.
- Code branch: `codex/ps1-stage-trace` in the nested dev repository; scientific code checkpoints `ceb6d56` and `bc5b4ca`, plus final report commit. Worktree `/home/kshukawa/syndiff_pipeline/dev/.claude/worktrees/codex-ps1-stage-trace`.
- Original processing code: `paper-pin` commit `648c0bcf4b559691177f7b4764f721ac54aebdc8`, footprint_v1 recipe `e17a198a4942aa2d`. Band weights r/i/z/y = 0.254/0.4368/0.1654/0.1438.
- Frozen input identities are in the preceding `multifield/<field>/inputs.json`. No production source, catalogue, template, fitting scene or parameters were changed.
- Final output locations: `injection/C4/`; `injection/resumed/{F1,F2,S22,C1}/`. Original interrupted checkpoints remain preserved.
- 30 real bright-star host configurations: F1 6, F2 8, S22 5, C1 3, C4 8. Three nominal injected magnitudes (15,17,19), five separations (0.35,0.7,1.2,2,3.5 nominal TESS pixels), one near and one clean-control injection per host per arm: **900 injections in 75 arms**. These are repeated controlled configurations, not 900 independent sky fields.
- F1/F2/C1/C4 use their already verified centre skycells. S22’s centre had only two geometrically eligible hosts before any response was measured; its preparation is preserved in `S22_centre_preparation/`. The next preselected skycell, 2437.022, supplies five hosts under the unchanged gates.
- Source profiles are compact empirical PS1 profiles estimated from isolated retained real stars. Per-cell scaling maps catalogue-equivalent magnitude to the known injected PS1 flux. Magnitude labels are approximate catalogue equivalents, not a claim of an exact TESS colour transformation.
- Injections enter the **linearized, band-combined raw PS1 image before segmentation**; the linear band combination was already verified. Original uncertainty is fixed and the same noise realization is used on both sides of each pair. No extra stochastic photon noise is added. Original missing pixels remain missing.
- One nominal TESS pixel is the declared 81.4 PS1-pixel radius scale. Actual WCS separations are saved; nominal 0.7 spans approximately 0.683–0.752 native TESS pixel over the five fields.

Commands, after activating the syndiff environment, with BLAS/OMP/Numba thread counts set to 1:

```bash
python dev/.claude/worktrees/codex-ps1-stage-trace/forward_epsf_wcs/diagnostics/ps1_injection.py --field C4
python dev/.claude/worktrees/codex-ps1-stage-trace/forward_epsf_wcs/diagnostics/ps1_injection_transport.py --field C4
python dev/.claude/worktrees/codex-ps1-stage-trace/forward_epsf_wcs/diagnostics/ps1_injection_single.py --field C4
# Other fields used the same commands; S22 initially adds --cell skycell.2437.022.
# Checkpoint resumes:
python dev/.claude/worktrees/codex-ps1-stage-trace/forward_epsf_wcs/diagnostics/ps1_injection.py --field F1 --resume --out-root /astro/armin/koji/syndiff/dev_runs/epsf_stage_trace_20261002/injection/resumed
# Repeat resume for F2, C1, S22. Transport/single commands use matching --input-root.
python dev/.claude/worktrees/codex-ps1-stage-trace/forward_epsf_wcs/diagnostics/ps1_removed_scene_audit.py
python dev/.claude/worktrees/codex-ps1-stage-trace/forward_epsf_wcs/diagnostics/ps1_injection_report.py
```

Exact launcher scripts are in `injection/condor/`; logs are in `injection/logs/`. All fields have `DONE.json`, `TRANSPORT_DONE.json`, and `tables/single_source.json` in their final output directories.

## Method: photon tagging versus collateral scene changes

For finite pixels, the exact removal-stage difference can be decomposed as

$$Z(I+S)-Z(I)=M_+S+(M_+-M_0)I.$$

- $I$: original linearized PS1 image, PS1-combination flux units per PS1 pixel; actual processing preserves its unavailable pixels, and missing values are treated as zero only for the output bookkeeping.
- $S$: realized added signal on available pixels, in the same units; float32 rounding and originally unavailable incident photons are recorded separately.
- $Z$: pinned detection/segmentation and connected-footprint removal operator.
- $M_+$ and $M_0$: binary retention indicators produced by that operator with and without the injected source. They are measured from the actual masks/output, not fitted parameters.
- $M_+S$: tagged injected photons that survive.
- $(M_+-M_0)I$: original scene light whose classification changes because of the injection—collateral changes.

The grid measures both terms. Net collateral changes range from about −0.7% to +7.7% of the arm’s incident flux, so they must not be silently counted as injected photons or ignored in a recovery claim. The final isolated-source tests rerun **one source only** and propagate every changed pixel, including those outside the injected PSF support.

For the science-image perturbation used in the paired recovery,

$$\Delta d=L_0(S_{\rm ideal})-L_M\!\left[Z(I+S)-Z(I)\right].$$

- $S_{\rm ideal}$: full known incident source packet before original PS1 missing-data losses, in PS1-combination flux units per pixel.
- $L_0$: the saved Gaussian convolution, frozen PS1-to-TESS binning and saved Hotpants kernel, with no PS1 bad-pixel suppression in the ideal source reference. It maps to electron/s per native TESS pixel.
- $L_M$: the same frozen operators with the actual bit-12 suppression used by template construction.
- $\Delta d$: change left in hp_d when the same ideal faint source is added to the science side and its actual processed contribution is subtracted on the template side, electron/s per native pixel. The saved constant background cancels between the pair.

This is applied to the same actual hp_d baseline. Compare a bright-source PSF plus free background plane against the same model with an extra faint-source PSF. Positions and PSFs remain fixed; weights are flat. The difference of before/after fitted bright flux isolates the injected contamination. Full ePSF/WCS optimization and production weighting are not being claimed as reproduced here.

## Where the known photons are lost

For the nominal 0.7-pixel injections, pooled over 30 host configurations:

| Injected magnitude | Near bright star: after detection | Near bright star: after footprint removal | Clean control: after removal | Injected removal records |
|---|---:|---:|---:|---:|
| T15, catalogued | 99.999% | 0% | 99.994% | 30/30 near sources |
| T17, catalogued | 99.783% | 0% | 98.931% | 30/30 near sources |
| T19, deliberately untracked | 97.629% | 0.019% | 91.174% | none, by design |

At both nominal 0.35 and 0.7 pixel, all 120 catalogued T15/T17 near injections lose their tagged photons in the final footprint output, and all are recorded. At larger separations most survive; the transition depends on the host’s footprint. This directly locates the dominant near-star loss in removal, before the Gaussian/TESS-grid operators.

The isolated-control losses are a different effect: thresholding discards faint wings (about 1.1% of tagged T17 light and 8.8% of tagged T19 light for these profiles/noise maps). These are **tagged-photon fractions**, not universal net photometric corrections: collateral scene/noise reclassification also changes the paired output.

The untracked arm is an explicit counterexample to “no removal record means no light can be erased.” T19 was deliberately left out of the synthetic catalogue; this does not assume every real T19 object is absent from Gaia. In each field, a tracked/untracked repeat at T17 and nominal 1.2 pixel produced identical image pixels: faint catalogue membership changes the accounting, not the removal image.

![Photon survival by stage and radius](/astro/armin/koji/syndiff/dev_runs/epsf_stage_trace_20261002/injection/figs/photon_budget.png)
![Measured signal images behind the photon budget](/astro/armin/koji/syndiff/dev_runs/epsf_stage_trace_20261002/injection/figs/photon_images.png)

## The real fitting scenes omit the recorded neighbours

The audit compared the actual source IDs, not merely config names or role counts. `scene_boot/scene_meta.json` has `tmag_max=13` in all five fields and every source in the corresponding `scene_bundle.npz` is at or brighter than that cutoff. Its existing nuisance stars are demoted members of that source set, not the fainter removed neighbours.

| Field | Recorded faint neighbours in audited science region | Present in fitting scene | Omitted centres on valid fitted pixels |
|---|---:|---:|---:|
| F1 | 5,990 | 0 | 3,952 |
| F2 | 5,708 | 0 | 3,185 |
| S22 | 4,224 | 0 | 2,659 |
| C1 | 3,584 | 0 | 2,478 |
| C4 | 10,288 | 0 | 6,700 |

There are **18,974 omitted neighbour centres on valid fitted pixels** across the five fields. “Science region” here excludes a 20-pixel edge and requires a match to the field catalogue; support counts refer to the rounded centre, not the entire PSF. They are bookkeeping counts, not measurements of every neighbour’s surviving flux.

The source of the mismatch is visible in [build_scene](/home/kshukawa/syndiff_pipeline/.claude/worktrees/paper-pin-chain/syndiff_pipeline/forward_model/chain/scene.py:220): it loads the source scene, swaps pixels/demotes stars/masks, and saves without appending newly removed Gaia neighbours. [swap_data](/home/kshukawa/syndiff_pipeline/.claude/worktrees/paper-pin-chain/syndiff_pipeline/forward_model/chain/scene.py:54) explicitly retains source geometry. Template removal and scene construction therefore disagree about which sources remain in hp_d.

![Actual omitted-source positions on template and fitting-input images](/astro/armin/koji/syndiff/dev_runs/epsf_stage_trace_20261002/injection/figs/scene_omission_examples.png)

The markers identify actual catalogue membership; they do not claim the faint star is separately resolved in the TESS image. Full per-source audit CSVs are in `injection/tables/<field>_removed_scene_audit.csv`.

## Complete one-source causal recovery, including collateral changes

A deterministic middle host in each field received one T17 source at nominal separation 0.7 pixel. The source was recorded as `catalog_neighbor` in all five. The known TESS-side added signal is 24.4–25.4 electron/s per case, defined by the actual saved forward operator.

| Field | Bright-flux error without neighbour (electron/s) | With neighbour (electron/s) |
|---|---:|---:|
| F1 | +19.8201 | −0.000000009 |
| F2 | +19.3121 | −0.000000381 |
| S22 | +19.7817 | +0.02125 |
| C1 | +17.3911 | −0.000000225 |
| C4 | +17.6223 | +0.04284 |

Including the faint neighbour removes at least **99.7% of this controlled bright-flux error** in every field. The tiny residuals in S22/C4 include collateral changes in real surrounding pixels. These are deterministic paired errors, not population estimates with statistical error bars. The common PSFs make this a conditional recovery test; it is not an independent validation of the real ePSF shape or a production retrain.

![Original PS1 contexts and actual injected-source removal](/astro/armin/koji/syndiff/dev_runs/epsf_stage_trace_20261002/injection/figs/injected_ps1_contexts.png)
![Complete paired recovery in all five fields](/astro/armin/koji/syndiff/dev_runs/epsf_stage_trace_20261002/injection/figs/causal_recovery_five_fields.png)

The omitted-source residual is visibly asymmetric/dipolar in individual stamps. A width or flux scalar alone would conceal that morphology. The known source model removes that feature.

## Nulls, errors found, and limits

- Baseline full removal and segmented images reproduce the published products exactly before injecting. The new S22 cell passes the same gate.
- Catalogue-on/off repeats give identical pixels in all five fields, proving that recording is not restoration or modelling.
- Gaussian and unmasked-binning flux checks agree at better than 8.4e-9 fractional error; no injected flux maps outside the tested assignment domain. Actual bit-12 suppression is measured separately, reaching 1.70% of an individual packet in the sampled cases.
- The first transport gate incorrectly assumed that equal local kernel sums guarantee point-source flux conservation for a spatially varying kernel. They do not: row sums and column sums differ across kernel blocks. A fixed-local-kernel null closes within 2.6e-8; the actual spatial point gain differs from the DC approximation by at most 0.021–0.065% across fields. Actual propagated ideal injections are used as the reference. The test definition was corrected and independently checked, not relaxed to pass the old assumption.
- Tagged photons alone do not include collateral changes; this is why the complete isolated-source reruns were added. No claim of exact recovery was based only on the tagged component.
- S22’s initial two-host preparation failed the sample-size precheck before responses were measured. It was preserved and the next preselected cell used with unchanged geometry gates.
- Execution moved from plscience4 to plscience3 and local tool handles were lost. Original partial outputs were preserved; resumes used separate copied directories. Initial Condor resumes failed before scientific execution because `MAMBA_EXE` was unset under `set -u`. Locally activated, logged resumes completed. One interrupted final transport was rerun from saved packets. All final science completion/gate files were checked; scheduler exit status alone was never accepted as proof.
- No production fix or ePSF/WCS training was performed. The experiment covers these five scenes, fixed kernels and sampled source profiles/locations. It does not certify the separate v3 rebuild or overlap seams.

![Spatial-kernel versus fixed-kernel normalization null](/astro/armin/koji/syndiff/dev_runs/epsf_stage_trace_20261002/injection/figs/kernel_normalization_null.png)

## Relation to the earlier “hole” claims

The unmatched star-versus-control contrast remains a mixture of source-population selection and real missing template light; its failure to correlate with catalogue-neighbour sums was not a valid reason to rule out neighbour contamination. The present experiment supplies the missing causal test and confirms a real scene-list omission.

**Do not equate the roughly −10 to −20 electron/s earlier population contrast with the +17 to +20 electron/s errors in these chosen injections.** They are different estimators, denominators and populations. Known-neighbour omission cannot directly account for all of a strictly zero-recorded-neighbour subset. Untracked light and detection/mask losses can contribute there, but their real-sky abundance and share of that remaining contrast have not been measured by this test.

The earlier suggestion of very large global T16–17 throughput loss is not supported by these clean-control injections, which retain about 99% of tagged T17 light. That prior isolated-aperture result still needs an estimator/population audit; it is not a reason to add a global template pedestal.

## Records and next action

Canonical summary: `injection/tables/summary.json`; photon budgets: `photon_summary.csv` and `collateral_budget.csv`; actual scene audit: `scene_audit_summary.json`; complete recovery: each field's `tables/single_source.json`, `single_source_ps1.npz`, `single_source_tess.npz`. All figures above were viewed before captioning. Code copies and provenance are under `injection/code/`.

The shared local canvas is updated in place at `/home/kshukawa/syndiff_pipeline/doc/ps1_stage_trace_canvas/index.html`. This session lacks the Claude Artifact publishing and SendUserFile tools, so the remote Claude URL is not republished.

**Final conclusion:** the near-star photon loss is localized to footprint removal, and the recorded-neighbour population is missing from the actual fitting scenes. Paired recovery proves that this omission can bias bright-star photometry and that modelling the erased neighbour removes the controlled error. Fix that scene-input mismatch first; then measure the improvement on real held-out data. The exact decomposition of every earlier population-stack residual remains unclaimed.