idleon-automation

Hoops findings — the June 2026 miss investigation (#37)

Settled conclusions from the 2026-06-09/10 investigation into why the hoops bot’s make rate was low (~25%) and inconsistent. Kept here so the “why” survives outside the GitHub thread; the per-session evidence lives in issue #37’s comments. Follow-up model work: issue #38.

The headline reframe: misses are not aim error

Shots whose ball arrives within ±120px of the hoop make at ~64% in the make_prob model-aim regime (model in-band ~58%); the pooled in-band rate across all eras and aim sources is only ~34% (n≈1.2k), because explore and legacy shots arrive in-band and still clank. Arrival proximity is necessary but not sufficient — the predictor is the lever, not the ±120px window. Clean makes cluster within a ~30px standard deviation (the broader made population is wider, scattered by lucky bounce-ins). The bot does not lose to near-misses — it loses to two categorically different failure modes that both masqueraded as “way short” in the data:

• Structure clanks. The ball reaches the hoop (ball_peak_x within ~25px of hoop_x), clips the rim front / pole, and bounces backward — and ball_x_at_rim_height then measures the post-bounce descent, recording a 100–350px “short miss”. Most of the historical way-short population was this. Any aim analysis must use the bounce-aware arrival (_shot_arrival_x: peak_x when backward drift exceeds MAX_BACK_DRIFT_PX, landing otherwise).
• Genuine over/undershoots, controlled by platform velocity at launch (below).

Velocity controls the launch — through arc shape, not just range

The ball inherits the platform’s vertical velocity at click time. platform_vy (least-squares slope over the ~1.5s of platform samples before the click; positive = downward) orders same-hoop outcomes monotonically. Notes on interpretation:

• Near a bob turnaround the trailing window mixes the previous half- cycle, so window-vy is best read as a phase proxy: strongly positive on a dir=up fire means “just after the bottom turnaround”.
• dir=up: slow launch (turnaround fire) → flat low arc (apex_y ~138) that reaches near hoops but clips the structure; fast launch (mid-swing) → high arc (apex ~81) with much more range.
• dir=down: slower descent at fire → longer flight (vy 28 → peak 887; vy 70–110 → peak 508–602). Monotonic and, unlike up, without a cliff — learnable structure.

The main-loop sample cadence is ~200–1000ms per tick (not the 15–30ms the POLL_INTERVAL comment suggests) — anything derived from the sample buffer must tolerate that.

Direction is a per-hoop policy

_required_direction_for(hoop_y, hoop_x); one predictor fitted per direction (training rows are direction-filtered).

• Very low hoops (y ≥ 530): dir=up is futile, dir=down works. A 56-shot exploration sweep at (691,550) covered the full bob range and every dir=up shot landed ≥68px past the hoop; the only reach-matching launch state is the bob-bottom flat arc, which clanks. The first dir=down shot at that hoop made (after 110+ dir=up misses).
• The near “clank band” (x ≤ 640): bracketing confirmed. dir=up arrives on target and clanks the rim front from every launch state (2/36 instrumented); dir=down undershoots by 50–110px with 2/12 makes while still cold-starting. The band boundary is genuinely 2-D — (659,350) clanks with up, (658,384) made with up — which is the kind of structure issue #38’s model should eventually learn.
• Mid/far hoops (x > 640): dir=up makes at ~80% in the instrumented slice. Untouched.

Make detection is multi-signal; OCR alone loses ~10% of makes

Post-shot score OCR drops out often enough that two safety nets exist:

• Hoop-respawn signal (made_source='respawn'): the hoop only teleports on a make, so a >30px hoop move after an unconfirmed shot retro-corrects the row. Guards: disabled at score ≥10 (drift moves the hoop by design), rim conf ≥0.9 required, clean_make re-classified from the shot’s own trajectory.
• Prompt-anchored score: while the “Make a shot to start” prompt is visible the score is definitionally 0 — the session_score anchor resets and pre-shot OCR reads are ignored (a single misread “7” once disabled make detection for an entire game).
• clean_make is never bypassed. Prompt confirmation proves a shot scored, not that the aim was right; lucky pole-tip bounce-ins count on the scoreboard but stay out of predictor training (the old prompt exemption was a loophole, removed).

Perturbation sweep: small, capped, hold-aware, cycling

• Capped at ±32px after σ scaling: every historical perturbed shot beyond ±32 landed >120px off with zero makes; the old ±240px swings were a miss spiral (perturbed shots made 14% vs 37% unperturbed).
• In-band misses (bounce-aware arrival within 60px) re-fire the same target — but at most twice in a row (MAX_CONSECUTIVE_HOLDS): at clank-prone hoops every arrival reads in-band and a hold loop once ran 54 shots.
• On exhaustion the sweep cycles instead of pinning at the last entry (pinning fired 25 identical shots whose outcome was pure phase luck).
• Residual-proportional correction (aim off by the measured miss distance) was considered and dropped: landing does not respond coherently to offset in exactly the regions where correction would activate.

Free-shot exploration

While the start prompt is up, misses cost no lives. The first shot at a fresh hoop stays on predictor aim; after it misses, targets are sampled uniformly across the observed bob range (target_source='explore'). One 56-shot grind produced a complete dir=up landing map of a hoop — the single most informative session of the investigation.

Training-data hygiene

Trajectory predictors train on bounce-aware arrival, not raw landing (fetch_clean_trajectories): floor-bounced shorts keep their row with peak_x as the honest reach; hoop clanks are dropped as reach-censored. The legacy launcher-distance filter both discarded the short-mislaunch population (the velocity signal) and admitted bogus post-bounce landings.

Schema columns added during the investigation: platform_vy, made_source, prompt_up, target_source.

Outcome

The measurement gate passed on 2026-06-10: 132 post-policy attempts (exploration excluded), 46 makes — 34.8%, Wilson 95% CI [0.273, 0.433], lower bound clear of the 25.5% baseline. #37 closed with that verdict.

Open at time of writing

• Measurement gate for closing #37 — passed (above).
• Make-probability model: issue #38.
• Loop tick rate (200–1000ms vs documented 15–30ms) unexplained — explained 2026-06-10 by profiling (scripts/profile_hoops_loop.py): the tick is compute-bound on per-poll template matching, same as the darts loop. Steady state was ~179ms (find_game_over 112ms full-frame every tick + find_platform 54ms + grab 13ms); between shots +63ms find_rim; pre-game +104ms find_game_prompt — stacking to the observed 200–1000ms. See TODO.md “Poll-loop tick rate” for the fixes.

Post-investigation addendum (2026-06-10): fire-latency regression

Every shot from 2026-05-09 (cc42529) to 2026-06-10 fired ~100ms after its fire decision: the start-prompt template match (104ms, full-frame) sat between the crossing detector and the click — re-introduced five days after the original click-latency fix. Launch y therefore trailed the sampled platform_y by roughly platform_vy * 0.1s (tens of px on fast bobs), which is the same mechanism as the velocity-driven over/undershoots documented above and inflated their magnitude during the whole #37 window. Fixed by computing the prompt match after the click from the already-held frame (identical result, zero latency). Interpretation caveats: rows before/after the fix embed systematically different decision→launch latencies, so velocity-coupled effects fitted on pre-fix data (and the predictors trained on it) will run slightly hot until post-fix sessions accumulate.

The back-rim notch (2026-06-26, #97)

The user observed the bot clanking the back of the rim — the ball reaches just past the hoop, bounces backward and falls out of the play area. Diagnosis against shots.db (n=840 clean shots) reframed it:

• Not a one-sided overshoot bias. Model-regime median bounce-aware arrival residual is ~−14px (slightly short). A naive “aim shorter” would trade back-rim clanks for 5%-make airballs (residual <−50).
• It’s a make-rate notch. Bounce-aware arrival residual in the back-rim zone makes ~0.40 vs ~0.78 swish (residual [−50,−15]) and ~0.79 far-bank (residual >120, off-backboard banks) — a notch flanked by higher make-rates on both sides, so the fix carves it out rather than shifting aim. Wilson CIs disjoint.
• Root cause is the backward bounce, not the landing. Within the notch, clean rim-drops (peak−landing ≤ 50) still make ~0.88; the low aggregate is entirely back-rim clanks (peak−landing > 50, peak past center) at ~0.24.
• The make_prob argmax (#38) can’t be improved here. Refitting the model and scoring each shot under its own state: notch-MISS states already score P(make) median 0.071 (vs 0.644 for notch-makes), so a P(make) penalty changes no pick — structurally inert. Arrival- prediction filters are unsound (reach-censored within 80px of the hoop). The lever is the miss-recovery path, keyed on classification of a fired shot’s trajectory.

Fix (make_prob argmax byte-for-byte untouched):

1. Asymmetric sweep gate (_sweep_should_advance): advance off a back-rim overshoot (residual > BACK_RIM_LO_PX=15) instead of the old symmetric |resid|≤60 hold, which re-fired back-rim clanks as “almost made” (304/604 in-band shots were back-rim clanks). Short/center side keeps the generous hold.
2. Directional recovery nudge, shipped as a logged 50/50 A/B (back_rim_recovery column, arm by shot-index parity à la darts #48): after a _is_back_rim_bounceout (backward bounce AND peak 15..110px past center — direction-safe, excludes front-rim clanks needing more reach and far-bank banks that score), force the next shot’s perturbation to BACK_RIM_NUDGE_PX=−16. Validate nudge-vs-control on recovery-shot make rate / recurrence (Wilson CIs) before flipping the nudge to default; the #38 model-pick make rate must not regress.

Open: preventive first-shot avoidance via a candidate prefilter classifier (#98, the only route that can reach the deep front-side notch slice the nudge can’t safely touch); lives_diff is unusable so the cost asymmetry of a back-rim miss can’t be measured from the DB (#99) — the user confirmed in-game it’s just one miss, so #97 optimizes make-rate only.