Reconcile UC deduction and benefit cap components under the protected floor

[vahid-ahmadi]

Five confirmed bugs in the merged UC deductions / protected floor code, plus the doc clarifications that follow from the first fix.

1. Deductions did not reconcile when the protected floor bound

universal_credit applied the floor to the sum of benefit_cap_reduction and uc_deductions at the point of computing the award. Neither component was rewritten, so under a floor reform both reported their unfloored values.

Reproduction (floor 0.85, benefit_cap_reduction £1,000, latent rate 0.30, standard allowance £4,801.70):

	before	after
uc_deductions	£960.34	£720.25
benefit cap reduction applied to UC	£1,000.00 (benefit_cap_reduction)	£0.00 (uc_benefit_cap_reduction)
components sum	£1,960.34	£720.25
reduction actually applied	£720.25	£720.25

Fix. The floor now lives in the components rather than in the award formula:

• New uc_maximum_reduction holds the allowance the floor leaves — max((1 − floor) × standard allowance, 0), infinite when the floor is zero.
• uc_deductions is limited to that allowance, so deductions bend only where they alone exceed it.
• New uc_benefit_cap_reduction takes the headroom deductions leave: min(benefit_cap_reduction, max(allowance − uc_deductions, 0)).
• universal_credit subtracts the two components separately, so reconciliation holds by construction.

Ordering: the benefit cap reduction absorbs the floor first. Under the Fair Repayment Rate the 15% deductions cap equals the allowance an 85% floor leaves, so cappable deductions alone essentially never breach the floor — only benefit cap reductions push past it. JRF's own worked example (standard allowance £92, deduction £14, benefit cap £59, floor £78) preserves the £14 deduction and eliminates the £59 cap reduction, which is exactly this ordering. Test test_the_benefit_cap_reduction_absorbs_the_floor_first pins it.

benefit_cap_reduction stays gross. It also drives Housing Benefit, which the UC floor does not protect, so flooring it would leak a UC reform into HB. Keeping it gross is also what avoids a dependency cycle: uc_deductions and uc_has_deduction both read benefit_cap_reduction for the one-penny award test, so flooring it in place would have made benefit_cap_reduction → uc_deductions → benefit_cap_reduction. The new chain is acyclic — universal_credit → uc_benefit_cap_reduction → {benefit_cap_reduction, uc_maximum_reduction, uc_deductions} — and the full suites run clean, which is what would surface a cycle.

Residual question, deliberately not changed. The above-cap "last resort" excess (last resort and child maintenance deductions), which current law exempts from the 15% deductions cap, is still subject to the floor where deductions alone exceed the allowance. JRF's briefing does not say whether their floor exempts those categories — its worked example involves only cappable deductions and the benefit cap — so this remains a modelling choice, documented in protected_floor.yaml, in uc_deductions.py, in the validation page, and pinned by test_protected_floor_binds_on_the_above_cap_excess.

2. A floor above 1 manufactured money

(1 − floor) × standard allowance goes negative above a floor of 1, and subtracting a negative reduction added to the award: protected_floor = 1.2 on a £6,000 entitlement returned £6,960.34. The allowance now clamps at zero, where the floor protects the whole standard allowance from any reduction. Test: test_a_floor_above_the_standard_allowance_adds_nothing.

3. uc_deductions applied without a UC claim

uc_deductions lacked defined_for = "would_claim_uc". With would_claim_uc = False and an imputed uc_latent_deduction_rate, universal_credit was £0 but uc_deductions was £720.25. Harmless today because the fallback uc_has_deduction formula gates on the claim, but it breaks the documented migration path where datasets impute the latent rate directly and the fallback retires. Guard added to uc_deductions and to the new uc_benefit_cap_reduction. Test: test_no_deductions_without_a_universal_credit_claim.

4. Wrong comment in test_protected_floor_binds_on_the_above_cap_excess

It read "Latent demand of 30% is 15% cappable plus a 15% last resort excess" — 15 + 15 = 30, but the test asserts a 20% rate. The real split is 25% cappable (the calibration cap), cut to 15% by the deductions cap, plus a 5% excess. Comment only; assertions were already right.

5. Parameter provenance

• deductions/cap.yaml: the 25% cap moves from 2021-04-01 to 2021-04-30, the actual change date and the convention the file already uses for the Fair Repayment Rate.
• deductions/minimum_payable.yaml and uc_deductions.py: the one-penny rule is Schedule 6 paragraph 3(1)(a) of SI 2013/380, not Schedule 6 generally.

Validated statistics are unaffected

Microsimulation tests are gated on the dataset, so this is reasoned rather than run:

• Current law leaves every value bit-identical. protected_floor is 0, so uc_maximum_reduction is infinite, uc_deductions passes through min(D, ∞) = D, and uc_benefit_cap_reduction is min(B, ∞) = B. universal_credit becomes max(pre − B − D, 0), which is what max(pre − min(B + D, ∞), 0) already evaluated to. Confirmed numerically: the unreformed case above returns deductions £960.34, cap reduction £1,000, UC £4,039.66 both before and after.
• The defined_for guard changes nothing in the microdata. uc_deductions was already zero without a claim, because uc_deduction_rate → uc_latent_deduction_rate returns zero unless uc_has_deduction, which requires would_claim_uc. The guard only matters once datasets impute the latent rate directly.
• The cap date change has zero numeric effect. convert_to_fiscal_year_parameters samples every parameter at 30 April of each year, and cap.yaml carries no fiscal_year_blend, so 2021 reads 0.25 under either date. The validated years are 2024 and 2025 in any case.
• Nothing else moved. The latent distribution, cap values, calibration cap, region factors and uc_deduction_rate are untouched. uc_has_deduction, uc_deduction_rate and the deduction draws are unchanged, so incidence, at-cap shares and mean amounts are unchanged.

The one doc-side change is a label clarification: the validation table's at-cap and above-cap rows are shares of all UC households, which is what the model computes (w[on_uc & …].sum() / uc_weight in test_uc_deductions_aggregates.py) and what DWP publishes. Figures are unchanged.

Tests

• policyengine_uk/tests/test_uc_deductions.py: 17 passed (4 new).
• policyengine_uk/tests/ -m "not microsimulation" (excluding tests/policy): 153 passed, 15 skipped.
• policyengine-core test policyengine_uk/tests/policy: 1118 passed.

🤖 Generated with Claude Code

[vahid-ahmadi]

Verified the five fixes independently against the branch — all confirmed:

• Current law bit-identical: unreformed repro returns uc_deductions=960.34, universal_credit=4,039.66, unchanged.
• Reconciliation holds: floor 0.85 with gross cap reduction £1,000 gives 720.25 + 0.00 = 720.25 applied.
• Floor > 1 clamped: returns £6,000, not £6,960.34.
• No-claim guard: uc_deductions = 0.00 (was £720.25).
• JRF worked example reproduces: SA £92.34/wk against their £92, allowance £13.85 against their £14, deduction survives whole, cap reduction absorbed to zero. That is a good independent check of the absorption ordering.

One residual worth documenting (not blocking)

uc_deductions still derives its available award from the gross benefit_cap_reduction, so where the gross reduction nearly exhausts the award the one-penny constraint bites before the floor allowance does, and the split moves:

floor 0.85, allowance = 15% of SA = 720.25, entitlement 6,000

  gross bcr    deductions   cap applied      UC     total reduction
      1,000        720.25          0.00   5,279.75         720.25
      4,000        720.25          0.00   5,279.75         720.25
      5,500        499.88        220.37   5,279.75         720.25
      5,900         99.88        620.37   5,279.75         720.25

The award is correct in every row and the components reconcile — this is strictly an attribution issue between the two components, not a UC error, so it is much smaller than the bug this PR fixes. But under the absorption ordering the deduction should survive whole at 720.25 throughout, and in the last two rows it does not.

It bites only when entitlement − gross benefit cap reduction < (1 − floor) × standard allowance, which is rare — but it is exactly the heavily-capped population that JRF-style floor analysis targets, and it would understate reported deductions there. A fix would compute the award in uc_deductions from the floor-limited cap reduction, though that risks reintroducing the cycle this PR carefully avoided.

Suggest either fixing it or noting it in the docs alongside the existing last-resort caveat.

[vahid-ahmadi]

@MaxGhenis review request — this fixes bugs in the deductions work merged in #1815.

The judgement call is the absorption ordering. The floor limits combined reductions; something has to give first. This PR makes the benefit cap reduction absorb the floor while the deduction survives, on the reasoning that under the FRR the deductions cap (15%) equals a typical floor allowance, so cappable deductions alone essentially never breach it. JRF's worked example supports this — their floor allowance is GBP 14/week and their deduction is GBP 14, so it survives whole and the GBP 59 cap reduction goes. The model reproduces that exactly (SA GBP 92.34/wk vs their GBP 92, allowance GBP 13.85 vs their GBP 14).

If you read their design differently — proportional abatement, say — that changes the split though not the award.

Three things worth your eye:

1. benefit_cap_reduction is deliberately left gross. Flooring it in place would leak a UC reform into Housing Benefit, which the floor does not protect, and would create a cycle through the one-penny test. The new uc_benefit_cap_reduction is UC-only.
2. A residual I found and documented (comment above): where the gross cap reduction nearly exhausts the award, the penny constraint bites before the floor allowance and the split moves, so a deduction that should survive whole does not. The award is right in every case and components reconcile — it is attribution only — but it affects exactly the heavily-capped population JRF-style analysis targets. Fix-or-document, your call.
3. Whether the above-cap last-resort excess should be exempt from the floor. Unchanged behaviour here, documented in the parameter, the variable and the docs. JRF's briefing does not settle it.

Current law is bit-identical (verified numerically), and the gated Test job passes, so the validated statistics in #1815 are intact.