Skip to content

fix(FR/IT): normalise state.level field for region/department hierarchy#1489

Merged
dr5hn merged 1 commit into
masterfrom
fix/states-level-normalise
Apr 27, 2026
Merged

fix(FR/IT): normalise state.level field for region/department hierarchy#1489
dr5hn merged 1 commit into
masterfrom
fix/states-level-normalise

Conversation

@dr5hn
Copy link
Copy Markdown
Owner

@dr5hn dr5hn commented Apr 27, 2026

Summary

Customer-facing follow-up to #1349 / #1352. Cities are already correctly parented onto departments (FR) and provinces (IT) via the recently merged remap PRs (#1395, #1394, #1393, #1400, #1484), but the state rows themselves still carried inconsistent level values. This PR normalises the field so downstream consumers can reliably filter level=1 (region) vs level=2 (department/province).

Driven by bin/scripts/fixes/states_level_normalise.py (idempotent, only touches level, only on FR/IT).

Before / After

France (FR)

level Before After
null 29 0
1 0 29
2 95 95

29 rows null → 1: 13 metropolitan regions, 2 metropolitan collectivities with special status (Paris, Métropole de Lyon), 1 European collectivity (Alsace), 5 overseas regions, 5 overseas collectivities, 1 overseas collectivity with special status (Nouvelle-Calédonie), 1 overseas territory (TAAF), 1 dependency (Clipperton). 95 metropolitan departments unchanged at 2.

Italy (IT)

level Before After
null 1 0
1 115 20
2 4 106
3 6 0

103 rows updated. Final state: level=1 is region (15) + autonomous region (5) = 20. level=2 is province (80) + metropolitan city (14) + free municipal consortium (6) + decentralized regional entity (4) + autonomous province (2) = 106.

Validation

  • Schema unchanged: only the level field was rewritten on 132 FR/IT rows; every other field byte-for-byte intact.
  • Cross-reference: country_id ↔ country_code consistent for all 250 FR/IT states; no duplicate ids introduced.
  • Idempotent: re-running the script reports zero changes.
  • Parent-id integrity check ran post-normalisation. After the fix, every FR metropolitan department (level=2) has a parent_id resolving to a level=1 FR row, and every IT province / metropolitan city / free municipal consortium / decentralized regional entity / autonomous province (level=2) has a parent_id resolving to a level=1 IT row — with one pre-existing exception unrelated to this PR: 8 Tuscany provinces (Pisa, Pistoia, Prato, Siena, Livorno, Lucca, Massa and Carrara, Grosseto, ids 1685/1687/1700/1734/1745/1749/1759/1787) carry parent_id=1764 ("Udine", a Friuli decentralized regional entity) instead of 1664 ("Tuscany"). This is a stale parent_id from feat(IT): remap cities to metropolitan cities and provinces (#1349) #1395 and is orthogonal to level normalisation; flagging here for a follow-up cleanup.

Notes

Customer-facing follow-up to #1349 (Italy) and #1352 (France). Cities
were re-parented onto departments (FR) and provinces (IT) by #1395 /
#1394 / #1393 / #1400 / #1484, but the state records themselves still
carried inconsistent 'level' values, blocking downstream filters like
"all departments == level=2" or "all regions == level=1".

bin/scripts/fixes/states_level_normalise.py drives the change:
  - FR: 29 region-tier rows None -> 1 (13 metro regions, 3 special
        metro collectivities incl. Corse + Alsace + Métropole de Lyon,
        13 overseas regions/collectivities/territories/dependency).
        95 metropolitan departments unchanged at level=2.
  - IT: 103 rows updated. Final state: 20 at level=1
        (15 region + 5 autonomous region) and 106 at level=2
        (80 province + 14 metropolitan city + 6 free municipal
        consortium + 4 decentralized regional entity + 2 autonomous
        province).

Only the 'level' field is touched; idempotent on re-run; non-FR/IT
states untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 27, 2026 15:58
@dosubot dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Apr 27, 2026
@dr5hn dr5hn merged commit b377016 into master Apr 27, 2026
2 of 3 checks passed
@dr5hn dr5hn deleted the fix/states-level-normalise branch April 27, 2026 15:59
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Normalises state.level for France and Italy so consumers can reliably distinguish region-tier (level=1) vs department/province-tier (level=2) states, aligning state metadata with the already-correct city parenting from prior remap PRs.

Changes:

  • Rewrites inconsistent/missing level values on FR/IT state records in contributions/states/states.json to a consistent 1 (region tier) or 2 (department/province tier).
  • Adds an idempotent fix script (bin/scripts/fixes/states_level_normalise.py) to apply and validate the normalisation (including a parent-id integrity diagnostic).

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.

File Description
contributions/states/states.json Normalises FR/IT level values (regions/collectivities → 1, departments/provinces → 2).
bin/scripts/fixes/states_level_normalise.py Adds a repeatable script to compute/apply the level normalisation and report unmapped types / parent-id issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data:states large-contribution size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants