Skip to content

Conversation

@ValueRaider
Copy link
Collaborator

@ValueRaider ValueRaider commented Jul 22, 2023

Several improvements to price repair

Repair 100x and split errors:

  • Handle stocks that recently suspended - use latest ACTIVE trading as baseline
  • Improve error identification:
    • Restrict repair to no older than 1 year before oldest split
    • To reduce false positives when checking for multiday split errors, only analyse 'Open' and 'Close' and use average change instead of nearest-to-1
    • For weekly intervals reduce threshold to 3x standard deviation (5x was too high), and for monthly increase to 6x
    • For multiday intervals, if errors only detected in 1 column then assume false positive => ignore

Repair missing div-adjust:

  • Fix repair of multiday intervals containing dividend

Price reconstruction:

  • Move to after repairing 100x and split errors, so calibration works properly
  • Fix maximum depth and reduce to 1
  • Restrict calibration to 'Open' and 'Close', because 'Low' and 'High' can differ significantly between e.g. 1d and day-of-1h

Miscellaneous:

  • Deprecate repair='silent', the logging module handles this
  • Improve tests for 100x and split errors
  • New test for 'repair missing div adjust'

Status: I've found a bug in this, need to commit fix but I'm busy at moment ... testing

Several improvements to price repair

Repair 100x and split errors:
- Handle stocks that recently suspended - use latest ACTIVE trading as baseline
- Improve error identification:
  - Restrict repair to no older than 1 year before oldest split
  - To reduce false positives when checking for multiday split errors,
    only analyse 'Open' and 'Close' and use average change instead of nearest-to-1
  - For weekly intervals reduce threshold to 3x standard deviation (5x was too high),
    and for monthly increase to 6x
  - For multiday intervas, if errors only detected in 1 column then assume false positive => ignore

Repair missing div-adjust:
- Fix repair of multiday intervals containing dividend

Price reconstruction:
- Move to after repairing 100x and split errors, so calibration works properly
- Fix maximum depth and reduce to 1
- Restrict calibration to 'Open' and 'Close', because 'Low' and 'High' can differ significantly between e.g. 1d and day-of-1h

Miscellaneous:
- Deprecate repair='silent', the logging module handles this
- Improve tests for 100x and split errors
- New test for 'repair missing div adjust'
Improve bad-split-repair on multiday intervals
Switch some repair log msgs from warning -> info
@ValueRaider ValueRaider merged commit 6c1e260 into dev Aug 1, 2023
@ValueRaider ValueRaider deleted the feature/price-repair-improvements branch August 1, 2023 19:03
@ValueRaider ValueRaider mentioned this pull request Aug 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants