WIP: Speed up queries via materialized view

[Muhammad-Moiz626]

Description

Our data model with multiple beliefs per event time (multiple horizons as well as sources) makes queries computationally expensive. Most-used are queries where we look only for the most recent beliefs about each event, when given any moment in time as max belief time.

Pre-computation with Postgres' materialized views can make queries much faster. This PR will enable hosts to use this performance enhancement.

• Created migrations file to create materialized view.
• Use the MV in queries.
• Use SQL Alchemy for MV.
• Automate the refreshing of the MV.
• Make MV optional / configurable.
• Document the process

...

Further Improvements

• the full queries (used in playbacks, getting fulls set of beliefs for each event) might also enjoy a materialized view, but maybe it is just too much data, then we should just limit playback to shorter time ranges
• give hosts a range of options (e.g. refresh time)
• actually move to TimeScaleDB, which auto-refreshes their materialized views on inserts

Related Items

Closes 108

...

---

Sign-off

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures

[Flix6x]

Some early review comments. Nice to see the refresh logic taking shape.

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #29345424 | 📁 Comparing e3277eb against latest (81d507f)

🔍 Preview build

Show files changed (5 files in total): 📝 5 modified | ➕ 0 added | ➖ 0 deleted

File	Status
configuration.html	📝 modified
genindex.html	📝 modified
_autosummary/flexmeasures.data.models.generic_assets.html	📝 modified
_autosummary/flexmeasures.data.models.time_series.html	📝 modified
_autosummary/flexmeasures.utils.validation_utils.html	📝 modified