Retention / cleanup strategy for changelog tables in Postgres #501
Unanswered
RoderickOxen
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi 👋
We’ve been running OpenFGA in production backed by Postgres, and noticed that the changelog table keep growing indefinitely. Over time, this is consuming a significant amount of storage space. (it get's worse for cronjobs which add or remove multiple tuples in a timeframe)
From the docs and schema I understand that:
changelogHorizonOffsetonly affects the ReadChanges API (filters out very recent entries to account for replica lag).Questions
Is there an official recommendation for handling changelog retention in Postgres?
For example: should I schedule a periodic SQL
DELETEjob, use table partitioning, or is there a planned feature for built-in cleanup?What are the risks of deleting rows older than N days from
changes/tuple_changesmanually?(e.g., does OpenFGA rely on having a full changelog history for correctness, or is it only needed for clients consuming ReadChanges?)
Would you consider adding:
Motivation
Without some retention strategy, our Postgres instance is accumulating many gigabytes of changelog data, which directly impacts our cloud costs.
For example currently I have:
(Not great for the monthly cloud payment)
We’d like to keep only ~3 days of changelogs (our consumers don’t need older data), but we want to ensure this won’t break OpenFGA or best practices.
Environment
Any input is much appreciated !
Beta Was this translation helpful? Give feedback.
All reactions