persistence: add wal and snapshot sections

xuniq · xuniq · commit 0031251571e7 · 2024-02-19T11:29:29.000+03:00
diff --git a/doc/enterprise/wal_extensions.rst b/doc/enterprise/wal_extensions.rst
@@ -8,13 +8,12 @@ For example, you can enable storing an old and new tuple for each CRUD operation
 This information might be helpful for implementing a CDC (Change Data Capture) utility
 that transforms a data replication stream.
 
-
 .. _wal_extensions_configuration:
 
 Configuration
 -------------
 
-To configure WAL extensions, use the ``wal_ext`` :ref:`configuration property <index-book_cfg>`.
+To configure WAL extensions, use the ``wal.ext`` :ref:`configuration property <configuration_reference_wal>`.
 Inside the ``wal_ext`` block, you can enable storing old and new tuples as follows:
 
 *   Set the ``old`` and ``new`` options to ``true`` to store old and new tuples in a write-ahead log for all spaces.
diff --git a/doc/reference/configuration/cfg_binary_logging_snapshots.rst b/doc/reference/configuration/cfg_binary_logging_snapshots.rst
@@ -36,8 +36,8 @@
     Since version 1.7.4.
 
     The maximum number of bytes in a single write-ahead log file.
-    When a request would cause an .xlog file to become larger than
-    ``wal_max_size``, Tarantool creates another WAL file.
+    When a request would cause an ``.xlog`` file to become larger than
+    ``wal_max_size``, Tarantool creates a new WAL file.
 
     | Type: integer
     | Default: 268435456 (256 * 1024 * 1024) bytes
@@ -58,7 +58,7 @@
     locations and moving snapshots to a separate disk.
     The limit also affects what
     :ref:`box.stat.vinyl().regulator <box_introspection-box_stat_vinyl_regulator>`
-    may show for the write rate of dumps to .run and .index files.
+    may show for the write rate of dumps to ``.run`` and ``.index`` files.
 
     | Type: float
     | Default: null
@@ -73,12 +73,14 @@
 
     Specify fiber-WAL-disk synchronization mode as:
 
-    * ``none``: write-ahead log is not maintained.
-      A node with ``wal_mode = none`` can't be replication master;
-    * ``write``: :ref:`fibers <fiber-fibers>` wait for their data to be written to
-      the write-ahead log (no :manpage:`fsync(2)`);
-    * ``fsync``: fibers wait for their data, :manpage:`fsync(2)`
-      follows each :manpage:`write(2)`;
+    *   ``none``: write-ahead log is not maintained.
+        A node with ``wal_mode = none`` can't be replication master.
+
+    *   ``write``: :ref:`fibers <fiber-fibers>` wait for their data to be written to
+        the write-ahead log (no :manpage:`fsync(2)`).
+
+    *   ``fsync``: fibers wait for their data, :manpage:`fsync(2)`
+        follows each :manpage:`write(2)`.
 
     | Type: string
     | Default: "write"
@@ -129,7 +131,7 @@
     Since version :doc:`2.6.3 </release/2.6.3>`.
 
     The delay (in seconds) used to prevent the :ref:`Tarantool garbage collector <cfg_checkpoint_daemon-garbage-collector>`
-    from immediately removing :ref:`write-ahead log<internals-wal>` files after a node restart.
+    from immediately removing :ref:`write-ahead log <internals-wal>` files after a node restart.
     This delay eliminates possible erroneous situations when the master deletes WALs
     needed by :ref:`replicas <replication-roles>` after restart.
     As a consequence, replicas sync with the master faster after its restart and
@@ -141,7 +143,7 @@
     .. NOTE::
 
         The ``wal_cleanup_delay`` option has no effect on nodes running as
-        :ref:`anonymous replicas<cfg_replication-replication_anon>`.
+        :ref:`anonymous replicas <cfg_replication-replication_anon>`.
 
     | Type: number
     | Default: 14400 seconds
diff --git a/doc/reference/configuration/configuration_reference.rst b/doc/reference/configuration/configuration_reference.rst
@@ -2389,3 +2389,249 @@ The ``security`` section defines configuration parameters related to various sec
     | Type: boolean
     | Default: false
     | Environment variable: TT_SECURITY_SECURE_ERASING
+
+..  _configuration_reference_snapshot:
+
+snapshot
+--------
+
+The ``snapshot`` section defines configuration parameters related to
+
+.. NOTE::
+
+    ``snapshot`` can be defined in any :ref:`scope <configuration_scopes>`.
+
+-   :ref:`snapshot.by.interval <configuration_reference_snapshot_by_interval>`
+-   :ref:`snapshot.by.wal_size <configuration_reference_snapshot_by_wal_size>`
+-   :ref:`snapshot.count <configuration_reference_snapshot_count>`
+-   :ref:`snapshot.dir <configuration_reference_snapshot_dir>`
+-   :ref:`snapshot.snap_io_rate_limit <configuration_reference_snapshot_snap_io_rate_limit>`
+
+..  _configuration_reference_snapshot_by_interval:
+
+..  confval:: snapshot.by.interval
+
+    Description
+
+    |
+    | Type: number
+    | Default: 3600
+    | Environment variable: TT_SNAPSHOT_BY_INTERVAL
+
+..  _configuration_reference_snapshot_by_wal_size:
+
+..  confval:: snapshot.by.wal_size
+
+    Description
+
+    |
+    | Type: number
+    | Default: 3600
+    | Environment variable: TT_SNAPSHOT_BY_WAL_SIZE
+
+..  _configuration_reference_snapshot_count:
+
+..  confval:: snapshot.count
+
+    Description
+
+    |
+    | Type: integer
+    | Default: 2
+    | Environment variable: TT_SNAPSHOT_COUNT
+
+..  _configuration_reference_snapshot_dir:
+
+..  confval:: snapshot.dir
+
+    Description
+
+    |
+    | Type: string
+    | Default: 'var/lib/{{ instance_name }}'
+    | Environment variable: TT_SNAPSHOT_DIR
+
+
+..  _configuration_reference_snapshot_snap_io_rate_limit:
+
+..  confval:: snapshot.snap_io_rate_limit
+
+    Reduce the throttling effect of :doc:`box.snapshot() </reference/reference_lua/box_snapshot>` on
+    INSERT/UPDATE/DELETE performance by setting a limit on how many
+    megabytes per second it can write to disk. The same can be
+    achieved by splitting :ref:`wal.dir <configuration_reference_wal_dir>` and
+    :ref:`memtx_dir <cfg_basic-memtx_dir>`
+    locations and moving snapshots to a separate disk.
+    The limit also affects what
+    :ref:`box.stat.vinyl().regulator <box_introspection-box_stat_vinyl_regulator>`
+    may show for the write rate of dumps to ``.run`` and ``.index`` files.
+
+    |
+    | Type: number
+    | Default: box.NULL
+    | Environment variable: TT_SNAPSHOT_SNAP_IO_RATE_LIMIT
+
+
+..  _configuration_reference_wal:
+
+wal
+---
+
+The ``wal`` section defines configuration parameters related to
+
+..  NOTE::
+
+    ``wal`` can be defined in any :ref:`scope <configuration_scopes>`.
+
+-   :ref:`wal.cleanup_delay <configuration_reference_wal_cleanup_delay>`
+-   :ref:`wal.dir <configuration_reference_wal_dir>`
+-   :ref:`wal.dir_rescan_delay <configuration_reference_wal_dir_rescan_delay>`
+-   :ref:`wal.ext.new <configuration_reference_wal_ext_new>`
+-   :ref:`wal.ext.old <configuration_reference_wal_ext_old>`
+-   :ref:`wal.ext.spaces <configuration_reference_wal_ext_spaces>`
+-   :ref:`wal.wal_max_size <configuration_reference_wal_max_size>`
+-   :ref:`wal.mode <configuration_reference_wal_mode>`
+-   :ref:`wal.queue_max_size <configuration_reference_wal_queue_max_size>`
+
+..  _configuration_reference_wal_cleanup_delay:
+
+..  confval:: wal.cleanup_delay
+
+    The delay in seconds used to prevent the :ref:`Tarantool garbage collector <cfg_checkpoint_daemon-garbage-collector>`
+    from immediately removing :ref:`write-ahead log <internals-wal>` files after a node restart.
+    This delay eliminates possible erroneous situations when the master deletes WALs
+    needed by :ref:`replicas <replication-roles>` after restart.
+    As a consequence, replicas sync with the master faster after its restart and
+    don't need to download all the data again.
+
+    Once all the nodes in the replica set are up and running,
+    automatic cleanup is started again even if ``wal.cleanup_delay`` has not expired.
+
+    .. NOTE::
+
+        The option has no effect on nodes running as
+        :ref:`anonymous replicas <cfg_replication-replication_anon>`.
+
+    |
+    | Type: number
+    | Default: 14400
+    | Environment variable: TT_WAL_CLEANUP_DELAY
+
+..  _configuration_reference_wal_dir:
+
+..  confval:: wal.dir
+
+    A directory where write-ahead log (.xlog) files are stored. Can be relative
+    to :ref:`work_dir <cfg_basic-work_dir>`. Sometimes ``wal_dir`` and
+    :ref:`memtx_dir <cfg_basic-memtx_dir>` are specified with different values, so
+    that write-ahead log files and snapshot files can be stored on different
+    disks. If not specified, defaults to ``work_dir``.
+
+    |
+    | Type: string
+    | Default: 'var/lib/{{ instance_name }}'
+    | Environment variable: TT_WAL_DIR
+
+..  _configuration_reference_wal_dir_rescan_delay:
+
+..  confval:: wal.dir_rescan_delay
+
+    Number of seconds between periodic scans of the write-ahead-log
+    file directory, when checking for changes to write-ahead-log
+    files for the sake of :ref:`replication <replication>` or :ref:`hot standby <index-hot_standby>`.
+
+    |
+    | Type: number
+    | Default: 2
+    | Environment variable: TT_WAL_DIR_RESCAN_DELAY
+
+..  _configuration_reference_wal_ext_new:
+
+..  confval:: wal.ext_new
+
+    (**Enterprise Edition only**)
+
+    Description
+
+    |
+    | Type: boolean
+    | Default: nil
+    | Environment variable: TT_WAL_EXT_NEW
+
+..  _configuration_reference_wal_ext_old:
+
+..  confval:: wal.ext_old
+
+    (**Enterprise Edition only**)
+    Description
+
+    |
+    | Type: boolean
+    | Default: nil
+    | Environment variable: TT_WAL_EXT_OLD
+
+..  _configuration_reference_wal_ext_spaces:
+
+..  confval:: wal.ext_spaces
+
+    (**Enterprise Edition only**)
+    Description
+
+    |
+    | Type: map
+    | Default: nil
+    | Environment variable: TT_WAL_EXT_SPACES
+
+
+..  _configuration_reference_wal_max_size:
+
+..  confval:: wal.wal_max_size
+
+    The maximum number of bytes in a single write-ahead log file.
+    When a request would cause an ``.xlog`` file to become larger than
+    ``wal_max_size``, Tarantool creates a new WAL file.
+
+    |
+    | Type: integer
+    | Default: 268435456
+    | Environment variable: TT_WAL_MAX_SIZE
+
+..  _configuration_reference_wal_mode:
+
+..  confval:: wal.mode
+
+    Specify fiber-WAL-disk synchronization mode as:
+
+    *   ``none``: write-ahead log is not maintained.
+        A node with ``wal.mode = none`` can't be replication master.
+
+    *   ``write``: :ref:`fibers <fiber-fibers>` wait for their data to be written to
+        the write-ahead log (no :manpage:`fsync(2)`).
+
+    *   ``fsync``: fibers wait for their data, :manpage:`fsync(2)`
+        follows each :manpage:`write(2)`.
+
+    |
+    | Type: string
+    | Default: 'write'
+    | Environment variable: TT_WAL_MODE
+
+..  _configuration_reference_wal_queue_max_size:
+
+..  confval:: wal.queue_max_size
+
+    The size of the queue in bytes used by a :ref:`replica <replication-roles>` to submit
+    new transactions to a :ref:`write-ahead log<internals-wal>` (WAL).
+    This option helps limit the rate at which a replica submits transactions to the WAL.
+    Limiting the queue size might be useful when a replica is trying to sync with a master and
+    reads new transactions faster than writing them to the WAL.
+
+    ..  NOTE::
+
+        You might consider increasing the ``wal.queue_max_size`` value in case of
+        large tuples (approximately one megabyte or larger).
+
+    |
+    | Type: integer
+    | Default: 16777216
+    | Environment variable: TT_WAL_QUEUE_MAX_SIZE
