memtx: apply review suggestions

Author: xuniq

doc/code_snippets/snippets/config/instances.enabled/memtx/config.yaml

@@ -1,11 +1,7 @@
 memtx:
-  memory: 100000000
-  allocator: 'small'
+  memory: 1073741824
   min_tuple_size: 8
-  max_tuple_size: 1048576
-  slab_alloc_factor: 1.05
-  slab_alloc_granularity: 8
-  sort_threads: 256
+  max_tuple_size: 5242880
 
 groups:
   group001:

doc/concepts/atomic/thread_model.rst

@@ -64,19 +64,18 @@ Supplementary threads
 
 There are also several supplementary threads that serve additional capabilities:
 
-* For :ref:`replication <replication-architecture>`, Tarantool creates a separate thread for each connected replica.
-  This thread reads a write-ahead log and sends it to the replica, following its position in the log.
-  Separate threads are required because each replica can point to a different position in the log and can run at different speeds.
+*   For :ref:`replication <replication-architecture>`, Tarantool creates a separate thread for each connected replica.
+    This thread reads a write-ahead log and sends it to the replica, following its position in the log.
+    Separate threads are required because each replica can point to a different position in the log and can run at different speeds.
 
-* There is a thread pool for ad hoc asynchronous tasks,
-  such as a DNS resolver or :ref:`fsync <configuration_reference_wal_mode>`.
+*   There is a thread pool for ad hoc asynchronous tasks, such as a DNS resolver or :ref:`fsync <configuration_reference_wal_mode>`.
 
-* There are OpenMP threads used to parallelize sorting
-  (hence, to parallelize building :ref:`indexes <concepts-data_model_indexes>`).
-  For example, this is applicable when Tarantool is restoring from a
-  :ref:`snapshot <internals-snapshot>` with a large amount of data
-  and needs to sort a secondary index if it is ordered by something other than the primary order.
+*   There is a thread pool that can be used for parallel sorting (hence, to parallelize building :ref:`indexes <concepts-data_model_indexes>`).
+    To configure it, use the :ref:`memtx.sort_threads <configuration_reference_memtx_sort_threads>` configuration option.
+    The option sets the number of threads used to sort keys of secondary indexes on loading ``memtx`` database.
 
-  ..  note::
+    ..  NOTE::
 
-    The maximum number of OpenMP threads can be controlled by the ``OMP_NUM_THREADS`` environment variable.
+        Since: :doc:`3.0.0 </release/3.0.0>`, this option replaces the approach when OpenMP threads are used to parallelize sorting.
+        For backward compatibility, the ``OMP_NUM_THREADS`` environment variable is still supported in Tarantool 3.0.
+        The variable defines the maximum number of OpenMP threads.

doc/concepts/configuration/configuration_memtx.rst

@@ -8,226 +8,71 @@ In-memory storage
 In Tarantool, all data is stored in random-access memory (RAM) by default.
 For this purpose, the :ref:`memtx <engines-memtx>` storage engine is used.
 
-This topic describes how to define settings related to in-memory storage in the
-:ref:`memtx <configuration_reference_memtx>` section of a :ref:`YAML configuration <configuration>`.
+This topic describes how to define basic settings related to in-memory storage in the
+:ref:`memtx <configuration_reference_memtx>` section of a :ref:`YAML configuration <configuration>`
+-- for example, :ref:`memory size <configuration_reference_memtx_memory>` and :ref:`maximum tuple size <>configuration_reference_memtx_max_size`.
+For the specific settings related to allocator or sorting threads,
+check the corresponding ``memtx`` options in the :ref:`Configuration reference <configuration_reference_memtx>`.
 
 ..  NOTE::
 
-    To calculate the required amount of memory, you can use the
+    To estimate the required amount of memory, you can use the
     `sizing calculator <https://www.tarantool.io/en/sizing_calculator/>`_.
 
 ..  _configuration_memtx-memory:
 
-Configure the memory size
--------------------------
+Memory size
+-----------
 
 In Tarantool, data is stored in spaces.
 Each space consists of tuples -- the database records.
-To specify the amount of memory that Tarantool allocates to actually store tuples, use the
-:ref:`memtx.memory <configuration_reference_snapshot_dir>` configuration option.
+To specify the amount of memory that Tarantool allocates to store tuples, use the
+:ref:`memtx.memory <configuration_reference_memtx_memory>` configuration option.
 
-In the example below, the memory size is 100000000 bytes, which is about 95 MB.
-The server does not exceeds this limit to allocate tuples.
-For indexes and connection information, the additional memory is used.
-
-When the ``memtx.memory`` limit is reached, ``INSERT`` or ``UPDATE`` requests fail with
-:ref:`ER_MEMORY_ISSUE <admin-troubleshoot-memory-issues>`.
+In the example below, the memory size is set to 1 GB (1073741824 bytes):
 
 ..  literalinclude:: /code_snippets/snippets/config/instances.enabled/memtx/config.yaml
     :language: yaml
     :start-at: memtx:
-    :end-at: 100000000
+    :end-at: 1073741824
     :dedent:
 
-..  _configuration_memtx-allocator:
-
-Specify the allocator type
---------------------------
-
-The allocators are the memory managers used for different purposes.
-To set the allocator type that manages memory for ``memtx`` tuples, use the :ref:`memtx.allocator <configuration_reference_memtx_allocator>`
-configuration option.
+The server does not exceed this limit to allocate tuples.
+For indexes and connection information, additional memory is used.
 
-Possible types:
-
-*   ``system`` -- the memory is allocated as needed, checking that the quota is not exceeded.
+When the ``memtx.memory`` limit is reached, ``INSERT`` or ``UPDATE`` requests fail with
+:ref:`ER_MEMORY_ISSUE <admin-troubleshoot-memory-issues>`.
 
-*   ``small`` (default) -- a `slab allocator <https://github.com/tarantool/small>`_ mechanism is used.
+..  _configuration_memtx-tuple-size:
 
-The example below shows how to specify the ``small`` allocator type:
+Tuple size
+----------
 
-..  literalinclude:: /code_snippets/snippets/config/instances.enabled/memtx/config.yaml
-    :language: yaml
-    :start-at: allocator:
-    :end-at: 'small'
-    :dedent:
-
-..  _configuration_memtx-min-max:
+You can configure the sizes of the smallest and the largest units to allocate.
 
-Specify the size limitations of the allocation unit
----------------------------------------------------
+..  _configuration_memtx-tuple-size-min:
 
-You can configure the sizes of the smallest and the largest units to allocate.
+Minimum size
+~~~~~~~~~~~~~
 
-To set the minimum size in bytes, use the :ref:`memtx.min_tuple_size <configuration_reference_memtx_min_size>` configuration option.
-The setting can be useful if the tuples that you store are small.
-The minimum size is a value between 8 and 1048280 inclusive.
+To set the minimum size in bytes, use the :ref:`memtx.min_tuple_size <configuration_reference_memtx_min_size>` configuration option:
 
 ..  literalinclude:: /code_snippets/snippets/config/instances.enabled/memtx/config.yaml
     :language: yaml
     :start-at: min_tuple_size: 8
     :end-at: min_tuple_size: 8
     :dedent:
 
-To set the maximum size in bytes, use the :ref:`memtx.max_tuple_size <configuration_reference_memtx_max_size>` configuration option.
-The setting can be useful if the tuples that you store are large.
-
-..  literalinclude:: /code_snippets/snippets/config/instances.enabled/memtx/config.yaml
-    :language: yaml
-    :start-at: max_tuple_size:
-    :end-at: 1048576
-    :dedent:
-
-
-..  _configuration_memtx-slab-alloc-granularity:
-
-Specify the granularity for slab allocation
--------------------------------------------
-
-If you use the :ref:`small <configuration_reference_memtx_allocator>` type of allocator, you can set the granularity of
-memory allocation in it.
-To do it, use the :ref:`memtx.slab_alloc_granularity <configuration_reference_memtx_slab_alloc_granularity>` configuration option.
-The granularity value should meet the following conditions:
-
-*   The value is a power of two.
-*   The value is greater than or equal to 4.
-
-If the tuples in space are small and have about the same size, set the option to 4 bytes to save memory.
-If the tuples are different-sized, increase the option value to allocate tuples from the same ``mempool`` (memory pool).
-
-In the example below, the value is increased to store different-sized tuples:
-
-..  literalinclude:: /code_snippets/snippets/config/instances.enabled/memtx/config.yaml
-    :language: yaml
-    :start-at: slab_alloc_granularity:
-    :end-at: 8
-    :dedent:
-
-..  _configuration_memtx-slab-alloc-factor:
-
-Specify the slab allocation factor
-----------------------------------
-
-The size and number of memory pools depends on allocation factor and granularity.
-The allocation factor is a multiplier used to calculate the sizes of memory chunks that tuples are stored in.
-To configure the allocation factor, use the :ref:`memtx.slab_alloc_factor <configuration_reference_memtx_slab_alloc_factor>`
-configuration option.
-The option is a value between 1 and 2.
-
-In the example below, the option is set to the default value:
-
-..  literalinclude:: /code_snippets/snippets/config/instances.enabled/memtx/config.yaml
-    :language: yaml
-    :start-at: slab_alloc_factor:
-    :end-at: 1.05
-    :dedent:
-
-..  _configuration_memtx-sort-threads:
+..  _configuration_memtx-tuple-size-max:
 
-Set the number of the sorting threads
--------------------------------------
+Maximum size
+~~~~~~~~~~~~
 
-It is possible to configure the number of threads that are used to sort keys of secondary indexes on loading
-``memtx`` database.
-To do it, use the :ref:`memtx.sort_threads <configuration_reference_memtx_sort_threads>` configuration option.
-
-In the example, the maximum number of threads is used (256):
+To set the maximum size in bytes, use the :ref:`memtx.max_tuple_size <configuration_reference_memtx_max_size>` configuration option.
+In the example, the maximum size is set to 5 MB:
 
 ..  literalinclude:: /code_snippets/snippets/config/instances.enabled/memtx/config.yaml
     :language: yaml
-    :start-at: sort_threads:
-    :end-at: 256
+    :start-at: max_tuple_size:
+    :end-at: 5242880
     :dedent:
-
-..  _configuration_memtx-statistics:
-
-Check the database statistics
------------------------------
-
-Tarantool provides the statistics about memory consumption for the given space or specific tuples.
-Available statistics:
-
-*   The amount of memory used for the :ref:`data of the specified space <configuration_memtx-statistics-space>`.
-*   The amount of :ref:`additional memory <configuration_memtx-statistics-additional>` used for the supplementary information.
-*   The :ref:`total memory usage <configuration_memtx-statistics-total>`.
-
-..  _configuration_memtx-statistics-space:
-
-Space
-~~~~~
-
-To get the get the amount of memory in bytes occupied by the specified space, use the :ref:`space_object:bsize() <box_space-bsize>` method.
-The function returns the total number of bytes in all tuples.
-
-..  code-block:: console
-
-    memtx:instance001> box.space.books:bsize()
-    ---
-    - 70348673
-    ...
-
-..  _configuration_memtx-statistics-additional:
-
-Additional memory
-~~~~~~~~~~~~~~~~~
-
-To check the usage of the additional memory, use the ``space_object:stat()`` method.
-The following information is provided:
-
--   ``header_size`` and ``field_map_size``: the size of service information.
--   ``data_size``: the actual size of data, which equals to ``space_object:bsize()``.
--   ``waste_size``: the size of memory wasted due to internal fragmentation in the `slab allocator <https://github.com/tarantool/small>`_.
-
-.. code-block:: console
-
-    memtx:instance001> box.space.books:stat()
-    ---
-    - tuple:
-        memtx:
-          waste_size: 1744011
-          data_size: 70348673
-          header_size: 2154132
-          field_map_size: 0
-        malloc:
-          waste_size: 0
-          data_size: 0
-          header_size: 0
-          field_map_size: 0
-    ...
-
-To get the usage of the additional memory (5 Mb) for the specified tuple, use ``tuple_object:info()``:
-
-.. code-block:: console
-
-    memtx:instance001> box.space.books:get('1853260622'):info()
-    ---
-    - data_size: 277
-      waste_size: 9
-      arena: memtx
-      field_map_size: 0
-      header_size: 10
-    ...
-
-..  _configuration_memtx-statistics-total:
-
-Total memory usage
-~~~~~~~~~~~~~~~~~~
-
-To get the total memory usage in bytes for the slab allocator, use the :ref:`box.slab.info() <box_slab_info>` function:
-
-..  code-block:: console
-
-    memtx:instance001> box.slab.info().items_used
-    ---
-    - 75302024
-    ...

doc/reference/configuration/cfg_storage.rst

@@ -22,7 +22,7 @@
 
     Since version 1.7.4.
 
-    How much memory Tarantool allocates to actually store tuples.
+    How much memory Tarantool allocates to store tuples.
     When the limit is reached, :ref:`INSERT <box_space-insert>` or
     :ref:`UPDATE <box_space-insert>` requests begin failing with
     error :errcode:`ER_MEMORY_ISSUE`. The server does not go beyond the
@@ -43,7 +43,6 @@
 
     Size of the largest allocation unit, for the memtx storage engine. It can be
     increased if it is necessary to store large tuples.
-    See also: :ref:`vinyl_max_tuple_size <cfg_storage-vinyl_max_tuple_size>`.
 
     | Type: integer
     | Default: 1024 * 1024 = 1048576 bytes
@@ -57,11 +56,11 @@
     Since version 1.7.4.
 
     Size of the smallest allocation unit. It can be decreased if most
-    of the tuples are very small. The value must be between 8 and 1048280
-    inclusive.
+    of the tuples are very small.
 
     | Type: integer
     | Default: 16 bytes
+    | Possible values: between 8 and 1048280 inclusive
     | Environment variable: TT_MEMTX_MIN_TUPLE_SIZE
     | Dynamic: no
 
@@ -71,13 +70,13 @@
 
     Since version :doc:`2.10.0 </release/2.10.0>`.
 
-    Specify the allocator used for memtx tuples.
-    The possible values are ``system``  and ``small``:
+    Specify the allocator that manages memory for ``memtx`` tuples.
+    Possible values:
 
-    *   ``system`` is based on the ``malloc`` function.
-        The allocator allocates memory as needed, checking that the quota is not exceeded.
+    *   ``system`` --  the memory is allocated as needed, checking that the quota is not exceeded.
+        THe allocator is based on the ``malloc`` function.
 
-    *   ``small`` is a special `slab allocator <https://github.com/tarantool/small>`_.
+    *   ``small`` -- a `slab allocator <https://github.com/tarantool/small>`_.
         The allocator repeatedly uses a memory block to allocate objects of the same type.
         Note that this allocator is prone to unresolvable fragmentation on specific workloads,
         so you can switch to ``system`` in such cases.
@@ -91,6 +90,8 @@
 
 ..  confval:: memtx_sort_threads
 
+    Since: :doc:`3.0.0 </release/3.0.0>`.
+
     The number of threads used to sort keys of secondary indexes on loading ``memtx`` database.
     The maximum value is 256, the minimum value is 1.
     The default is to use all available cores.
@@ -108,12 +109,13 @@
     The multiplier for computing the sizes of memory
     chunks that tuples are stored in. A lower value may result in less wasted
     memory depending on the total amount of memory available and the
-    distribution of item sizes. Allowed values range from 1 to 2.
+    distribution of item sizes.
 
     See also: :ref:`slab_alloc_granularity <cfg_storage-slab_alloc_granularity>`
 
     | Type: float
     | Default: 1.05
+    | Possible values: between 1 and 2 inclusive
     | Environment variable: TT_SLAB_ALLOC_FACTOR
     | Dynamic: no
 
@@ -124,13 +126,15 @@
     Since version :doc:`2.8.1 </release/2.8.1>`.
 
     Specify the granularity (in bytes) of memory allocation in the :ref:`small allocator <cfg_storage-memtx_allocator>`.
-    The value of ``slab_alloc_granularity`` should be a power of two and should be greater than or equal to 4.
-    Below are few recommendations on how to adjust the ``slab_alloc_granularity`` value:
+    The ``memtx.slab_alloc_granularity`` value should meet the following conditions:
+
+    *   The value is a power of two.
+    *   The value is greater than or equal to 4.
 
-    * To store small tuples of approximately the same size, set ``slab_alloc_granularity`` to 4 bytes to save memory.
+    Below are few recommendations on how to adjust the ``memtx.slab_alloc_granularity`` option:
 
-    * To store tuples of different sizes, you can increase the ``slab_alloc_granularity`` value.
-      This results in allocating tuples from the same ``mempool``.
+    *   If the tuples in space are small and have about the same size, set the option to 4 bytes to save memory.
+    *   If the tuples are different-sized, increase the option value to allocate tuples from the same ``mempool`` (memory pool).
 
     See also: :ref:`slab_alloc_factor <cfg_storage-slab_alloc_factor>`