DOC-12487 Maintain durable writes #3810

ggray-cb · 2025-05-21T21:09:48Z

This PR incorporates changes for the Morpheus feature MB-43068 Maintain Durable Write Availability after losing replica

Changes in this PR (links lead to preview site. See this page for username & password)

What's New entry:
Durability. Heavy edits to remove passive voice and meet other doc standards. Added new section: Maintaining Durable Writes During Single Replica Failovers to cover new feature.
Creating and Editing Buckets added new parameter entry for durabilityImpossibleFallback

… topics before adding content for the maintain durability feature.

BenHuddleston

As a general comment, mixing the rewrite of the existing docs and the new feature makes this a harder review

BenHuddleston · 2025-05-22T08:35:05Z

modules/introduction/partials/new-features-80.adoc

@@ -108,6 +108,21 @@ and limits the number of metrics to 100.
 Additional information sent by clients at connection time can be found in the logs.


+[[new-feature-800-maintain-durable-writes]]
+https://jira.issues.couchbase.com/browse/MB-43068[MB-43068] Optionally Maintain Durable Writes During Single Replica Failovers::


I don't think that this title accurately reflects what this does. Things do not behave differently if there is a "Single" failover, or more. "Replica" is somewhat overloaded as we don't fail over replicas. It also does not state what we "Maintain".

Suggest:
"Optionally Maintain Durable Write Availability Without Majority After Failover"

BenHuddleston · 2025-05-22T08:35:59Z

modules/introduction/partials/new-features-80.adoc

@@ -108,6 +108,21 @@ and limits the number of metrics to 100.
 Additional information sent by clients at connection time can be found in the logs.


+[[new-feature-800-maintain-durable-writes]]
+https://jira.issues.couchbase.com/browse/MB-43068[MB-43068] Optionally Maintain Durable Writes During Single Replica Failovers::
+In a bucket with a single replica, you can enable an option named `durabilityImpossibleFallback` that allows durable writes to succeed even when they cannot meet their majority requirements.


Should not mention the number of replicas

BenHuddleston · 2025-05-22T08:37:44Z

modules/introduction/partials/new-features-80.adoc

+https://jira.issues.couchbase.com/browse/MB-43068[MB-43068] Optionally Maintain Durable Writes During Single Replica Failovers::
+In a bucket with a single replica, you can enable an option named `durabilityImpossibleFallback` that allows durable writes to succeed even when they cannot meet their majority requirements.
+This option is off by default.
+This is a temporary setting to allow clients to continue to write data when nodes are unavailable due to failovers.  


What is "temporary" about this setting? I don't think that we have any intent to remove it, and it does not turn itself off.

BenHuddleston · 2025-05-22T08:39:33Z

modules/introduction/partials/new-features-80.adoc

+In a bucket with a single replica, you can enable an option named `durabilityImpossibleFallback` that allows durable writes to succeed even when they cannot meet their majority requirements.
+This option is off by default.
+This is a temporary setting to allow clients to continue to write data when nodes are unavailable due to failovers.  
+For example, you can enable this option while you're performing an upgrade using the graceful failover followed by a delta recovery method.


This example applies only to the single replica case. If/when you remove the previous references to a single replica, you may wish/need to add that here.

BenHuddleston · 2025-05-22T08:53:20Z

modules/learn/pages/data/durability.adoc

-Once the write has been committed as specified by the requirements, Couchbase Server notifies the client of success.
-If commitment was not possible, Couchbase Server notifies the client of failure; and the data retains its former value throughout the cluster.
+After a write meets its durability requirements, Couchbase Server notifies the client of success. 
+If the write does not meet the durability requirements, Couchbase Server notifies the client that the write failed.


I would replace "If the write does not meet the durability requirements" with "If the write cannot meet the durability requirements".

To me, "does not" could mean that we find out after attempting it, after which we will not return that the write failed, we will return the ambiguous response.

BenHuddleston · 2025-05-22T09:39:23Z

modules/learn/partials/maintain-durability-warning.adoc

+.Potential Data Loss
+====
+Enabling `durabilityImpossibleFallback` degrades the promise that durable writes offer: that Couchbase Server has persisted the data in a way that should survive node failure.
+When enabled for a bucket, this setting  makes durable writes to it during a replica failover no more safe from data loss than regular asynchronous writes.


double space "setting makes"

BenHuddleston · 2025-05-22T09:39:43Z

modules/learn/partials/maintain-durability-warning.adoc

+.Potential Data Loss
+====
+Enabling `durabilityImpossibleFallback` degrades the promise that durable writes offer: that Couchbase Server has persisted the data in a way that should survive node failure.
+When enabled for a bucket, this setting  makes durable writes to it during a replica failover no more safe from data loss than regular asynchronous writes.


Same comment as before on "replica" failover.

BenHuddleston · 2025-05-22T09:41:38Z

modules/rest-api/pages/rest-bucket-create.adoc

+
+Overrides Couchbase Server's default behavior when it cannot meet a durable write's majority requirement.
+When set to the default `disabled` setting, Couchbase Server reports to clients that a durable write that cannot meet its majority requirement has failed.
+It also rolls back any data changes by the write across all nodes in the cluster.


It does not roll back anything. This check is performed before even attempting to contact a replica based on the configuration that the cluster manager passes to the data service.

BenHuddleston · 2025-05-22T09:41:55Z

modules/rest-api/pages/rest-bucket-create.adoc

+Overrides Couchbase Server's default behavior when it cannot meet a durable write's majority requirement.
+When set to the default `disabled` setting, Couchbase Server reports to clients that a durable write that cannot meet its majority requirement has failed.
+It also rolls back any data changes by the write across all nodes in the cluster.
+If you set this value to `fallbackToActiveAck`, Couchbase Server reports the write as successful even if it could not meet the  majority requirement.


double space "the majority"

BenHuddleston · 2025-05-22T09:42:49Z

modules/rest-api/pages/rest-bucket-create.adoc

+Overrides Couchbase Server's default behavior when it cannot meet a durable write's majority requirement.
+When set to the default `disabled` setting, Couchbase Server reports to clients that a durable write that cannot meet its majority requirement has failed.
+It also rolls back any data changes by the write across all nodes in the cluster.
+If you set this value to `fallbackToActiveAck`, Couchbase Server reports the write as successful even if it could not meet the  majority requirement.


This is also somewhat ambiguous/incorrect, it's still possible to see the ambiguous response if the replica is configured but the write times out for some reason.

hyunjuV

@ggray-cb
I've looked at the changes in this PR, and Ben Huddleston's comments. I do not have additional comments.

@BenHuddleston
Thank you for reviewing!

rao-shwe

Hi @ggray-cb
I've completed a round of editorial review. But it seems like technical review implementation is pending. Once that is complete, I'll do another round of editorial review.

rao-shwe · 2025-06-23T04:35:47Z

modules/learn/pages/data/durability.adoc


-This form of write is referred to as a _durable_ or _synchronous_ write.
+Couchbase Server supports durability for up to two replicas.
+It does not support durability for buckets with three replicas.


Change to:
"...with three or more replicas."

rao-shwe · 2025-06-23T04:40:21Z

modules/learn/pages/data/durability.adoc

-Such a write may be appropriate when saving data whose loss could have a considerable, negative impact.
-For example, data corresponding to a financial transaction.
+* A durable write is synchronous and provides durability guarantees. 
+Use this type of write for data where loss could have significant negative consequences, such as financial transactions.


Change to:
"..where the loss could result in significant..."

rao-shwe · 2025-06-23T04:42:27Z

modules/learn/pages/data/durability.adoc

+For a write to be durable, it must meet a majority requirement.
+The majority requirement is based on the number of replicas defined for the bucket.
+
+The following table shows the  majority requirement for each replica setting:


Remove the extra space before "majority".

rao-shwe · 2025-06-23T04:49:02Z

modules/learn/pages/data/durability.adoc

 |===

 [WARNING]
 ====

-In consequence of the correspondences listed above, if a bucket is configured with one replica, and a node fails, durable writes are immediately unavailable for any vBucket whose data resides on the failed node.
-
+As shown by the table, if you configure a bucket with one replica and a node fails, you cannot perform durable write for any vBucket whose data was on the failed node.


Can it be changed to:
"... you cannot perform durable write for any vBucket data that was on the failed node."

rao-shwe · 2025-06-23T05:27:36Z

modules/learn/pages/data/durability.adoc

+[[maintaining-durable-writes]]
+== Maintaining Durable Writes During Single Replica Failovers
+
+As described in <<#majority>>, a bucket with one replicas must meet a majority requirement of two nodes for a durable write to succeed.


Change to:
".. a bucket with one replica must meet..."

rao-shwe · 2025-06-23T05:32:48Z

modules/rest-api/pages/rest-bucket-create.adoc

+
+For information about the `durabilityImpossibleFallback` setting, see xref:learn:data/durability.adoc#maintaining-durable-writes[Maintaining Durable Writes During Single Replica Failovers].
+
+You can modify this parameter for existing buckets.


Change to:
".. for the existing buckets".

rao-shwe · 2025-06-23T05:34:45Z

modules/introduction/partials/new-features-80.adoc

+[[new-feature-800-maintain-durable-writes]]
+https://jira.issues.couchbase.com/browse/MB-43068[MB-43068] Optionally Maintain Durable Writes During Single Replica Failovers::
+In a bucket with a single replica, you can enable an option named `durabilityImpossibleFallback` that allows durable writes to succeed even when they cannot meet their majority requirements.
+This option is off by default.


Can it be changed to:
"This option is off by default."

ggray-cb added 7 commits May 20, 2025 13:18

Adding what's new changes. Also committing clean up of the DUrability…

32fb999

… topics before adding content for the maintain durability feature.

Copyedits.

53fa155

Rough draft complete for feature

97d9666

Some minor tweaks

526a91a

Last minute tweaks

b02b05d

Added What's New anchor

1b5634c

Fixed typo

cc52598

ggray-cb requested review from dave-finlay, BenHuddleston, hyunjuV, shivaniguptasf and a team May 21, 2025 21:09

BenHuddleston reviewed May 22, 2025

View reviewed changes

hyunjuV reviewed Jun 17, 2025

View reviewed changes

rao-shwe requested changes Jun 23, 2025

View reviewed changes


		For information about the `durabilityImpossibleFallback` setting, see xref:learn:data/durability.adoc#maintaining-durable-writes[Maintaining Durable Writes During Single Replica Failovers].

		You can modify this parameter for existing buckets.

DOC-12487 Maintain durable writes #3810

Are you sure you want to change the base?

DOC-12487 Maintain durable writes #3810

Uh oh!

Conversation

ggray-cb commented May 21, 2025 • edited by jira bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BenHuddleston left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hyunjuV left a comment

Choose a reason for hiding this comment

Uh oh!

rao-shwe left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ggray-cb commented May 21, 2025 •

edited by jira bot

Loading