Clarify retry notification code

[deepbluev7]

When looking into tweaking the backoff behaviour on my server, I came across this code:

synapse/synapse/util/retryutils.py

Lines 252 to 262 in f4fc83a

	if self.notifier:
	# Inform the relevant places that the remote server is back up.
	self.notifier.notify_remote_server_up(self.destination)
	if self.replication_client:
	# If we're on a worker we try and inform master about this. The
	# replication client doesn't hook into the notifier to avoid
	# infinite loops where we send a `REMOTE_SERVER_UP` command to
	# master, which then echoes it back to us which in turn pokes
	# the notifier.
	self.replication_client.send_remote_server_up(self.destination)

Specifically the code seems to notify every worker (and master), that a particular destination is up, even when it decided a destination is down and in my opinion should only be writing the new backoff interval to the database. This would be fairly easy to fix by just exiting that function if the retry interval is 0, however I am not familiar enough with Synapse internals to be sure that this doesn't break other stuff. It might also be the case, that this code does something completely different, than I assumed. In any case, I would appreciate if someone could clarify what that code is doing and possibly add a comment, why notifying the other workers is the right thing to do. I would assume this would lead to the backoff incrementing to quickly, but maybe it isn't possible to hit this case.

Originally I asked the question in #synapse and it was suggested to just file an issue to not forget about it: https://matrix.to/#/%23synapse-dev%3Amatrix.org/%24-5RtnWrJbfFjGIH8vKkpgOE7UVt2M9aNK6PThkvMsiI

Additionally we discovered, where that code was added and that someone else asked about this code as well at the time: #12500 (comment)

I hope this issue makes sense. I don't really know what to do about the code, I am just documenting, that multiple people think it is weird. :)

[clokep]

See #16506.