Skip to content

MSC3666: Bundled aggregations for server side search #3666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Apr 10, 2022
Merged

MSC3666: Bundled aggregations for server side search #3666

merged 6 commits into from
Apr 10, 2022

Conversation

clokep
Copy link
Member

@clokep clokep commented Jan 21, 2022 •
edited
Loading

@clokep clokep changed the title MSCXXXX: Bundled aggregations for server side search MSC3666: Bundled aggregations for server side search Jan 21, 2022
clokep
clokep commented Jan 21, 2022
View reviewed changes
@turt2live turt2live added client-server Client-Server API kind:feature MSC for not-core and not-maintenance stuff needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. proposal A matrix spec change proposal labels Jan 21, 2022
This was referenced Jan 25, 2022
Closed
Merged
@clokep clokep marked this pull request as ready for review January 26, 2022 18:52
@clokep
Copy link
Member Author

clokep commented Jan 26, 2022

This now has an implementation which I've linked in the description: matrix-org/synapse#11837.

@clokep clokep removed the needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. label Feb 8, 2022
@richvdh
Copy link
Member

richvdh commented Feb 8, 2022

@clokep says: Hello! 👋 I believe that both MSC3666 and MSC3715 are ready for consideration by the SCT. They're both minor improvements that build on MSC2675.

turt2live
turt2live previously requested changes Feb 8, 2022
View reviewed changes
squahtx pushed a commit to matrix-org/synapse that referenced this pull request Feb 15, 2022
Merge tag 'v1.53.0rc1' into develop
85e24d9 
Synapse 1.53.0rc1 (2022-02-15)
==============================

Features
--------

- Add experimental support for sending to-device messages to application services, as specified by [MSC2409](matrix-org/matrix-spec-proposals#2409). ([\#11215](#11215), [\#11966](#11966))
- Remove account data (including client config, push rules and ignored users) upon user deactivation. ([\#11655](#11655))
- Experimental support for [MSC3666](matrix-org/matrix-spec-proposals#3666): including bundled aggregations in server side search results. ([\#11837](#11837))
- Enable cache time-based expiry by default. The `expiry_time` config flag has been superseded by `expire_caches` and `cache_entry_ttl`. ([\#11849](#11849))
- Add a callback to allow modules to allow or forbid a 3PID (email address, phone number) from being associated to a local account. ([\#11854](#11854))
- Stabilize support and remove unstable endpoints for [MSC3231](matrix-org/matrix-spec-proposals#3231). Clients must switch to the stable identifier and endpoint. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#stablisation-of-msc3231) for more information. ([\#11867](#11867))
- Allow modules to retrieve the current instance's server name and worker name. ([\#11868](#11868))
- Use a dedicated configurable rate limiter for 3PID invites. ([\#11892](#11892))
- Support the stable API endpoint for [MSC3283](matrix-org/matrix-spec-proposals#3283): new settings in `/capabilities` endpoint. ([\#11933](#11933), [\#11989](#11989))
- Support the `dir` parameter on the `/relations` endpoint, per [MSC3715](matrix-org/matrix-spec-proposals#3715). ([\#11941](#11941))
- Experimental implementation of [MSC3706](matrix-org/matrix-spec-proposals#3706): extensions to `/send_join` to support reduced response size. ([\#11967](#11967))

Bugfixes
--------

- Fix [MSC2716](matrix-org/matrix-spec-proposals#2716) historical messages backfilling in random order on remote homeservers. ([\#11114](#11114))
- Fix a bug introduced in Synapse 1.51.0 where incoming federation transactions containing at least one EDU would be dropped if debug logging was enabled for `synapse.8631_debug`. ([\#11890](#11890))
- Fix a long-standing bug where some unknown endpoints would return HTML error pages instead of JSON `M_UNRECOGNIZED` errors. ([\#11930](#11930))
- Implement an allow list of content types for which we will attempt to preview a URL. This prevents Synapse from making useless longer-lived connections to streaming media servers. ([\#11936](#11936))
- Fix a long-standing bug where pagination tokens from `/sync` and `/messages` could not be provided to the `/relations` API. ([\#11952](#11952))
- Require that modules register their callbacks using keyword arguments. ([\#11975](#11975))
- Fix a long-standing bug where `M_WRONG_ROOM_KEYS_VERSION` errors would not include the specced `current_version` field. ([\#11988](#11988))

Improved Documentation
----------------------

- Fix typo in User Admin API: unpind -> unbind. ([\#11859](#11859))
- Document images returned by the User List Media Admin API can include those generated by URL previews. ([\#11862](#11862))
- Remove outdated MSC1711 FAQ document. ([\#11907](#11907))
- Correct the structured logging configuration example. Contributed by Brad Jones. ([\#11946](#11946))
- Add information on the Synapse release cycle. ([\#11954](#11954))
- Fix broken link in the README to the admin API for password reset. ([\#11955](#11955))

Deprecations and Removals
-------------------------

- Drop support for `webclient` listeners and configuring `web_client_location` to a non-HTTP(S) URL. Deprecated configurations are a configuration error. ([\#11895](#11895))
- Remove deprecated `user_may_create_room_with_invites` spam checker callback. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#removal-of-user_may_create_room_with_invites) for more information. ([\#11950](#11950))
- No longer build `.deb` packages for Ubuntu 21.04 Hirsute Hippo, which has now EOLed. ([\#11961](#11961))

Internal Changes
----------------

- Enhance user registration test helpers to make them more useful for tests involving application services and devices. ([\#11615](#11615), [\#11616](#11616))
- Improve performance when fetching bundled aggregations for multiple events. ([\#11660](#11660), [\#11752](#11752))
- Fix type errors introduced by new annotations in the Prometheus Client library. ([\#11832](#11832))
- Add missing type hints to replication code. ([\#11856](#11856), [\#11938](#11938))
- Ensure that `opentracing` scopes are activated and closed at the right time. ([\#11869](#11869))
- Improve opentracing for incoming federation requests. ([\#11870](#11870))
- Improve internal docstrings in `synapse.util.caches`. ([\#11876](#11876))
- Do not needlessly clear the `get_users_in_room` and `get_users_in_room_with_profiles` caches when any room state changes. ([\#11878](#11878))
- Convert `ApplicationServiceTestCase` to use `simple_async_mock`. ([\#11880](#11880))
- Remove experimental changes to the default push rules which were introduced in Synapse 1.19.0 but never enabled. ([\#11884](#11884))
- Disable coverage calculation for olddeps build. ([\#11888](#11888))
- Preparation to support sending device list updates to application services. ([\#11905](#11905))
- Add a test that checks users receive their own device list updates down `/sync`. ([\#11909](#11909))
- Run Complement tests sequentially. ([\#11910](#11910))
- Various refactors to the application service notifier code. ([\#11911](#11911), [\#11912](#11912))
- Tests: replace mocked `Authenticator` with the real thing. ([\#11913](#11913))
- Various refactors to the typing notifications code. ([\#11914](#11914))
- Use the proper type for the `Content-Length` header in the `UploadResource`. ([\#11927](#11927))
- Remove an unnecessary ignoring of type hints due to fixes in upstream packages. ([\#11939](#11939))
- Add missing type hints. ([\#11953](#11953))
- Fix an import cycle in `synapse.event_auth`. ([\#11965](#11965))
- Unpin `frozendict` but exclude the known bad version 2.1.2. ([\#11969](#11969))
- Prepare for rename of default Complement branch. ([\#11971](#11971))
- Fetch Synapse's version using a helper from `matrix-common`. ([\#11979](#11979))
@turt2live
Copy link
Member

@matrix-org/spec-core-team the author has mentioned that this is now re-ready for review (it was already on the board, but in an awkward state. That state is now fixed)

@turt2live turt2live self-requested a review February 23, 2022 22:46
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Feb 25, 2022
@Midar
Update chat/matrix-synapse to 1.53.0
7750a4e 
Synapse 1.53.0 (2022-02-22)
===========================

No significant changes.


Synapse 1.53.0rc1 (2022-02-15)
==============================

Features
--------

- Add experimental support for sending to-device messages to application services, as specified by [MSC2409](matrix-org/matrix-spec-proposals#2409). ([\#11215](matrix-org/synapse#11215), [\#11966](matrix-org/synapse#11966))
- Remove account data (including client config, push rules and ignored users) upon user deactivation. ([\#11655](matrix-org/synapse#11655))
- Experimental support for [MSC3666](matrix-org/matrix-spec-proposals#3666): including bundled aggregations in server side search results. ([\#11837](matrix-org/synapse#11837))
- Enable cache time-based expiry by default. The `expiry_time` config flag has been superseded by `expire_caches` and `cache_entry_ttl`. ([\#11849](matrix-org/synapse#11849))
- Add a callback to allow modules to allow or forbid a 3PID (email address, phone number) from being associated to a local account. ([\#11854](matrix-org/synapse#11854))
- Stabilize support and remove unstable endpoints for [MSC3231](matrix-org/matrix-spec-proposals#3231). Clients must switch to the stable identifier and endpoint. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#stablisation-of-msc3231) for more information. ([\#11867](matrix-org/synapse#11867))
- Allow modules to retrieve the current instance's server name and worker name. ([\#11868](matrix-org/synapse#11868))
- Use a dedicated configurable rate limiter for 3PID invites. ([\#11892](matrix-org/synapse#11892))
- Support the stable API endpoint for [MSC3283](matrix-org/matrix-spec-proposals#3283): new settings in `/capabilities` endpoint. ([\#11933](matrix-org/synapse#11933), [\#11989](matrix-org/synapse#11989))
- Support the `dir` parameter on the `/relations` endpoint, per [MSC3715](matrix-org/matrix-spec-proposals#3715). ([\#11941](matrix-org/synapse#11941))
- Experimental implementation of [MSC3706](matrix-org/matrix-spec-proposals#3706): extensions to `/send_join` to support reduced response size. ([\#11967](matrix-org/synapse#11967))


Bugfixes
--------

- Fix [MSC2716](matrix-org/matrix-spec-proposals#2716) historical messages backfilling in random order on remote homeservers. ([\#11114](matrix-org/synapse#11114))
- Fix a bug introduced in Synapse 1.51.0 where incoming federation transactions containing at least one EDU would be dropped if debug logging was enabled for `synapse.8631_debug`. ([\#11890](matrix-org/synapse#11890))
- Fix a long-standing bug where some unknown endpoints would return HTML error pages instead of JSON `M_UNRECOGNIZED` errors. ([\#11930](matrix-org/synapse#11930))
- Implement an allow list of content types for which we will attempt to preview a URL. This prevents Synapse from making useless longer-lived connections to streaming media servers. ([\#11936](matrix-org/synapse#11936))
- Fix a long-standing bug where pagination tokens from `/sync` and `/messages` could not be provided to the `/relations` API. ([\#11952](matrix-org/synapse#11952))
- Require that modules register their callbacks using keyword arguments. ([\#11975](matrix-org/synapse#11975))
- Fix a long-standing bug where `M_WRONG_ROOM_KEYS_VERSION` errors would not include the specced `current_version` field. ([\#11988](matrix-org/synapse#11988))


Improved Documentation
----------------------

- Fix typo in User Admin API: unpind -> unbind. ([\#11859](matrix-org/synapse#11859))
- Document images returned by the User List Media Admin API can include those generated by URL previews. ([\#11862](matrix-org/synapse#11862))
- Remove outdated MSC1711 FAQ document. ([\#11907](matrix-org/synapse#11907))
- Correct the structured logging configuration example. Contributed by Brad Jones. ([\#11946](matrix-org/synapse#11946))
- Add information on the Synapse release cycle. ([\#11954](matrix-org/synapse#11954))
- Fix broken link in the README to the admin API for password reset. ([\#11955](matrix-org/synapse#11955))


Deprecations and Removals
-------------------------

- Drop support for `webclient` listeners and configuring `web_client_location` to a non-HTTP(S) URL. Deprecated configurations are a configuration error. ([\#11895](matrix-org/synapse#11895))
- Remove deprecated `user_may_create_room_with_invites` spam checker callback. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#removal-of-user_may_create_room_with_invites) for more information. ([\#11950](matrix-org/synapse#11950))
- No longer build `.deb` packages for Ubuntu 21.04 Hirsute Hippo, which has now EOLed. ([\#11961](matrix-org/synapse#11961))


Internal Changes
----------------

- Enhance user registration test helpers to make them more useful for tests involving application services and devices. ([\#11615](matrix-org/synapse#11615), [\#11616](matrix-org/synapse#11616))
- Improve performance when fetching bundled aggregations for multiple events. ([\#11660](matrix-org/synapse#11660), [\#11752](matrix-org/synapse#11752))
- Fix type errors introduced by new annotations in the Prometheus Client library. ([\#11832](matrix-org/synapse#11832))
- Add missing type hints to replication code. ([\#11856](matrix-org/synapse#11856), [\#11938](matrix-org/synapse#11938))
- Ensure that `opentracing` scopes are activated and closed at the right time. ([\#11869](matrix-org/synapse#11869))
- Improve opentracing for incoming federation requests. ([\#11870](matrix-org/synapse#11870))
- Improve internal docstrings in `synapse.util.caches`. ([\#11876](matrix-org/synapse#11876))
- Do not needlessly clear the `get_users_in_room` and `get_users_in_room_with_profiles` caches when any room state changes. ([\#11878](matrix-org/synapse#11878))
- Convert `ApplicationServiceTestCase` to use `simple_async_mock`. ([\#11880](matrix-org/synapse#11880))
- Remove experimental changes to the default push rules which were introduced in Synapse 1.19.0 but never enabled. ([\#11884](matrix-org/synapse#11884))
- Disable coverage calculation for olddeps build. ([\#11888](matrix-org/synapse#11888))
- Preparation to support sending device list updates to application services. ([\#11905](matrix-org/synapse#11905))
- Add a test that checks users receive their own device list updates down `/sync`. ([\#11909](matrix-org/synapse#11909))
- Run Complement tests sequentially. ([\#11910](matrix-org/synapse#11910))
- Various refactors to the application service notifier code. ([\#11911](matrix-org/synapse#11911), [\#11912](matrix-org/synapse#11912))
- Tests: replace mocked `Authenticator` with the real thing. ([\#11913](matrix-org/synapse#11913))
- Various refactors to the typing notifications code. ([\#11914](matrix-org/synapse#11914))
- Use the proper type for the `Content-Length` header in the `UploadResource`. ([\#11927](matrix-org/synapse#11927))
- Remove an unnecessary ignoring of type hints due to fixes in upstream packages. ([\#11939](matrix-org/synapse#11939))
- Add missing type hints. ([\#11953](matrix-org/synapse#11953))
- Fix an import cycle in `synapse.event_auth`. ([\#11965](matrix-org/synapse#11965))
- Unpin `frozendict` but exclude the known bad version 2.1.2. ([\#11969](matrix-org/synapse#11969))
- Prepare for rename of default Complement branch. ([\#11971](matrix-org/synapse#11971))
- Fetch Synapse's version using a helper from `matrix-common`. ([\#11979](matrix-org/synapse#11979))


Synapse 1.52.0 (2022-02-08)
===========================

No significant changes since 1.52.0rc1.

Note that [Twisted 22.1.0](https://github.com/twisted/twisted/releases/tag/twisted-22.1.0)
has recently been released, which fixes a [security issue](GHSA-92x2-jw7w-xvvx)
within the Twisted library. We do not believe Synapse is affected by this vulnerability,
though we advise server administrators who installed Synapse via pip to upgrade Twisted
with `pip install --upgrade Twisted` as a matter of good practice. The Docker image
`matrixdotorg/synapse` and the Debian packages from `packages.matrix.org` are using the
updated library.


Synapse 1.52.0rc1 (2022-02-01)
==============================

Features
--------

- Remove account data (including client config, push rules and ignored users) upon user deactivation. ([\#11621](matrix-org/synapse#11621), [\#11788](matrix-org/synapse#11788), [\#11789](matrix-org/synapse#11789))
- Add an admin API to reset connection timeouts for remote server. ([\#11639](matrix-org/synapse#11639))
- Add an admin API to get a list of rooms that federate with a given remote homeserver. ([\#11658](matrix-org/synapse#11658))
- Add a config flag to inhibit `M_USER_IN_USE` during registration. ([\#11743](matrix-org/synapse#11743))
- Add a module callback to set username at registration. ([\#11790](matrix-org/synapse#11790))
- Allow configuring a maximum file size as well as a list of allowed content types for avatars. ([\#11846](matrix-org/synapse#11846))


Bugfixes
--------

- Include the bundled aggregations in the `/sync` response, per [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\#11612](matrix-org/synapse#11612))
- Fix a long-standing bug when previewing Reddit URLs which do not contain an image. ([\#11767](matrix-org/synapse#11767))
- Fix a long-standing bug that media streams could cause long-lived connections when generating URL previews. ([\#11784](matrix-org/synapse#11784))
- Include a `prev_content` field in state events sent to Application Services. Contributed by @totallynotvaishnav. ([\#11798](matrix-org/synapse#11798))
- Fix a bug introduced in Synapse 0.33.3 causing requests to sometimes log strings such as `HTTPStatus.OK` instead of integer status codes. ([\#11827](matrix-org/synapse#11827))


Improved Documentation
----------------------

- Update pypi installation docs to indicate that we now support Python 3.10. ([\#11820](matrix-org/synapse#11820))
- Add missing steps to the contribution submission process in the documentation.  Contributed by @sequentialread. ([\#11821](matrix-org/synapse#11821))
- Remove not needed old table of contents in documentation. ([\#11860](matrix-org/synapse#11860))
- Consolidate the `access_token` information at the top of each relevant page in the Admin API documentation. ([\#11861](matrix-org/synapse#11861))


Deprecations and Removals
-------------------------

- Drop support for Python 3.6, which is EOL. ([\#11683](matrix-org/synapse#11683))
- Remove the `experimental_msc1849_support_enabled` flag as the features are now stable. ([\#11843](matrix-org/synapse#11843))


Internal Changes
----------------

- Preparation for database schema simplifications: add `state_key` and `rejection_reason` columns to `events` table. ([\#11792](matrix-org/synapse#11792))
- Add `FrozenEvent.get_state_key` and use it in a couple of places. ([\#11793](matrix-org/synapse#11793))
- Preparation for database schema simplifications: stop reading from `event_reference_hashes`. ([\#11794](matrix-org/synapse#11794))
- Drop unused table `public_room_list_stream`. ([\#11795](matrix-org/synapse#11795))
- Preparation for reducing Postgres serialization errors: allow setting transaction isolation level. Contributed by Nick @ Beeper. ([\#11799](matrix-org/synapse#11799), [\#11847](matrix-org/synapse#11847))
- Docker: skip the initial amd64-only build and go straight to multiarch. ([\#11810](matrix-org/synapse#11810))
- Run Complement on the Github Actions VM and not inside a Docker container. ([\#11811](matrix-org/synapse#11811))
- Log module names at startup. ([\#11813](matrix-org/synapse#11813))
- Improve type safety of bundled aggregations code. ([\#11815](matrix-org/synapse#11815))
- Correct a type annotation in the event validation logic. ([\#11817](matrix-org/synapse#11817), [\#11830](matrix-org/synapse#11830))
- Minor updates and documentation for database schema delta files. ([\#11823](matrix-org/synapse#11823))
- Workaround a type annotation problem in `prometheus_client` 0.13.0. ([\#11834](matrix-org/synapse#11834))
- Minor performance improvement in room state lookup. ([\#11836](matrix-org/synapse#11836))
- Fix some indentation inconsistencies in the sample config. ([\#11838](matrix-org/synapse#11838))
- Add type hints to `tests/rest/admin`. ([\#11851](matrix-org/synapse#11851))
Fizzadar added a commit to Fizzadar/synapse that referenced this pull request Mar 7, 2022
@Fizzadar
Merge tag 'v1.53.0' into beeper-master
2c92316 
Synapse 1.53.0 (2022-02-22)
===========================

No significant changes.

Synapse 1.53.0rc1 (2022-02-15)
==============================

Features
--------

- Add experimental support for sending to-device messages to application services, as specified by [MSC2409](matrix-org/matrix-spec-proposals#2409). ([\matrix-org#11215](matrix-org#11215), [\matrix-org#11966](matrix-org#11966))
- Remove account data (including client config, push rules and ignored users) upon user deactivation. ([\matrix-org#11655](matrix-org#11655))
- Experimental support for [MSC3666](matrix-org/matrix-spec-proposals#3666): including bundled aggregations in server side search results. ([\matrix-org#11837](matrix-org#11837))
- Enable cache time-based expiry by default. The `expiry_time` config flag has been superseded by `expire_caches` and `cache_entry_ttl`. ([\matrix-org#11849](matrix-org#11849))
- Add a callback to allow modules to allow or forbid a 3PID (email address, phone number) from being associated to a local account. ([\matrix-org#11854](matrix-org#11854))
- Stabilize support and remove unstable endpoints for [MSC3231](matrix-org/matrix-spec-proposals#3231). Clients must switch to the stable identifier and endpoint. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#stablisation-of-msc3231) for more information. ([\matrix-org#11867](matrix-org#11867))
- Allow modules to retrieve the current instance's server name and worker name. ([\matrix-org#11868](matrix-org#11868))
- Use a dedicated configurable rate limiter for 3PID invites. ([\matrix-org#11892](matrix-org#11892))
- Support the stable API endpoint for [MSC3283](matrix-org/matrix-spec-proposals#3283): new settings in `/capabilities` endpoint. ([\matrix-org#11933](matrix-org#11933), [\matrix-org#11989](matrix-org#11989))
- Support the `dir` parameter on the `/relations` endpoint, per [MSC3715](matrix-org/matrix-spec-proposals#3715). ([\matrix-org#11941](matrix-org#11941))
- Experimental implementation of [MSC3706](matrix-org/matrix-spec-proposals#3706): extensions to `/send_join` to support reduced response size. ([\matrix-org#11967](matrix-org#11967))

Bugfixes
--------

- Fix [MSC2716](matrix-org/matrix-spec-proposals#2716) historical messages backfilling in random order on remote homeservers. ([\matrix-org#11114](matrix-org#11114))
- Fix a bug introduced in Synapse 1.51.0 where incoming federation transactions containing at least one EDU would be dropped if debug logging was enabled for `synapse.8631_debug`. ([\matrix-org#11890](matrix-org#11890))
- Fix a long-standing bug where some unknown endpoints would return HTML error pages instead of JSON `M_UNRECOGNIZED` errors. ([\matrix-org#11930](matrix-org#11930))
- Implement an allow list of content types for which we will attempt to preview a URL. This prevents Synapse from making useless longer-lived connections to streaming media servers. ([\matrix-org#11936](matrix-org#11936))
- Fix a long-standing bug where pagination tokens from `/sync` and `/messages` could not be provided to the `/relations` API. ([\matrix-org#11952](matrix-org#11952))
- Require that modules register their callbacks using keyword arguments. ([\matrix-org#11975](matrix-org#11975))
- Fix a long-standing bug where `M_WRONG_ROOM_KEYS_VERSION` errors would not include the specced `current_version` field. ([\matrix-org#11988](matrix-org#11988))

Improved Documentation
----------------------

- Fix typo in User Admin API: unpind -> unbind. ([\matrix-org#11859](matrix-org#11859))
- Document images returned by the User List Media Admin API can include those generated by URL previews. ([\matrix-org#11862](matrix-org#11862))
- Remove outdated MSC1711 FAQ document. ([\matrix-org#11907](matrix-org#11907))
- Correct the structured logging configuration example. Contributed by Brad Jones. ([\matrix-org#11946](matrix-org#11946))
- Add information on the Synapse release cycle. ([\matrix-org#11954](matrix-org#11954))
- Fix broken link in the README to the admin API for password reset. ([\matrix-org#11955](matrix-org#11955))

Deprecations and Removals
-------------------------

- Drop support for `webclient` listeners and configuring `web_client_location` to a non-HTTP(S) URL. Deprecated configurations are a configuration error. ([\matrix-org#11895](matrix-org#11895))
- Remove deprecated `user_may_create_room_with_invites` spam checker callback. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#removal-of-user_may_create_room_with_invites) for more information. ([\matrix-org#11950](matrix-org#11950))
- No longer build `.deb` packages for Ubuntu 21.04 Hirsute Hippo, which has now EOLed. ([\matrix-org#11961](matrix-org#11961))

Internal Changes
----------------

- Enhance user registration test helpers to make them more useful for tests involving application services and devices. ([\matrix-org#11615](matrix-org#11615), [\matrix-org#11616](matrix-org#11616))
- Improve performance when fetching bundled aggregations for multiple events. ([\matrix-org#11660](matrix-org#11660), [\matrix-org#11752](matrix-org#11752))
- Fix type errors introduced by new annotations in the Prometheus Client library. ([\matrix-org#11832](matrix-org#11832))
- Add missing type hints to replication code. ([\matrix-org#11856](matrix-org#11856), [\matrix-org#11938](matrix-org#11938))
- Ensure that `opentracing` scopes are activated and closed at the right time. ([\matrix-org#11869](matrix-org#11869))
- Improve opentracing for incoming federation requests. ([\matrix-org#11870](matrix-org#11870))
- Improve internal docstrings in `synapse.util.caches`. ([\matrix-org#11876](matrix-org#11876))
- Do not needlessly clear the `get_users_in_room` and `get_users_in_room_with_profiles` caches when any room state changes. ([\matrix-org#11878](matrix-org#11878))
- Convert `ApplicationServiceTestCase` to use `simple_async_mock`. ([\matrix-org#11880](matrix-org#11880))
- Remove experimental changes to the default push rules which were introduced in Synapse 1.19.0 but never enabled. ([\matrix-org#11884](matrix-org#11884))
- Disable coverage calculation for olddeps build. ([\matrix-org#11888](matrix-org#11888))
- Preparation to support sending device list updates to application services. ([\matrix-org#11905](matrix-org#11905))
- Add a test that checks users receive their own device list updates down `/sync`. ([\matrix-org#11909](matrix-org#11909))
- Run Complement tests sequentially. ([\matrix-org#11910](matrix-org#11910))
- Various refactors to the application service notifier code. ([\matrix-org#11911](matrix-org#11911), [\matrix-org#11912](matrix-org#11912))
- Tests: replace mocked `Authenticator` with the real thing. ([\matrix-org#11913](matrix-org#11913))
- Various refactors to the typing notifications code. ([\matrix-org#11914](matrix-org#11914))
- Use the proper type for the `Content-Length` header in the `UploadResource`. ([\matrix-org#11927](matrix-org#11927))
- Remove an unnecessary ignoring of type hints due to fixes in upstream packages. ([\matrix-org#11939](matrix-org#11939))
- Add missing type hints. ([\matrix-org#11953](matrix-org#11953))
- Fix an import cycle in `synapse.event_auth`. ([\matrix-org#11965](matrix-org#11965))
- Unpin `frozendict` but exclude the known bad version 2.1.2. ([\matrix-org#11969](matrix-org#11969))
- Prepare for rename of default Complement branch. ([\matrix-org#11971](matrix-org#11971))
- Fetch Synapse's version using a helper from `matrix-common`. ([\matrix-org#11979](matrix-org#11979))
@erikjohnston
Copy link
Member

@mscbot fcp merge

@mscbot
Copy link
Collaborator

mscbot commented Mar 8, 2022 •
edited by dbkr
Loading

Team member @erikjohnston has proposed to merge this. The next step is review by the rest of the tagged people:

Once at least 75% of reviewers approve (and there are no outstanding concerns), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for information about what commands tagged team members can give me.

@mscbot mscbot added disposition-merge proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. labels Mar 8, 2022
richvdh
richvdh reviewed Mar 22, 2022
View reviewed changes
Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm modulo a couple of minor wording tweaks

clokep and others added 2 commits March 22, 2022 11:34
@richvdh richvdh dismissed turt2live’s stale review March 22, 2022 15:37

outdated review

This was referenced Mar 28, 2022
Closed
Closed
@mscbot
Copy link
Collaborator

mscbot commented Apr 5, 2022

🔔 This is now entering its final comment period, as per the review above. 🔔

@mscbot mscbot added final-comment-period This MSC has entered a final comment period in interest to approval, postpone, or delete in 5 days. and removed proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. labels Apr 5, 2022
@clokep clokep mentioned this pull request Apr 6, 2022
Closed
@turt2live turt2live removed their request for review April 9, 2022 04:39
@mscbot
Copy link
Collaborator

mscbot commented Apr 10, 2022

The final comment period, with a disposition to merge, as per the review above, is now complete.

@mscbot mscbot added finished-final-comment-period and removed disposition-merge final-comment-period This MSC has entered a final comment period in interest to approval, postpone, or delete in 5 days. labels Apr 10, 2022
@turt2live turt2live merged commit 919ca2f into main Apr 10, 2022
@turt2live turt2live added spec-pr-missing Proposal has been implemented and is being used in the wild but hasn't yet been added to the spec and removed finished-final-comment-period labels Apr 10, 2022
This was referenced Apr 11, 2022
Merged
Open
babolivier added a commit to matrix-org/synapse-dinsic that referenced this pull request Apr 28, 2022
@babolivier
Merge tag 'v1.53.0' into babolivier/dinsic_1.58
c3ee93e 
Synapse 1.53.0 (2022-02-22)
===========================

No significant changes.

Synapse 1.53.0rc1 (2022-02-15)
==============================

Features
--------

- Add experimental support for sending to-device messages to application services, as specified by [MSC2409](matrix-org/matrix-spec-proposals#2409). ([\#11215](matrix-org/synapse#11215), [\#11966](matrix-org/synapse#11966))
- Remove account data (including client config, push rules and ignored users) upon user deactivation. ([\#11655](matrix-org/synapse#11655))
- Experimental support for [MSC3666](matrix-org/matrix-spec-proposals#3666): including bundled aggregations in server side search results. ([\#11837](matrix-org/synapse#11837))
- Enable cache time-based expiry by default. The `expiry_time` config flag has been superseded by `expire_caches` and `cache_entry_ttl`. ([\#11849](matrix-org/synapse#11849))
- Add a callback to allow modules to allow or forbid a 3PID (email address, phone number) from being associated to a local account. ([\#11854](matrix-org/synapse#11854))
- Stabilize support and remove unstable endpoints for [MSC3231](matrix-org/matrix-spec-proposals#3231). Clients must switch to the stable identifier and endpoint. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#stablisation-of-msc3231) for more information. ([\#11867](matrix-org/synapse#11867))
- Allow modules to retrieve the current instance's server name and worker name. ([\#11868](matrix-org/synapse#11868))
- Use a dedicated configurable rate limiter for 3PID invites. ([\#11892](matrix-org/synapse#11892))
- Support the stable API endpoint for [MSC3283](matrix-org/matrix-spec-proposals#3283): new settings in `/capabilities` endpoint. ([\#11933](matrix-org/synapse#11933), [\#11989](matrix-org/synapse#11989))
- Support the `dir` parameter on the `/relations` endpoint, per [MSC3715](matrix-org/matrix-spec-proposals#3715). ([\#11941](matrix-org/synapse#11941))
- Experimental implementation of [MSC3706](matrix-org/matrix-spec-proposals#3706): extensions to `/send_join` to support reduced response size. ([\#11967](matrix-org/synapse#11967))

Bugfixes
--------

- Fix [MSC2716](matrix-org/matrix-spec-proposals#2716) historical messages backfilling in random order on remote homeservers. ([\#11114](matrix-org/synapse#11114))
- Fix a bug introduced in Synapse 1.51.0 where incoming federation transactions containing at least one EDU would be dropped if debug logging was enabled for `synapse.8631_debug`. ([\#11890](matrix-org/synapse#11890))
- Fix a long-standing bug where some unknown endpoints would return HTML error pages instead of JSON `M_UNRECOGNIZED` errors. ([\#11930](matrix-org/synapse#11930))
- Implement an allow list of content types for which we will attempt to preview a URL. This prevents Synapse from making useless longer-lived connections to streaming media servers. ([\#11936](matrix-org/synapse#11936))
- Fix a long-standing bug where pagination tokens from `/sync` and `/messages` could not be provided to the `/relations` API. ([\#11952](matrix-org/synapse#11952))
- Require that modules register their callbacks using keyword arguments. ([\#11975](matrix-org/synapse#11975))
- Fix a long-standing bug where `M_WRONG_ROOM_KEYS_VERSION` errors would not include the specced `current_version` field. ([\#11988](matrix-org/synapse#11988))

Improved Documentation
----------------------

- Fix typo in User Admin API: unpind -> unbind. ([\#11859](matrix-org/synapse#11859))
- Document images returned by the User List Media Admin API can include those generated by URL previews. ([\#11862](matrix-org/synapse#11862))
- Remove outdated MSC1711 FAQ document. ([\#11907](matrix-org/synapse#11907))
- Correct the structured logging configuration example. Contributed by Brad Jones. ([\#11946](matrix-org/synapse#11946))
- Add information on the Synapse release cycle. ([\#11954](matrix-org/synapse#11954))
- Fix broken link in the README to the admin API for password reset. ([\#11955](matrix-org/synapse#11955))

Deprecations and Removals
-------------------------

- Drop support for `webclient` listeners and configuring `web_client_location` to a non-HTTP(S) URL. Deprecated configurations are a configuration error. ([\#11895](matrix-org/synapse#11895))
- Remove deprecated `user_may_create_room_with_invites` spam checker callback. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#removal-of-user_may_create_room_with_invites) for more information. ([\#11950](matrix-org/synapse#11950))
- No longer build `.deb` packages for Ubuntu 21.04 Hirsute Hippo, which has now EOLed. ([\#11961](matrix-org/synapse#11961))

Internal Changes
----------------

- Enhance user registration test helpers to make them more useful for tests involving application services and devices. ([\#11615](matrix-org/synapse#11615), [\#11616](matrix-org/synapse#11616))
- Improve performance when fetching bundled aggregations for multiple events. ([\#11660](matrix-org/synapse#11660), [\#11752](matrix-org/synapse#11752))
- Fix type errors introduced by new annotations in the Prometheus Client library. ([\#11832](matrix-org/synapse#11832))
- Add missing type hints to replication code. ([\#11856](matrix-org/synapse#11856), [\#11938](matrix-org/synapse#11938))
- Ensure that `opentracing` scopes are activated and closed at the right time. ([\#11869](matrix-org/synapse#11869))
- Improve opentracing for incoming federation requests. ([\#11870](matrix-org/synapse#11870))
- Improve internal docstrings in `synapse.util.caches`. ([\#11876](matrix-org/synapse#11876))
- Do not needlessly clear the `get_users_in_room` and `get_users_in_room_with_profiles` caches when any room state changes. ([\#11878](matrix-org/synapse#11878))
- Convert `ApplicationServiceTestCase` to use `simple_async_mock`. ([\#11880](matrix-org/synapse#11880))
- Remove experimental changes to the default push rules which were introduced in Synapse 1.19.0 but never enabled. ([\#11884](matrix-org/synapse#11884))
- Disable coverage calculation for olddeps build. ([\#11888](matrix-org/synapse#11888))
- Preparation to support sending device list updates to application services. ([\#11905](matrix-org/synapse#11905))
- Add a test that checks users receive their own device list updates down `/sync`. ([\#11909](matrix-org/synapse#11909))
- Run Complement tests sequentially. ([\#11910](matrix-org/synapse#11910))
- Various refactors to the application service notifier code. ([\#11911](matrix-org/synapse#11911), [\#11912](matrix-org/synapse#11912))
- Tests: replace mocked `Authenticator` with the real thing. ([\#11913](matrix-org/synapse#11913))
- Various refactors to the typing notifications code. ([\#11914](matrix-org/synapse#11914))
- Use the proper type for the `Content-Length` header in the `UploadResource`. ([\#11927](matrix-org/synapse#11927))
- Remove an unnecessary ignoring of type hints due to fixes in upstream packages. ([\#11939](matrix-org/synapse#11939))
- Add missing type hints. ([\#11953](matrix-org/synapse#11953))
- Fix an import cycle in `synapse.event_auth`. ([\#11965](matrix-org/synapse#11965))
- Unpin `frozendict` but exclude the known bad version 2.1.2. ([\#11969](matrix-org/synapse#11969))
- Prepare for rename of default Complement branch. ([\#11971](matrix-org/synapse#11971))
- Fetch Synapse's version using a helper from `matrix-common`. ([\#11979](matrix-org/synapse#11979))
babolivier added a commit to matrix-org/synapse-dinsic that referenced this pull request Apr 28, 2022
@babolivier
Merge tag 'v1.57.0' into babolivier/dinsic_1.58
6e9f1e3 
Synapse 1.57.0 (2022-04-19)
===========================

This version includes a [change](matrix-org/synapse#12209) to the way transaction IDs are managed for application services. If your deployment uses a dedicated worker for application service traffic, **it must be stopped** when the database is upgraded (which normally happens when the main process is upgraded), to ensure the change is made safely without any risk of reusing transaction IDs.

See the [upgrade notes](https://github.com/matrix-org/synapse/blob/v1.57.0rc1/docs/upgrade.md#upgrading-to-v1570) for more details.

No significant changes since 1.57.0rc1.

Synapse 1.57.0rc1 (2022-04-12)
==============================

Features
--------

- Send device list changes to application services as specified by [MSC3202](matrix-org/matrix-spec-proposals#3202), using unstable prefixes. The `msc3202_transaction_extensions` experimental homeserver config option must be enabled and `org.matrix.msc3202: true` must be present in the application service registration file for device list changes to be sent. The "left" field is currently always empty. ([\#11881](matrix-org/synapse#11881))
- Optimise fetching large quantities of missing room state over federation. ([\#12040](matrix-org/synapse#12040))
- Offload the `update_client_ip` background job from the main process to the background worker, when using Redis-based replication. ([\#12251](matrix-org/synapse#12251))
- Move `update_client_ip` background job from the main process to the background worker. ([\#12252](matrix-org/synapse#12252))
- Add a module callback to react to new 3PID (email address, phone number) associations. ([\#12302](matrix-org/synapse#12302))
- Add a configuration option to remove a specific set of rooms from sync responses. ([\#12310](matrix-org/synapse#12310))
- Add a module callback to react to account data changes. ([\#12327](matrix-org/synapse#12327))
- Allow setting user admin status using the module API. Contributed by Famedly. ([\#12341](matrix-org/synapse#12341))
- Reduce overhead of restarting synchrotrons. ([\#12367](matrix-org/synapse#12367), [\#12372](matrix-org/synapse#12372))
- Update `/messages` to use historic pagination tokens if no `from` query parameter is given. ([\#12370](matrix-org/synapse#12370))
- Add a module API for reading and writing global account data. ([\#12391](matrix-org/synapse#12391))
- Support the stable `v1` endpoint for `/relations`, per [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\#12403](matrix-org/synapse#12403))
- Include bundled aggregations in search results
  ([MSC3666](matrix-org/matrix-spec-proposals#3666)). ([\#12436](matrix-org/synapse#12436))

Bugfixes
--------

- Fix a long-standing bug where updates to the server notices user profile (display name/avatar URL) in the configuration would not be applied to pre-existing rooms. Contributed by Jorge Florian. ([\#12115](matrix-org/synapse#12115))
- Fix a long-standing bug where events from ignored users were still considered for bundled aggregations. ([\#12235](matrix-org/synapse#12235), [\#12338](matrix-org/synapse#12338))
- Fix non-member state events not resolving for historical events when used in [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` `state_events_at_start`. ([\#12329](matrix-org/synapse#12329))
- Fix a long-standing bug affecting URL previews that would generate a 500 response instead of a 403 if the previewed URL includes a port that isn't allowed by the relevant blacklist. ([\#12333](matrix-org/synapse#12333))
- Default to `private` room visibility rather than `public` when a client does not specify one, according to spec. ([\#12350](matrix-org/synapse#12350))
- Fix a spec compliance issue where requests to the `/publicRooms` federation API would specify `limit` as a string. ([\#12364](matrix-org/synapse#12364), [\#12410](matrix-org/synapse#12410))
- Fix a bug introduced in Synapse 1.49.0 which caused the `synapse_event_persisted_position` metric to have invalid values. ([\#12390](matrix-org/synapse#12390))

Updates to the Docker image
---------------------------

- Bundle locked versions of dependencies into the Docker image. ([\#12385](matrix-org/synapse#12385), [\#12439](matrix-org/synapse#12439))
- Fix up healthcheck generation for workers docker image. ([\#12405](matrix-org/synapse#12405))

Improved Documentation
----------------------

- Clarify documentation for running SyTest against Synapse, including use of Postgres and worker mode. ([\#12271](matrix-org/synapse#12271))
- Document the behaviour of `LoggingTransaction.call_after` and `LoggingTransaction.call_on_exception` methods when transactions are retried. ([\#12315](matrix-org/synapse#12315))
- Update dead links in `check-newsfragment.sh` to point to the correct documentation URL. ([\#12331](matrix-org/synapse#12331))
- Upgrade the version of `mdbook` in CI to 0.4.17. ([\#12339](matrix-org/synapse#12339))
- Updates to the Room DAG concepts development document to clarify that we mark events as outliers because we don't have any state for them. ([\#12345](matrix-org/synapse#12345))
- Update the link to Redis pub/sub documentation in the workers documentation. ([\#12369](matrix-org/synapse#12369))
- Remove documentation for converting a legacy structured logging configuration to the new format. ([\#12392](matrix-org/synapse#12392))

Deprecations and Removals
-------------------------

- Remove the unused and unstable `/aggregations` endpoint which was removed from [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\#12293](matrix-org/synapse#12293))

Internal Changes
----------------

- Remove lingering unstable references to MSC2403 (knocking). ([\#12165](matrix-org/synapse#12165))
- Avoid trying to calculate the state at outlier events. ([\#12191](matrix-org/synapse#12191), [\#12316](matrix-org/synapse#12316), [\#12330](matrix-org/synapse#12330), [\#12332](matrix-org/synapse#12332), [\#12409](matrix-org/synapse#12409))
- Omit sending "offline" presence updates to application services after they are initially configured. ([\#12193](matrix-org/synapse#12193))
- Switch to using a sequence to generate AS transaction IDs. Contributed by Nick @ Beeper. If running synapse with a dedicated appservice worker, this MUST be stopped before upgrading the main process and database. ([\#12209](matrix-org/synapse#12209))
- Add missing type hints for storage. ([\#12267](matrix-org/synapse#12267))
- Add missing type definitions for scripts in docker folder. Contributed by Jorge Florian. ([\#12280](matrix-org/synapse#12280))
- Move [MSC2654](matrix-org/matrix-spec-proposals#2654) support behind an experimental configuration flag. ([\#12295](matrix-org/synapse#12295))
- Update docstrings to explain how to decipher live and historic pagination tokens. ([\#12317](matrix-org/synapse#12317))
- Add ground work for speeding up device list updates for users in large numbers of rooms. ([\#12321](matrix-org/synapse#12321))
- Fix typechecker problems exposed by signedjson 1.1.2. ([\#12326](matrix-org/synapse#12326))
- Remove the `tox` packaging job: it will be redundant once #11537 lands. ([\#12334](matrix-org/synapse#12334))
- Ignore `.envrc` for `direnv` users. ([\#12335](matrix-org/synapse#12335))
- Remove the (broadly unused, dev-only) dockerfile for pg tests. ([\#12336](matrix-org/synapse#12336))
- Remove redundant `get_success` calls in test code. ([\#12346](matrix-org/synapse#12346))
- Add type annotations for `tests/unittest.py`. ([\#12347](matrix-org/synapse#12347))
- Move single-use methods out of `TestCase`. ([\#12348](matrix-org/synapse#12348))
- Remove broken and unused development scripts. ([\#12349](matrix-org/synapse#12349), [\#12351](matrix-org/synapse#12351), [\#12355](matrix-org/synapse#12355))
- Convert `Linearizer` tests from `inlineCallbacks` to async. ([\#12353](matrix-org/synapse#12353))
- Update docstrings for `ReadWriteLock` tests. ([\#12354](matrix-org/synapse#12354))
- Refactor `Linearizer`, convert methods to async and use an async context manager. ([\#12357](matrix-org/synapse#12357))
- Fix a long-standing bug where `Linearizer`s could get stuck if a cancellation were to happen at the wrong time. ([\#12358](matrix-org/synapse#12358))
- Make `StreamToken.from_string` and `RoomStreamToken.parse` propagate cancellations instead of replacing them with `SynapseError`s. ([\#12366](matrix-org/synapse#12366))
- Add type hints to tests files. ([\#12371](matrix-org/synapse#12371))
- Allow specifying the Postgres database's port when running unit tests with Postgres. ([\#12376](matrix-org/synapse#12376))
- Remove temporary pin of signedjson<=1.1.1 that was added in Synapse 1.56.0. ([\#12379](matrix-org/synapse#12379))
- Add opentracing spans to calls to external cache. ([\#12380](matrix-org/synapse#12380))
- Lay groundwork for using `poetry` to manage Synapse's dependencies. ([\#12381](matrix-org/synapse#12381), [\#12407](matrix-org/synapse#12407), [\#12412](matrix-org/synapse#12412), [\#12418](matrix-org/synapse#12418))
- Make missing `importlib_metadata` dependency explicit. ([\#12384](matrix-org/synapse#12384), [\#12400](matrix-org/synapse#12400))
- Update type annotations for compatiblity with prometheus_client 0.14. ([\#12389](matrix-org/synapse#12389))
- Remove support for the unstable identifiers specified in [MSC3288](matrix-org/matrix-spec-proposals#3288). ([\#12398](matrix-org/synapse#12398))
- Add missing type hints to configuration classes. ([\#12402](matrix-org/synapse#12402))
- Add files used to build the Docker image used for complement testing into the Synapse repository. ([\#12404](matrix-org/synapse#12404))
- Do not include groups in the sync response when disabled. ([\#12408](matrix-org/synapse#12408))
- Improve type hints related to HTTP query parameters. ([\#12415](matrix-org/synapse#12415))
- Stop maintaining a list of lint targets. ([\#12420](matrix-org/synapse#12420))
- Make `synapse._scripts` pass type checks. ([\#12421](matrix-org/synapse#12421), [\#12422](matrix-org/synapse#12422))
- Add some type hints to datastore. ([\#12423](matrix-org/synapse#12423))
- Enable certificate checking during complement tests. ([\#12435](matrix-org/synapse#12435))
- Explicitly specify the `tls` extra for Twisted dependency. ([\#12444](matrix-org/synapse#12444))
PiotrKozimor added a commit to globekeeper/synapse that referenced this pull request May 5, 2022
@PiotrKozimor
Merge tag 'v1.57.0' into develop
30e6e3f 
Synapse 1.57.0 (2022-04-19)
===========================

This version includes a [change](matrix-org#12209) to the way transaction IDs are managed for application services. If your deployment uses a dedicated worker for application service traffic, **it must be stopped** when the database is upgraded (which normally happens when the main process is upgraded), to ensure the change is made safely without any risk of reusing transaction IDs.

See the [upgrade notes](https://github.com/matrix-org/synapse/blob/v1.57.0rc1/docs/upgrade.md#upgrading-to-v1570) for more details.

No significant changes since 1.57.0rc1.

Synapse 1.57.0rc1 (2022-04-12)
==============================

Features
--------

- Send device list changes to application services as specified by [MSC3202](matrix-org/matrix-spec-proposals#3202), using unstable prefixes. The `msc3202_transaction_extensions` experimental homeserver config option must be enabled and `org.matrix.msc3202: true` must be present in the application service registration file for device list changes to be sent. The "left" field is currently always empty. ([\matrix-org#11881](matrix-org#11881))
- Optimise fetching large quantities of missing room state over federation. ([\matrix-org#12040](matrix-org#12040))
- Offload the `update_client_ip` background job from the main process to the background worker, when using Redis-based replication. ([\matrix-org#12251](matrix-org#12251))
- Move `update_client_ip` background job from the main process to the background worker. ([\matrix-org#12252](matrix-org#12252))
- Add a module callback to react to new 3PID (email address, phone number) associations. ([\matrix-org#12302](matrix-org#12302))
- Add a configuration option to remove a specific set of rooms from sync responses. ([\matrix-org#12310](matrix-org#12310))
- Add a module callback to react to account data changes. ([\matrix-org#12327](matrix-org#12327))
- Allow setting user admin status using the module API. Contributed by Famedly. ([\matrix-org#12341](matrix-org#12341))
- Reduce overhead of restarting synchrotrons. ([\matrix-org#12367](matrix-org#12367), [\matrix-org#12372](matrix-org#12372))
- Update `/messages` to use historic pagination tokens if no `from` query parameter is given. ([\matrix-org#12370](matrix-org#12370))
- Add a module API for reading and writing global account data. ([\matrix-org#12391](matrix-org#12391))
- Support the stable `v1` endpoint for `/relations`, per [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\matrix-org#12403](matrix-org#12403))
- Include bundled aggregations in search results
  ([MSC3666](matrix-org/matrix-spec-proposals#3666)). ([\matrix-org#12436](matrix-org#12436))

Bugfixes
--------

- Fix a long-standing bug where updates to the server notices user profile (display name/avatar URL) in the configuration would not be applied to pre-existing rooms. Contributed by Jorge Florian. ([\matrix-org#12115](matrix-org#12115))
- Fix a long-standing bug where events from ignored users were still considered for bundled aggregations. ([\matrix-org#12235](matrix-org#12235), [\matrix-org#12338](matrix-org#12338))
- Fix non-member state events not resolving for historical events when used in [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` `state_events_at_start`. ([\matrix-org#12329](matrix-org#12329))
- Fix a long-standing bug affecting URL previews that would generate a 500 response instead of a 403 if the previewed URL includes a port that isn't allowed by the relevant blacklist. ([\matrix-org#12333](matrix-org#12333))
- Default to `private` room visibility rather than `public` when a client does not specify one, according to spec. ([\matrix-org#12350](matrix-org#12350))
- Fix a spec compliance issue where requests to the `/publicRooms` federation API would specify `limit` as a string. ([\matrix-org#12364](matrix-org#12364), [\matrix-org#12410](matrix-org#12410))
- Fix a bug introduced in Synapse 1.49.0 which caused the `synapse_event_persisted_position` metric to have invalid values. ([\matrix-org#12390](matrix-org#12390))

Updates to the Docker image
---------------------------

- Bundle locked versions of dependencies into the Docker image. ([\matrix-org#12385](matrix-org#12385), [\matrix-org#12439](matrix-org#12439))
- Fix up healthcheck generation for workers docker image. ([\matrix-org#12405](matrix-org#12405))

Improved Documentation
----------------------

- Clarify documentation for running SyTest against Synapse, including use of Postgres and worker mode. ([\matrix-org#12271](matrix-org#12271))
- Document the behaviour of `LoggingTransaction.call_after` and `LoggingTransaction.call_on_exception` methods when transactions are retried. ([\matrix-org#12315](matrix-org#12315))
- Update dead links in `check-newsfragment.sh` to point to the correct documentation URL. ([\matrix-org#12331](matrix-org#12331))
- Upgrade the version of `mdbook` in CI to 0.4.17. ([\matrix-org#12339](matrix-org#12339))
- Updates to the Room DAG concepts development document to clarify that we mark events as outliers because we don't have any state for them. ([\matrix-org#12345](matrix-org#12345))
- Update the link to Redis pub/sub documentation in the workers documentation. ([\matrix-org#12369](matrix-org#12369))
- Remove documentation for converting a legacy structured logging configuration to the new format. ([\matrix-org#12392](matrix-org#12392))

Deprecations and Removals
-------------------------

- Remove the unused and unstable `/aggregations` endpoint which was removed from [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\matrix-org#12293](matrix-org#12293))

Internal Changes
----------------

- Remove lingering unstable references to MSC2403 (knocking). ([\matrix-org#12165](matrix-org#12165))
- Avoid trying to calculate the state at outlier events. ([\matrix-org#12191](matrix-org#12191), [\matrix-org#12316](matrix-org#12316), [\matrix-org#12330](matrix-org#12330), [\matrix-org#12332](matrix-org#12332), [\matrix-org#12409](matrix-org#12409))
- Omit sending "offline" presence updates to application services after they are initially configured. ([\matrix-org#12193](matrix-org#12193))
- Switch to using a sequence to generate AS transaction IDs. Contributed by Nick @ Beeper. If running synapse with a dedicated appservice worker, this MUST be stopped before upgrading the main process and database. ([\matrix-org#12209](matrix-org#12209))
- Add missing type hints for storage. ([\matrix-org#12267](matrix-org#12267))
- Add missing type definitions for scripts in docker folder. Contributed by Jorge Florian. ([\matrix-org#12280](matrix-org#12280))
- Move [MSC2654](matrix-org/matrix-spec-proposals#2654) support behind an experimental configuration flag. ([\matrix-org#12295](matrix-org#12295))
- Update docstrings to explain how to decipher live and historic pagination tokens. ([\matrix-org#12317](matrix-org#12317))
- Add ground work for speeding up device list updates for users in large numbers of rooms. ([\matrix-org#12321](matrix-org#12321))
- Fix typechecker problems exposed by signedjson 1.1.2. ([\matrix-org#12326](matrix-org#12326))
- Remove the `tox` packaging job: it will be redundant once matrix-org#11537 lands. ([\matrix-org#12334](matrix-org#12334))
- Ignore `.envrc` for `direnv` users. ([\matrix-org#12335](matrix-org#12335))
- Remove the (broadly unused, dev-only) dockerfile for pg tests. ([\matrix-org#12336](matrix-org#12336))
- Remove redundant `get_success` calls in test code. ([\matrix-org#12346](matrix-org#12346))
- Add type annotations for `tests/unittest.py`. ([\matrix-org#12347](matrix-org#12347))
- Move single-use methods out of `TestCase`. ([\matrix-org#12348](matrix-org#12348))
- Remove broken and unused development scripts. ([\matrix-org#12349](matrix-org#12349), [\matrix-org#12351](matrix-org#12351), [\matrix-org#12355](matrix-org#12355))
- Convert `Linearizer` tests from `inlineCallbacks` to async. ([\matrix-org#12353](matrix-org#12353))
- Update docstrings for `ReadWriteLock` tests. ([\matrix-org#12354](matrix-org#12354))
- Refactor `Linearizer`, convert methods to async and use an async context manager. ([\matrix-org#12357](matrix-org#12357))
- Fix a long-standing bug where `Linearizer`s could get stuck if a cancellation were to happen at the wrong time. ([\matrix-org#12358](matrix-org#12358))
- Make `StreamToken.from_string` and `RoomStreamToken.parse` propagate cancellations instead of replacing them with `SynapseError`s. ([\matrix-org#12366](matrix-org#12366))
- Add type hints to tests files. ([\matrix-org#12371](matrix-org#12371))
- Allow specifying the Postgres database's port when running unit tests with Postgres. ([\matrix-org#12376](matrix-org#12376))
- Remove temporary pin of signedjson<=1.1.1 that was added in Synapse 1.56.0. ([\matrix-org#12379](matrix-org#12379))
- Add opentracing spans to calls to external cache. ([\matrix-org#12380](matrix-org#12380))
- Lay groundwork for using `poetry` to manage Synapse's dependencies. ([\matrix-org#12381](matrix-org#12381), [\matrix-org#12407](matrix-org#12407), [\matrix-org#12412](matrix-org#12412), [\matrix-org#12418](matrix-org#12418))
- Make missing `importlib_metadata` dependency explicit. ([\matrix-org#12384](matrix-org#12384), [\matrix-org#12400](matrix-org#12400))
- Update type annotations for compatiblity with prometheus_client 0.14. ([\matrix-org#12389](matrix-org#12389))
- Remove support for the unstable identifiers specified in [MSC3288](matrix-org/matrix-spec-proposals#3288). ([\matrix-org#12398](matrix-org#12398))
- Add missing type hints to configuration classes. ([\matrix-org#12402](matrix-org#12402))
- Add files used to build the Docker image used for complement testing into the Synapse repository. ([\matrix-org#12404](matrix-org#12404))
- Do not include groups in the sync response when disabled. ([\matrix-org#12408](matrix-org#12408))
- Improve type hints related to HTTP query parameters. ([\matrix-org#12415](matrix-org#12415))
- Stop maintaining a list of lint targets. ([\matrix-org#12420](matrix-org#12420))
- Make `synapse._scripts` pass type checks. ([\matrix-org#12421](matrix-org#12421), [\matrix-org#12422](matrix-org#12422))
- Add some type hints to datastore. ([\matrix-org#12423](matrix-org#12423))
- Enable certificate checking during complement tests. ([\matrix-org#12435](matrix-org#12435))
- Explicitly specify the `tls` extra for Twisted dependency. ([\matrix-org#12444](matrix-org#12444))
@turt2live turt2live self-assigned this May 5, 2022
@turt2live turt2live mentioned this pull request May 14, 2022
Merged
4 tasks
@turt2live
Copy link
Member

Spec PR: matrix-org/matrix-spec#1062

@turt2live turt2live added spec-pr-in-review A proposal which has been PR'd against the spec and is in review and removed spec-pr-missing Proposal has been implemented and is being used in the wild but hasn't yet been added to the spec labels May 27, 2022
@turt2live turt2live added merged A proposal whose PR has merged into the spec! and removed spec-pr-in-review A proposal which has been PR'd against the spec and is in review labels Jun 8, 2022
@turt2live
Copy link
Member

Merged 🎉

@zecakeh zecakeh mentioned this pull request Jun 10, 2022
Closed
66 tasks
@clokep clokep deleted the clokep/bundled-aggregations-search branch May 17, 2023 12:22
@S7evinK S7evinK mentioned this pull request Sep 29, 2023
Open
16 tasks
@matrixbot matrixbot mentioned this pull request Nov 1, 2024
Open
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richvdh richvdh richvdh left review comments

@anoadragon453 anoadragon453 anoadragon453 approved these changes

@turt2live turt2live turt2live left review comments

Assignees

@turt2live turt2live

Labels
client-server Client-Server API kind:feature MSC for not-core and not-maintenance stuff merged A proposal whose PR has merged into the spec! proposal A matrix spec change proposal
Projects
Spec Core Team Workflow
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@clokep @richvdh @turt2live @erikjohnston @mscbot @anoadragon453