feat: user sync stream

[shenlong-tanwen]

Description

• Handles user updates and deletions through the sync stream flow
• Handles partner updates and deletions through the sync stream flow

[alextran1502]

Hi @shenlong-tanwen, I added a button to trigger the syncUser event to see how it works. I notice that the isolate isn't always triggered; I expect that if you log out and log back in, the sync mechanism will trigger again when you press the button. But it only works again after I hot restart the app. I think it might have to be something with the .fetch method in _userSyncCache.fetch. Can you check this?

[shenlong-tanwen]

I think it might have to be something with the .fetch method in _userSyncCache.fetch

Yes, the only thing that could prevent us from creating another isolate is the AsyncCache. It basically ensures that we start a new isolate only after the completion of the previous one. In your case, I suspect the previous sync was in progress while you logged out and logged back in. However, it might be a good idea to have an option to stop the previous isolate when the user logs out and start a new once after they log in. I'll handle this soon. For now, I've added some additional logs to be printed during the sync process. This should help you confirm whether the previous one has stopped before you invoke the next sync

[mertalev]

Is there no .flush() you could call or something?

[shenlong-tanwen]

There is one but that can be used only when the logs are buffered. We do not buffer logs on isolates so it is not much useful here. Also, there is an issue with the flush logic currently with how Isar works and so it is disabled for now as well - #16599

[mertalev]

Why disable buffering?

[shenlong-tanwen]

On main isolate, we buffer logs and flush them every 5 seconds. However, an isolate does not need to necessarily run for 5 seconds. There is also a chance that an isolate might be killed before the logs are updated, so we disable buffering all together within the isolates

[mertalev]

I think we should avoid dynamic to the extent possible. Ideally, only the JSON parsing would involve dynamic, and parse directly into strongly typed objects that share a type (either through an interface, by inheritance or maybe through generics). As it is, dynamic leaks into the events, the response converter map, the main sync handler, etc.

[shenlong-tanwen]

True, we went with having strongly typed DTOs initially. But it resulted in us creating a lot of duplicate domain classes that were already available from the OpenAPI generated code. The generated codes are not strongly typed so we do not have a common base class that we could use here.

[mertalev]

Between the grouping by type, the data mapping here and only the last ack being called, the shape of the sync events seems wrong. Shouldn't the event be a type, an array of data and a single ack to begin with?

[shenlong-tanwen]

Each event has a data, it's type and it's ack. If we handle each event separately, we would store and send only the ack of the last successful operation for a particular type, but since we group similar events and handle them in batch, we had to do them this way.