Cleanup "libraries" field in DillTarget when their invalidated

Before this CL we removed from from the builders map, but left them in
the libraries list.
When using the incremental compiler via kernel worker (where we
continually replace dill builders) that meant that this list would just
keep growing.

This CL cleans it up and the builders map and the list should now be
in sync

An internal benchmark via kernel worker of lots of outline
calculations in worker mode with reuse and the incremental compiler
goes from ~215 seconds to ~185 seconds.

Change-Id: I254558d517f544fe31b5673e0dc8748048f0c5ba
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/103630
Reviewed-by: Kevin Millikin <[email protected]>

Author: jensjoha

pkg/front_end/lib/src/fasta/incremental_compiler.dart

@@ -220,11 +220,19 @@ class IncrementalCompiler implements IncrementalKernelGenerator {
           notReused: notReusedLibraries);
       Set<Uri> reusedLibraryUris =
           new Set<Uri>.from(reusedLibraries.map((b) => b.uri));
+      bool removedBuilders = false;
       for (Uri uri in new Set<Uri>.from(dillLoadedData.loader.builders.keys)
         ..removeAll(reusedLibraryUris)) {
         LibraryBuilder builder = dillLoadedData.loader.builders.remove(uri);
         userBuilders?.remove(uri);
         CompilerContext.current.uriToSource.remove(builder.fileUri);
+        removedBuilders = true;
+      }
+      if (removedBuilders) {
+        dillLoadedData.loader.libraries.clear();
+        for (LibraryBuilder builder in dillLoadedData.loader.builders.values) {
+          dillLoadedData.loader.libraries.add(builder.target);
+        }
       }
 
       if (hasToCheckPackageUris) {