[build-script] Allow to tune dsymutil parallelism #34795
Conversation
This should enable scaling when using machines with large amount of RAM. To better support machines with lower spec, process one binary per dsymutil invocation (reverting swiftlang#34149). Add some (limited) facilities to gather the time taken to execute dsymutil to better assist in tuning the parameter. Addresses rdar://71018443
edymtt
requested review from
compnerd,
gottesmm,
JDevlieghere and
shahmishal
|
@swift-ci please smoke test |
|
@swift-ci please build toolchain |
|
@swift-ci Please Test Source Compatibility Release |
| # Run sequentially -- dsymutil is multithreaded and can be memory intensive | ||
| # Tweak carefully the amount of parallelism -- dsymutil can be memory intensive and | ||
| # as such too many instance can exhaust the memory and slow down/panic the machine | ||
| printJSONStartTimestamp dsymutil |
There was a problem hiding this comment.
I recognize this is not strictly required to add the parameter, but it seemed to make sense given we are now able to tune this in a more direct way
|
Linux Toolchain (Ubuntu 16.04) Install command |
|
macOS Toolchain Install command |
|
@swift-ci please test |
|
@swift-ci please build toolchain |
|
@swift-ci Please Test Source Compatibility Release |
|
Linux Toolchain (Ubuntu 16.04) Install command |
|
macOS Toolchain Install command |
|
@swift-ci please smoke test |
|
@swift-ci please build toolchain |
|
@swift-ci Please Test Source Compatibility Release |
|
@swift-ci please smoke test Linux |
|
Linux Toolchain (Ubuntu 16.04) Install command |
|
@swift-ci please build toolchain macOS |
|
@swift-ci please smoke test Linux |
|
@swift-ci please build toolchain macOS |
|
@swift-ci please smoke test Linux |
|
@swift-ci please build toolchain macOS |
1 similar comment
|
@swift-ci please build toolchain macOS |
|
@swift-ci Please Test Source Compatibility Release |
|
macOS Toolchain Install command |
|
@swift-ci Please Test Source Compatibility Release |
1 similar comment
|
@swift-ci Please Test Source Compatibility Release |
* 'main' of github.com:apple/swift: (67 commits) [build-script] Allow to tune dsymutil parallelism (swiftlang#34795) [Testing] Add missing REQUIRES [concurrency] SILGen: emit @asyncHandler functions. [concurrency] SILGen: allow the Builtin.createAsyncTaskFuture to have a non-generic closure argument. [concurrency] stdlib: add a _runAsyncHandler compiler intrinsic. Mangling: add support for mangling the body-function of asyncHandlers Make sure ~AutoDiffLinearMapContext() is called. fix SourceLoc-related crasher and add tests [AutoDiff] Bump-pointer allocate pullback structs in loops. (swiftlang#34886) update differentiable programming manifesto [Async CC] Always add full type metadata to bindings. [cxx-interop] Fix assertion to allow variadic members. [ome] Remove bad pattern of having a global SILBuilder with a global SILBuilderWithContext and multiple local SILBuilderWithScope. [ome] Invoke simplifyInstruction after lowering ownership and use replaceAllSimplifiedUsesAndErase instead of a manual RAUW. Partially revert Float16 availability changes (swiftlang#34847) Add a field reflection function that constructs keypaths. (swiftlang#34815) Allow the creation of a shadow variable when the type is a refcounted pointer (swiftlang#34835) [CMake] Extend copy-legacy-layouts dependency to swiftmodules (swiftlang#34846) [sil] Remove usage from TypeLowering of SILBuilder::create*AndFold(). [allocbox-to-stack] Fix an ossa bug in PromotedParamCloner. ...
This should enable scaling when using machines with large amount of
RAM.
To better support machines with lower spec, process one binary per
dsymutil invocation (reverting #34149).
Add some (limited) facilities to gather the time taken to execute
dsymutil to better assist in tuning the parameter -- these are printed in
JSON format in the log to allow for easier scraping
Addresses rdar://71018443