Skip to content

addTypeToIntersection performance improvement #32388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 17, 2019

Conversation

miloszpp
Copy link
Contributor

Fixes #29289

Changes the type of types argument from Type[] to Map<Type> so that membership checking can be done in O(1) instead of O(n).

@miloszpp
Copy link
Contributor Author

@typescript-bot perf test

@orta
Copy link
Contributor

orta commented Jul 15, 2019

Maybe I can run it

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 15, 2019

Heya @orta, I've started to run the perf test suite on this PR at 8871463. You can monitor the build here. It should now contribute to this PR's status checks.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@orta
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..32388

Metric master 32388 Delta Best Worst
Angular - node (v12.1.0, x64)
Memory used 318,287k (± 0.01%) 316,249k (± 0.02%) -2,039k (- 0.64%) 316,034k 316,396k
Parse Time 1.39s (± 0.70%) 1.38s (± 0.56%) -0.01s (- 0.72%) 1.37s 1.40s
Bind Time 0.74s (± 0.60%) 0.73s (± 0.41%) -0.01s (- 0.81%) 0.73s 0.74s
Check Time 4.18s (± 0.39%) 4.15s (± 0.34%) -0.04s (- 0.91%) 4.11s 4.18s
Emit Time 5.24s (± 0.45%) 5.24s (± 0.77%) +0.00s (+ 0.04%) 5.17s 5.32s
Total Time 11.55s (± 0.28%) 11.50s (± 0.47%) -0.05s (- 0.40%) 11.39s 11.63s
Monaco - node (v12.1.0, x64)
Memory used 345,738k (± 0.01%) 345,697k (± 0.02%) -41k (- 0.01%) 345,553k 345,794k
Parse Time 1.17s (± 0.71%) 1.18s (± 0.52%) +0.00s (+ 0.34%) 1.16s 1.19s
Bind Time 0.67s (± 0.99%) 0.67s (± 0.71%) -0.00s (- 0.00%) 0.67s 0.69s
Check Time 4.22s (± 0.57%) 4.22s (± 0.49%) +0.00s (+ 0.09%) 4.17s 4.27s
Emit Time 2.86s (± 0.69%) 2.85s (± 0.40%) -0.01s (- 0.21%) 2.83s 2.88s
Total Time 8.93s (± 0.52%) 8.93s (± 0.33%) -0.00s (- 0.01%) 8.85s 8.98s
TFS - node (v12.1.0, x64)
Memory used 301,347k (± 0.02%) 301,237k (± 0.02%) -110k (- 0.04%) 301,149k 301,341k
Parse Time 0.91s (± 0.36%) 0.91s (± 0.54%) -0.00s (- 0.44%) 0.90s 0.92s
Bind Time 0.62s (± 1.05%) 0.61s (± 0.80%) -0.00s (- 0.65%) 0.61s 0.63s
Check Time 3.77s (± 0.45%) 3.79s (± 0.79%) +0.02s (+ 0.50%) 3.74s 3.88s
Emit Time 2.94s (± 0.77%) 2.94s (± 0.78%) +0.00s (+ 0.14%) 2.89s 2.99s
Total Time 8.24s (± 0.39%) 8.25s (± 0.60%) +0.01s (+ 0.15%) 8.17s 8.38s
Angular - node (v8.9.0, x64)
Memory used 336,754k (± 0.01%) 334,661k (± 0.01%) -2,094k (- 0.62%) 334,515k 334,759k
Parse Time 1.78s (± 0.19%) 1.78s (± 0.52%) -0.01s (- 0.28%) 1.76s 1.81s
Bind Time 0.80s (± 0.62%) 0.80s (± 0.65%) -0.00s (- 0.50%) 0.79s 0.81s
Check Time 4.95s (± 1.33%) 4.95s (± 1.25%) +0.00s (+ 0.10%) 4.83s 5.10s
Emit Time 6.05s (± 1.75%) 5.99s (± 2.03%) -0.06s (- 0.99%) 5.68s 6.29s
Total Time 13.59s (± 0.65%) 13.53s (± 0.71%) -0.06s (- 0.44%) 13.21s 13.74s
Monaco - node (v8.9.0, x64)
Memory used 363,072k (± 0.02%) 362,994k (± 0.02%) -78k (- 0.02%) 362,867k 363,249k
Parse Time 1.44s (± 0.52%) 1.43s (± 0.39%) -0.00s (- 0.35%) 1.42s 1.44s
Bind Time 0.89s (± 2.28%) 0.90s (± 2.41%) +0.01s (+ 0.90%) 0.86s 0.94s
Check Time 5.16s (± 1.90%) 5.14s (± 1.66%) -0.02s (- 0.43%) 5.00s 5.27s
Emit Time 3.15s (± 6.41%) 3.21s (± 6.10%) +0.06s (+ 1.90%) 2.92s 3.50s
Total Time 10.64s (± 1.32%) 10.68s (± 1.26%) +0.05s (+ 0.43%) 10.44s 10.90s
TFS - node (v8.9.0, x64)
Memory used 317,039k (± 0.01%) 316,998k (± 0.02%) -41k (- 0.01%) 316,877k 317,086k
Parse Time 1.14s (± 0.62%) 1.14s (± 0.68%) +0.00s (+ 0.00%) 1.12s 1.16s
Bind Time 0.66s (± 0.72%) 0.66s (± 0.79%) -0.00s (- 0.30%) 0.65s 0.67s
Check Time 4.42s (± 0.87%) 4.40s (± 0.55%) -0.01s (- 0.27%) 4.32s 4.45s
Emit Time 3.20s (± 2.36%) 3.19s (± 2.52%) -0.00s (- 0.13%) 2.97s 3.30s
Total Time 9.41s (± 1.10%) 9.39s (± 1.03%) -0.02s (- 0.22%) 9.13s 9.52s
Angular - node (v8.9.0, x86)
Memory used 190,792k (± 0.04%) 189,719k (± 0.03%) -1,073k (- 0.56%) 189,618k 189,822k
Parse Time 1.73s (± 0.43%) 1.73s (± 0.76%) +0.00s (+ 0.12%) 1.70s 1.77s
Bind Time 0.94s (± 0.53%) 0.93s (± 0.88%) -0.00s (- 0.32%) 0.92s 0.96s
Check Time 4.52s (± 0.48%) 4.51s (± 0.46%) -0.01s (- 0.27%) 4.45s 4.55s
Emit Time 5.75s (± 1.02%) 5.77s (± 1.41%) +0.02s (+ 0.31%) 5.64s 5.98s
Total Time 12.93s (± 0.50%) 12.93s (± 0.72%) +0.00s (+ 0.03%) 12.77s 13.15s
Monaco - node (v8.9.0, x86)
Memory used 202,703k (± 0.02%) 202,709k (± 0.03%) +7k (+ 0.00%) 202,585k 202,801k
Parse Time 1.49s (± 0.49%) 1.49s (± 0.65%) -0.00s (- 0.07%) 1.47s 1.51s
Bind Time 0.72s (± 0.97%) 0.72s (± 1.37%) -0.00s (- 0.42%) 0.70s 0.75s
Check Time 4.83s (± 0.62%) 4.83s (± 0.68%) +0.01s (+ 0.15%) 4.79s 4.95s
Emit Time 3.20s (± 0.96%) 3.17s (± 0.54%) -0.02s (- 0.75%) 3.13s 3.21s
Total Time 10.24s (± 0.49%) 10.22s (± 0.42%) -0.02s (- 0.22%) 10.13s 10.34s
TFS - node (v8.9.0, x86)
Memory used 178,072k (± 0.02%) 177,993k (± 0.02%) -79k (- 0.04%) 177,897k 178,057k
Parse Time 1.20s (± 0.82%) 1.19s (± 0.56%) -0.01s (- 0.67%) 1.18s 1.21s
Bind Time 0.63s (± 1.18%) 0.63s (± 0.75%) -0.00s (- 0.47%) 0.62s 0.64s
Check Time 4.24s (± 0.75%) 4.23s (± 0.56%) -0.01s (- 0.33%) 4.18s 4.29s
Emit Time 2.86s (± 1.06%) 2.83s (± 0.85%) -0.03s (- 0.94%) 2.76s 2.88s
Total Time 8.94s (± 0.55%) 8.89s (± 0.43%) -0.05s (- 0.55%) 8.81s 8.97s
Angular - node (v9.0.0, x64)
Memory used 336,284k (± 0.02%) 334,365k (± 0.02%) -1,919k (- 0.57%) 334,223k 334,505k
Parse Time 1.63s (± 0.64%) 1.62s (± 0.62%) -0.01s (- 0.43%) 1.60s 1.64s
Bind Time 0.74s (± 1.11%) 0.74s (± 0.75%) -0.00s (- 0.27%) 0.73s 0.76s
Check Time 4.53s (± 1.27%) 4.51s (± 1.43%) -0.02s (- 0.42%) 4.38s 4.72s
Emit Time 5.76s (± 1.42%) 5.79s (± 2.35%) +0.03s (+ 0.52%) 5.45s 6.03s
Total Time 12.67s (± 0.60%) 12.66s (± 1.04%) -0.00s (- 0.02%) 12.37s 12.93s
Monaco - node (v9.0.0, x64)
Memory used 362,847k (± 0.03%) 362,907k (± 0.03%) +59k (+ 0.02%) 362,543k 363,023k
Parse Time 1.29s (± 0.68%) 1.28s (± 0.35%) -0.01s (- 0.93%) 1.27s 1.29s
Bind Time 0.86s (± 0.69%) 0.86s (± 0.69%) 0.00s ( 0.00%) 0.84s 0.87s
Check Time 4.84s (± 0.53%) 4.84s (± 0.54%) -0.01s (- 0.12%) 4.79s 4.90s
Emit Time 3.36s (± 0.36%) 3.35s (± 0.57%) -0.01s (- 0.27%) 3.31s 3.39s
Total Time 10.34s (± 0.35%) 10.32s (± 0.35%) -0.03s (- 0.26%) 10.23s 10.41s
TFS - node (v9.0.0, x64)
Memory used 316,834k (± 0.02%) 316,839k (± 0.02%) +5k (+ 0.00%) 316,750k 316,963k
Parse Time 1.01s (± 0.81%) 1.01s (± 0.22%) -0.00s (- 0.39%) 1.00s 1.01s
Bind Time 0.61s (± 0.60%) 0.61s (± 0.65%) -0.00s (- 0.65%) 0.60s 0.62s
Check Time 4.37s (± 1.46%) 4.32s (± 0.72%) -0.05s (- 1.08%) 4.26s 4.41s
Emit Time 3.15s (± 2.57%) 3.20s (± 0.46%) +0.05s (+ 1.72%) 3.15s 3.23s
Total Time 9.14s (± 0.42%) 9.14s (± 0.37%) -0.00s (- 0.03%) 9.08s 9.24s
Angular - node (v9.0.0, x86)
Memory used 190,912k (± 0.02%) 189,821k (± 0.03%) -1,091k (- 0.57%) 189,742k 189,984k
Parse Time 1.55s (± 0.84%) 1.54s (± 0.65%) -0.00s (- 0.13%) 1.52s 1.56s
Bind Time 0.88s (± 0.87%) 0.87s (± 0.78%) -0.00s (- 0.11%) 0.86s 0.89s
Check Time 4.22s (± 0.40%) 4.20s (± 0.36%) -0.01s (- 0.36%) 4.17s 4.23s
Emit Time 5.45s (± 0.42%) 5.48s (± 1.12%) +0.03s (+ 0.57%) 5.42s 5.70s
Total Time 12.09s (± 0.28%) 12.10s (± 0.67%) +0.02s (+ 0.13%) 11.98s 12.37s
Monaco - node (v9.0.0, x86)
Memory used 202,684k (± 0.02%) 202,645k (± 0.02%) -39k (- 0.02%) 202,569k 202,741k
Parse Time 1.31s (± 0.67%) 1.31s (± 0.58%) -0.00s (- 0.31%) 1.29s 1.33s
Bind Time 0.64s (± 1.04%) 0.64s (± 0.75%) -0.00s (- 0.47%) 0.63s 0.65s
Check Time 4.67s (± 0.42%) 4.63s (± 0.52%) -0.04s (- 0.81%) 4.58s 4.68s
Emit Time 3.13s (± 1.09%) 3.12s (± 0.82%) -0.01s (- 0.26%) 3.08s 3.18s
Total Time 9.75s (± 0.50%) 9.70s (± 0.31%) -0.05s (- 0.52%) 9.64s 9.78s
TFS - node (v9.0.0, x86)
Memory used 178,119k (± 0.02%) 178,019k (± 0.02%) -100k (- 0.06%) 177,937k 178,065k
Parse Time 1.02s (± 0.87%) 1.02s (± 0.54%) -0.00s (- 0.29%) 1.01s 1.03s
Bind Time 0.58s (± 0.63%) 0.57s (± 0.58%) -0.01s (- 1.04%) 0.56s 0.58s
Check Time 4.10s (± 0.40%) 4.08s (± 0.72%) -0.02s (- 0.49%) 4.02s 4.16s
Emit Time 2.79s (± 2.14%) 2.77s (± 0.70%) -0.03s (- 0.93%) 2.71s 2.80s
Total Time 8.49s (± 0.71%) 8.44s (± 0.44%) -0.06s (- 0.66%) 8.36s 8.55s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-142-generic
Architecturex64
Available Memory16 GB
Available Memory1 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
  • node (v9.0.0, x64)
  • node (v9.0.0, x86)
Scenarios
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Angular - node (v9.0.0, x64)
  • Angular - node (v9.0.0, x86)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • Monaco - node (v9.0.0, x64)
  • Monaco - node (v9.0.0, x86)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • TFS - node (v9.0.0, x64)
  • TFS - node (v9.0.0, x86)
Benchmark Name Iterations
Current 32388 10
Baseline master 10

@miloszpp
Copy link
Contributor Author

Thanks @orta.

Do I need to ping someone to review the PR?

@orta
Copy link
Contributor

orta commented Jul 16, 2019

Nah, someone who knows the are of the codebase better than I will come along and review - you're in a good spot.

const flags = type.flags;
if (flags & TypeFlags.Intersection) {
return addTypesToIntersection(typeSet, includes, (<IntersectionType>type).types);
}
if (isEmptyAnonymousObjectType(type)) {
if (!(includes & TypeFlags.IncludesEmptyObject)) {
includes |= TypeFlags.IncludesEmptyObject;
typeSet.push(type);
typeSet.set(type.id.toString(), type);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use the getTypeId function here, since IIRC, type IDs are lazily assigned.

@miloszpp
Copy link
Contributor Author

Thanks @weswigham. I've made the change.

BTW, the implementation of getTypeId is simply type.id :)

@weswigham
Copy link
Member

BTW, the implementation of getTypeId is simply type.id :)

🤦‍♂ I'm thinking of getSymbolId, which is deferred - no need to use it.

@miloszpp
Copy link
Contributor Author

@weswigham I'm not sure I understand - what do you mean by deferred?

Also, getSymbolId accepts a Symbol not a Type.

Should I revert my latest change?

@weswigham
Copy link
Member

Yeah, just revert the original change I asked for - I was incorrect.

@miloszpp
Copy link
Contributor Author

@weswigham done.

@weswigham
Copy link
Member

Thanks - the perf results are promising on our general suites (with results ranging from neutral to slightly positive), but I'm going to hold off merging for a second because I want to write up a microbenchmark where we can definitively see the change on (because I was once told in the past that this would be a premature optimization and that the memory pressure of swapping between an array and a map would probably make it a wash). I'll put one together shortly...

@weswigham
Copy link
Member

Ooof, nice - at a glance (haven't done the math/many iterations/clean machine), but on my microbenchmark with a 10000 element intersection (of unique but structurally identical object types), check time is dropped in half with this change. Nice.

@weswigham
Copy link
Member

weswigham commented Jul 17, 2019

Yep, the numbers look real good in microbenchmarks. On master, I've collected these timings from my laptop over 50 iterations per datum (where the datum is the number of intersection elements in a simple type Big = {x: number} & {x: number} ...):

Datum: 10
Min: 0.02
Max: 0.03
Avg: 0.02
Datum: 100
Min: 0.02
Max: 0.05
Avg: 0.03
Datum: 1000
Min: 0.09
Max: 0.13
Avg: 0.10
Datum: 10000
Min: 1.12
Max: 1.73
Avg: 1.26
Datum: 100000
Min: 92.15
Max: 98.34
Avg: 94.91

(a 10x increase in size is a 100x increase in time after a point)

While with this simple fix:

Datum: 10
Min: 0.02
Max: 0.08
Avg: 0.03
Datum: 100
Min: 0.03
Max: 0.1
Avg: 0.04
Datum: 1000
Min: 0.09
Max: 0.16
Avg: 0.11
Datum: 10000
Min: 0.47
Max: 0.93
Avg: 0.57
Datum: 100000
Min: 1.47
Max: 1.85
Avg: 1.67

With 1000 or fewer elements, the timings are similar, but above that... on master the perf is significantly worse.

@weswigham weswigham merged commit 8f2ed0d into microsoft:master Jul 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

addTypeToIntersection shouldn't use contains
4 participants