-
Notifications
You must be signed in to change notification settings - Fork 12.9k
Clean up outdated string comparison logic #19452
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
src/compiler/program.ts
Outdated
@@ -1105,7 +1105,8 @@ namespace ts { | |||
return compareStrings(file.fileName, getDefaultLibraryFileName(), /*ignoreCase*/ !host.useCaseSensitiveFileNames()) === Comparison.EqualTo; | |||
} | |||
else { | |||
return forEach(options.lib, libFileName => compareStrings(file.fileName, combinePaths(defaultLibraryPath, libFileName), /*ignoreCase*/ !host.useCaseSensitiveFileNames()) === Comparison.EqualTo); | |||
const stringComparer = host.useCaseSensitiveFileNames() ? compareStringsCaseSensitive : compareStringsCaseInsensitive; | |||
return forEach(options.lib, libFileName => stringComparer(file.fileName, combinePaths(defaultLibraryPath, libFileName)) === Comparison.EqualTo); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this really depend on the locale the compiler is run in? We should avoid a situation where there are compiler errors in some countries but not others.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's already how we do it today. In general we should use the sorting rules of the system locale, as that is most pleasing to the end user. In this case, differing locales could affect compile-time behavior. In .NET I would leverage the "invariant culture" for such things, but there is no such concept in Intl
. We could consider using "en-US"
as the "invariant culture", in which case we would want a compareStringsUI
that uses the default culture for things like sorting diagnostic messages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that "en-US"
most likely won't correctly handle Turkish i on a case-insensitive file system. "en-US"
treats i = I
, whereas in a Turkish locale, i ≠ I
but rather i = İ
and ı = I
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not a sorting issue but an equality issue. Why not just use exact equality in the case-sensitive case and JS's builtin .toLowerCase() in the case-insensitive case? Or look at what the windows file system does, since I think that's the only one that matters for the case-insensitive case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
toLowerCase
does not correctly handle ẞ
(German uppercase sharp s), see https://en.wikipedia.org/wiki/Capital_%E1%BA%9E for reference.
Perhaps we should have an equalStrings
function for equality cases. Its still generally better to leverage Intl
than to perform a strict ordinal comparison as there exists the possibility that two unicode characters are still considered equal (such as cases where there exists both a single character form as well as a multi-character form that uses combining diacritic markers). If we added an equalStrings
function for that case, then we would use an Intl.Collator
whose usage
is "search"
rather than "sort"
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
HFS on darwin is also case-insensitive.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The most reliable way to compare two physical paths that point to actual files would be to perform an fs.statSync
on both paths and compare the dev
, rdev
, and ino
values. However, a stat is expensive.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did some performance comparisons of CS and CI comparisons using Intl.Collator
, localeCompare
, and ordinal comparison. While ordinal wins handily in all cases (which is expected), Intl.Collator
is oddly slower than localeCompare for CS (which is unexpected given there is no transformation).
While there may be some benefit to continuing to use Intl.Collator
or localeCompare
for case-sensitive comparisons, we don't do that today so I'll revert that part of this PR.
src/services/navigateTo.ts
Outdated
@@ -176,11 +176,8 @@ namespace ts.NavigateTo { | |||
function compareNavigateToItems(i1: RawNavigateToItem, i2: RawNavigateToItem): number { | |||
// TODO(cyrusn): get the gamut of comparisons that VS already uses here. | |||
// Right now we just sort by kind first, and then by name of the item. | |||
// We first sort case insensitively. So "Aaa" will come before "bar". |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
compareStringsCaseSensitive
already sorts this correctly. It is possible this was written to address the Node v0.10.0 issue with String.prototype.localeCompare
, but it no longer seems necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After reverting compareStringsCaseSensitive
to just use an ordinal relational comparison, it seems that "aaa"
is once again greater than "Aaa"
(due to the fact the char code for "a" is 97 while the char code for "A" is 65).
@Andy-MS, any other comments? |
src/compiler/core.ts
Outdated
@@ -656,13 +650,13 @@ namespace ts { | |||
} | |||
|
|||
// TODO: fixme (N^2) - add optional comparer so collection can be sorted before deduplication. | |||
export function deduplicate<T>(array: ReadonlyArray<T>, areEqual?: (a: T, b: T) => boolean): T[] { | |||
export function deduplicate<T>(array: ReadonlyArray<T>, equalityComparer: (a: T, b: T) => boolean = equateValues): T[] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the TODO
fixed? We have a comparer and we ensure it always has a value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. I didn't add this TODO, but I believe it is talking about a Comparer<T>
to sort the array, not the EqualityComparer<T>
which just checks for equivalence. I'm not planning on making that change in this PR.
src/compiler/core.ts
Outdated
/** | ||
* Compare two values for their equality. | ||
*/ | ||
export function equateValues<T>(a: T | undefined, b: T | undefined) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: Accepting T | undefined
for an arbitrary T
is redundant because T
could already be a union type.
src/compiler/core.ts
Outdated
} | ||
|
||
// Gets string comparers compatible with the current host | ||
function createCaseInsensitiveStringCollator() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: Only called once immediately after its definition, just use an IIFE?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks cleaner without the IIFE.
src/compiler/core.ts
Outdated
* If supported by the host, the default locale is used for comparisons. Otherwise, an ordinal | ||
* comparison is used. | ||
*/ | ||
export function compareStringsCaseInsensitive(a: string | undefined, b: string | undefined) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mention in the name of the method that this is locale-sensitive.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
src/compiler/core.ts
Outdated
* If supported by the host, the default locale is used for comparisons. Otherwise, an ordinal | ||
* comparison is used. | ||
*/ | ||
export function equateStringsCaseInsensitive(a: string | undefined, b: string | undefined) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When does this have a different result from ===
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its a case-insensitive comparison, so when comparing Aaa
to aaa
, etc.
src/compiler/core.ts
Outdated
/** | ||
* Performs a case-sensitive equality comparison between two strings. | ||
*/ | ||
export function equateStringsCaseSensitive(a: string | undefined, b: string | undefined) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
export const equateStringsCaseSensitive: (a: string | undefined, b: string | undefined) => boolean = equateValues;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
src/compiler/core.ts
Outdated
return equateValues(a, b); | ||
} | ||
|
||
export function equateStrings(a: string | undefined, b: string | undefined, ignoreCase?: boolean) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make ignoreCase
mandatory, else we should be using ===
directly.
@@ -1904,8 +2003,9 @@ namespace ts { | |||
const aComponents = getNormalizedPathComponents(a, currentDirectory); | |||
const bComponents = getNormalizedPathComponents(b, currentDirectory); | |||
const sharedLength = Math.min(aComponents.length, bComponents.length); | |||
const comparer = ignoreCase ? compareStringsCaseInsensitive : compareStringsCaseSensitive; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be easier to read if you just called compareStrings
inside the loop -- then we wouldn't need a compareStringsCaseSensitive
function and could just use ===
directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's what this did before. However, that means we are evaluating the ignoreCase
condition for each iteration of the loop. This saves a few cycles.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would think that an indirect function call isn't any faster than an if?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've actually removed compareStrings
now. Comparisons need to be explicit and intentional:
compareValues
now only compares numberscompareStringsCaseSensitive
compares strings using an ordinal case-sensitive comparisoncompareStringsCaseSensitiveUI
compares strings using a dictionary-ordered case-sensitive comparison using the current UI localecompareStringsCaseInsensitive
compares strings using an ordinal case-insensitive comparisoncompareStringsCaseInsensitiveUI
compares strings using a dictionary-ordered case-insensitive comparison using the current UI locale
src/compiler/parser.ts
Outdated
@@ -6069,7 +6069,7 @@ namespace ts { | |||
const checkJsDirectiveMatchResult = checkJsDirectiveRegEx.exec(comment); | |||
if (checkJsDirectiveMatchResult) { | |||
checkJsDirective = { | |||
enabled: compareStrings(checkJsDirectiveMatchResult[1], "@ts-check", /*ignoreCase*/ true) === Comparison.EqualTo, | |||
enabled: equateStrings(checkJsDirectiveMatchResult[1], "@ts-check", /*ignoreCase*/ true), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can just call equateStringsCaseInsensitive
directly.
But this case it might make more sense to just use ASCII capitalization (.toLowerCase()
) instead of anything locale-sensitive.
src/compiler/core.ts
Outdated
/** | ||
* Compare two values for their equality. | ||
*/ | ||
export function equateValues<T>(a: T, b: T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not used as an export
src/compiler/core.ts
Outdated
&& a.toUpperCase() === b.toUpperCase(); | ||
} | ||
|
||
export function equateStringsCaseSensitive(a: string, b: string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Export not used
src/harness/harness.ts
Outdated
@@ -1698,7 +1698,8 @@ namespace Harness { | |||
|
|||
export function *iterateOutputs(outputFiles: Harness.Compiler.GeneratedFile[]): IterableIterator<[string, string]> { | |||
// Collect, test, and sort the fileNames | |||
outputFiles.sort((a, b) => ts.compareStrings(cleanName(a.fileName), cleanName(b.fileName))); | |||
const comparer = ts.getStringComparer(/*ignoreCase*/ false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just use compareStringsCaseInsensitive
directly
@@ -13,8 +13,9 @@ namespace ts.projectSystem { | |||
describe("CompileOnSave affected list", () => { | |||
function sendAffectedFileRequestAndCheckResult(session: server.Session, request: server.protocol.Request, expectedFileList: { projectFileName: string, files: FileOrFolder[] }[]) { | |||
const response = session.executeCommand(request).response as server.protocol.CompileOnSaveAffectedFileListSingleProject[]; | |||
const actualResult = response.sort((list1, list2) => compareStrings(list1.projectFileName, list2.projectFileName)); | |||
expectedFileList = expectedFileList.sort((list1, list2) => compareStrings(list1.projectFileName, list2.projectFileName)); | |||
const comparer = getStringComparer(/*ignoreCase*/ false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just use compareStringsCaseInsensitive
directly
src/compiler/core.ts
Outdated
return comparer(a, b); | ||
} | ||
|
||
export function compareStringsCaseSensitiveUI(a: string, b: string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is called CaseSensitive
but /*caseInsensitive*/ true
?
return i1.matchKind - i2.matchKind || | ||
ts.compareStringsCaseInsensitive(i1.name, i2.name) || | ||
ts.compareStrings(i1.name, i2.name); | ||
return compareValues(i1.matchKind, i2.matchKind) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may run inside of a unit test, causing that test to fail for developers from other countries...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've added a beforeEach
/afterEach
that explicitly sets the UI locale to "en-US"
before running each test.
src/compiler/core.ts
Outdated
* @param comparer An optional `Comparer` used to sort entries before comparison. If supplied, | ||
* results are returned in the original order found in `array`. | ||
*/ | ||
export function deduplicate<T>(array: ReadonlyArray<T>, equalityComparer?: EqualityComparer<T>, comparer?: Comparer<T>): T[] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You could make equalityComparer
and comparer
required. This causes in error in combineProjectOutput
, which happens to have a comparer
handy and have sorted the array already prior to deduplicating -- it should probably use toDeduplicatedSortedArray
.
src/compiler/core.ts
Outdated
@@ -837,13 +863,28 @@ namespace ts { | |||
} | |||
|
|||
/** | |||
* Creates an array of integers starting at `from` (inclusive) and ending at `to` (exclusive). | |||
*/ | |||
function sequence(from: number, to: number) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could use function indices(array: any[]): number[] { return array.map((_, i) => i); }
src/compiler/core.ts
Outdated
stableSortIndices(array, indices, comparer); | ||
|
||
const deduplicated: number[] = []; | ||
loop: for (const sourceIndex of indices) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I might be missing something, but it seems like for an array with no duplicates this will do O(n^2) work? We loop over indices
which has the same length as the input, then we loop over deduplicated
which will have on average length n/2
, and with no duplicates the continue
doesn't matter.
src/compiler/core.ts
Outdated
} | ||
})(); | ||
|
||
let uiCS: Comparer<string> | undefined; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: use a longer name. Also, might want to put these in an object to ensure they are all set at once.
Also, might as well just set uiCS
and uiCI
eagerly at the call to setUILocale
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather they were not set at once or created eagerly, I'd prefer these to be created lazily as needed (since they may not be needed).
src/compiler/core.ts
Outdated
@@ -1926,9 +2185,10 @@ namespace ts { | |||
return false; | |||
} | |||
|
|||
// File-system comparisons should use predictable ordering |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment isn't going to make sense in this context unless you mention that this is contrasted with using locale-sensitive operations.
src/compiler/types.ts
Outdated
} | ||
|
||
/* @internal */ | ||
export type Selector<T, U> = (v: T) => U; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not so hot on this, this is basically just a way to avoid writing an argument type?
src/harness/fourslash.ts
Outdated
@@ -3129,6 +3129,7 @@ Actual: ${stringify(fullActual)}`); | |||
${code} | |||
})`; | |||
try { | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Undo
src/harness/runner.ts
Outdated
@@ -207,6 +207,11 @@ function beginTests() { | |||
ts.Debug.enableDebugInfo(); | |||
} | |||
|
|||
// run tests in en-US by default. | |||
const savedUILocale = ts.getUILocale(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you need to set savedUILocale
inside the call to beforeEach
in case the locale was set sometime between the declaration of savedUILocale
and the test running.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Highly unlikely, but I'll make the change.
src/compiler/core.ts
Outdated
/** | ||
* Compare two values for their order relative to each other. | ||
*/ | ||
export function compareValues(a: number, b: number) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename to compareNumbers
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather leave the name as is for the time being. Too many engineers on the team are used to it having this name. I've added an additional comment indicating where to find the string comparison functions, to ease the transition.
src/compiler/checker.ts
Outdated
function binarySearchTypes(types: Type[], type: Type): number { | ||
let low = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we explicitly had two copies of this function to make sure this one is always optimized. we heavily rely on this function in the checker.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interestingly enough, this PR seems to have improved check time. I'll revert this specific change and re-run the benchmarks to see if there is any difference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The difference between the version in the PR and just reverting this single change is negligible (0% to 0.4% better after reverting the change). In most cases that's just within the standard error. I'd argue that manual inlining has only a minor overall impact and it might be worth the change to just have a single binary search algorithm.
This change seems to improve compile-time across the board: compareStrings.txt Notably, check time is improved by 4-6%. |
Unifying deduplication for diagnostics also seems to improve perf: |
src/compiler/core.ts
Outdated
|
||
export function contains<T>(array: ReadonlyArray<T>, value: T, equalityComparer?: EqualityComparer<T>): boolean { | ||
if (array) { | ||
return equalityComparer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was this really faster than just using equateValues
as a default?
src/compiler/core.ts
Outdated
/** | ||
* Deduplicates an unsorted array. | ||
* @param equalityComparer An optional `EqualityComparer` used to determine if two values are duplicates. | ||
* @param comparer An optional `Comparer` used to sort entries before comparison. If supplied, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If a comparer isn't supplied then we should also keep the original order because we push in that order.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Results are always supplied in the original order. I'll update the comment.
src/compiler/core.ts
Outdated
@@ -738,7 +802,7 @@ namespace ts { | |||
* are not present in `arrayA` but are present in `arrayB`. Assumes both arrays are sorted | |||
* based on the provided comparer. | |||
*/ | |||
export function relativeComplement<T>(arrayA: T[] | undefined, arrayB: T[] | undefined, comparer: Comparer<T> = compareValues, offsetA = 0, offsetB = 0): T[] | undefined { | |||
export function relativeComplement<T>(arrayA: T[] | undefined, arrayB: T[] | undefined, comparer: Comparer<T>, offsetA = 0, offsetB = 0): T[] | undefined { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like the offsetA
and offsetB
parameters are never used.
src/compiler/core.ts
Outdated
return a < b ? Comparison.LessThan : a > b ? Comparison.GreaterThan : Comparison.EqualTo; | ||
} | ||
|
||
function compareCaseSensitiveDictionaryOrder(a: string, b: string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not really case sensitive except in the case of a tie. Is there any case where we don't want ties broken? Could we not just always do this?
Actually, this code is probably never hit by tests due to being a fallback -- is this consistent with the behavior of the function it's a fallback for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this preserves the ordering expectation initially expected by the compareNavigateToItems
function in services/navigateTo.ts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a) Since this is a fallback, shouldn't the others have the same behavior?
b) Maybe compareNavigateToItems
should just use a case-insensitive comparison?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a) Both Intl.Collator
and correct implementations of localeCompare
already perform dictionary collation for case-sensitive comparisons.
b) That would result in possible cases of mixed capitalization in results like Aaa
, aaa
, AAa
where we actually want it sorted as AAa
, Aaa
, aaa
.
for (const basePath of patterns.basePaths) { | ||
visitDirectory(basePath, combinePaths(currentDirectory, basePath), depth); | ||
} | ||
|
||
return flatten<string>(results); | ||
|
||
function visitDirectory(path: string, absolutePath: string, depth: number | undefined) { | ||
let { files, directories } = getFileSystemEntries(path); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I liked the deconstruction... note that files
and directories
are each only used once, so you could move the sort
call inside the for loop header to avoid mutating locals.
src/services/textChanges.ts
Outdated
@@ -581,7 +581,7 @@ namespace ts.textChanges { | |||
return applyFormatting(nonformattedText, sourceFile, initialIndentation, delta, this.rulesProvider); | |||
} | |||
|
|||
private static normalize(changes: Change[]): Change[] { | |||
private static normalize(changes: Change[]) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did the return type change?
//// }) | ||
//// /*1*/ | ||
|
||
verify.numberOfErrorsInCurrentFile(1); | ||
verify.numberOfErrorsInCurrentFile(2); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so what is the extra error that showed up here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I fixed an issue in 967a426 where we were losing some deferred global diagnostics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When we would call sortAndDeduplicateDiagnostics
we would sort the diagnostics in place: 967a426#diff-66cec844e251a1918deb897eabca206bL1856
After changing that to sort into a new array, I found that we had a few cases where we would modify the sorted array when adding deferred diagnostics. This was the fix: 967a426#diff-942ae3cd2a8bbd85ed86a60cd7c43307R2324
@mhegazy I ran an updated benchmark with TFS compiling clean and this still reduces compile time even when there are no errors to deduplicate. |
While looking at a different change I noticed that we had some incorrect logic in our string comparison functions due to code motion through multiple changes.
In general, the best mechanism to compare strings (in either a case-sensitive or a case-insensitive manner) is to leverage
Intl.Collator
. This allows us to perform string comparisons using the host culture without needing to perform string transformations for case-insensitive comparisons.When
Intl
is not available, we first try to fall back toString.prototype.localeCompare
before we finally fall back to an ordinal comparison using<
.In 58d69cd we found that
localeCompare
did not provide accurate results in Node v0.10 and introduced a fix to address this case.In d7e33c9 we switched to using
Intl.Collator
for string comparisons in core.tsIn 2bb4abc we moved the
localeCompare
fix to core.ts, but in doing so it seems we ended up with an incorrect check. Rather than falling back toString.prototype.localeCompare
whenIntl
is not available, we only perform the check whenIntl
is available. As such, we will never actually hit the case on Line 1060 as Node v0.10 does not supportIntl
.This change cleans up our string comparison logic and fallback behavior such that:
Intl.Collator
is used, if available, for case-insensitive string comparisons.Intl.Collator
is unavailable,String.prototype.localeCompare
andString.prototype.toLocaleUpperCase
are used (except in the case of Node v0.10.0 where it provides incorrect results). For case-insensitive comparisons we usetoLocaleUpperCase
to properly support comparisons of Unicode characters such as ẞ (German uppercase sharp s) that do not properly round-trip to lowercase and the Turkish (dotted) i and (dotless) ı when the host locale specifies the Turkish alphabet.<
and>
), as well as usingString.prototype.toUpperCase
for case-insensitive comparisons.toUpperCase
still allows for case-insensitive comparisons of ẞ (German uppercase sharp s), but cannot properly handle the Turkish i.This change does not introduce any new tests, as its minimum criteria is to avoid regressions in existing tests.