In some places, a placeholder is not identified correctly

[HassanTaleb90]

Description

In this example:

func <#name#>(<#parameters#>) -> <#return type#> {
	<#function body#>
}

<#name#> // FunctionDeclSyntax
<#parameters#> // FunctionParameterSyntax
<#return type#> //EditorPlaceholderTypeSyntax (no problem)
<#function body#> //EditorPlaceholderExprSyntax (no problem)

Other examples tested:

class <#name#>: <#super class#> {  // problem
	<#code#>
}

enum <#name#> {  // problem
case <#case#>
}

struct <#name#> {  // problem
	<#fields#>
}

init(<#parameters#>) {  // problem
    <#statements#>
}
convenience init(<#parameters#>) {  // problem
    <#statements#>
}

required init(<#parameters#>) {  // problem
    <#statements#>
}

import <#module#>  // problem

if #available(<#{i, mac, tv, watch}#>OS <#x.y.z#>, *) {  // 2 problems
<#API available statements#>
} else {
<#fallback statements#>
}

{ (<#parameters#>) -> <#return type#> in  // problem
    <#statements#>
}

protocol <#name#> {  // problem
	<#requirements#>
}

struct <#name#>: OptionSet {  // problem
	let rawValue: <#integer type#>

	static let <#optionA#> = <#name#>(rawValue: 1 << 0)
	static let <#optionB#> = <#name#>(rawValue: 1 << 1)
	static let <#optionC#> = <#name#>(rawValue: 1 << 2)

	static let all: <#name#> = [.<#optionA#>, .<#optionB#>, .<#optionC#>] // problem
}

typealias <#type name#> = <#type expression#>  // problem

func test<#Name#>() {  // problem
    <#statements#>
}

[ahoppen]

Tracked in Apple’s issue tracker as rdar://115296709

[ahoppen]

OK, I have thought about this some more. We need to revisit how editor placeholders are represented in the tree as a whole. What we should do is the following:

• Editor placeholders continue to be lexed as identifiers with the error TokenDiagnostic.Kind.editorPlaceholder set.
• We should delete all the EditorPlaceholder* nodes, undoing Handle pattern as editor placeholder #2150 + removing EditorPlaceholderExpr and EditorPlaceholderDecl (I think that’s all the editor placeholder nodes that we have)
• TokenSyntax should have a isEditorPlaceholder property that returns true if the token is an identifier and starts with <#.

The results will be that all cases are classified in the way that the issue describes as the “problem” case. Checking for editor placeholders will be a matter of looking for tokens that return true for isEditorPlacholder.

The reason why I think this is the right design is that eg. for func <#foo#>(), we can’t create an EditorPlaceholder* node for <#foo#> since <#foo#> is parsed as the function’s name and that’s of type TokenSyntax. We also considered introducing a new token kind for editor placeholder but that would mean that the parser had to check for editor placeholders everywhere it expects an identifier and editor placeholder would need to start being a token option for every identifier token in the tree, which doesn’t scale. Lastly, we considered parsing the editor placeholder as an unexpected token in these cases, followed by recovery picking up a missing identifier, but that would make it really hard for ASTGen to e.g. extract the type annotation of the editor placeholder when parsing an expression and really, the editor placeholder stands in place of the name.

[HassanTaleb90]

It is also useful to identify the placeholders in trivia.

/// <#Description#>
/// - Parameter str: <#str description#>
/// - Returns: <#description#>

[HassanTaleb90]

It is also useful to identify the placeholders in trivia.

/// <#Description#>
/// - Parameter str: <#str description#>
/// - Returns: <#description#>

@kimdv

[ahoppen]

Placeholders in comments aren’t something we can or want to represent in the syntax tree IMO. The only way we could achieve that is by introducing another trivia kind for placeholders but that would

1. Raise ambiguities about when placeholders are trivia and when they are tokens
2. It’s not even clear to me how you would classify the text after the placeholder if you have e.g. // my <#placeholder#> text. It’s not a line comment since it doesn’t start with //, so what would it be?
3. Chop the trivia into even more pieces, which would make it harder for a library to parse e.g. doc comments as Markdown and render them nicely, since, on every placeholder, it would now need to decide whether the following trivia piece is still part of the comment on the same line or if it’s part of a new logical trivia section.

[HassanTaleb90]

What about using regex to identify placeholders?

Placeholders in comments aren’t something we can or want to represent in the syntax tree IMO. The only way we could achieve that is by introducing another trivia kind for placeholders but that would

1. Raise ambiguities about when placeholders are trivia and when they are tokens
2. It’s not even clear to me how you would classify the text after the placeholder if you have e.g. // my <#placeholder#> text. It’s not a line comment since it doesn’t start with //, so what would it be?
3. Chop the trivia into even more pieces, which would make it harder for a library to parse e.g. doc comments as Markdown and render them nicely, since, on every placeholder, it would now need to decide whether the following trivia piece is still part of the comment on the same line or if it’s part of a new logical trivia section.

[ahoppen]

How exactly do you think that API would look like?