Generic components with lower case type: "expected a valid html element"

[lukechu10]

Problem

When using a generic component with a type param in lower case, there is an "expected a valid html element" error.

Steps To Reproduce
The following code does not parse

html! {
    <Test<bool>>
        // something
    </Test<bool>>
}

Edit: it seems like there is only an error when the generic type is in lowercase (e.g. bool). PascalCased types don't seem to cause any problem.

Expected behavior
Should parse correctly.

Screenshots

Environment:

• Yew version: master (b694865)
• Rust version: 1.49 (stable)
• Target, if relevant: wasm32-unknown-unknown
• Build tool, if relevant: trunk
• OS, if relevant: windows 10

Questionnaire

• I'm interested in fixing this myself but don't know where to start
• I would like to fix and I have a solution
• I don't have time to fix this right now, but maybe later

I do have some experience with parsing but none with writing proc macros.

[lukechu10]

Update

Doesn't seem to occur with self-closing tags.

[siku2]

You've uncovered a ridiculous issue here...
Funnily enough this issue isn't even caused by the parsing logic, it happens much earlier.
The parser itself would've been fine this construct (probably ;) ) but Yew starts by "peeking" at the code first to determine what kind of HTML construct it has to parse in the first place.
The reason why you get this specific error is because the peeking logic has a bug which resulted in Yew having no clue how to parse this.

Cause of the problem

Part of the peek mechanism for close tags checks whether the type within </ > is a valid component type.
For Yew, a valid component type must start with an uppercase letter and consist of only ASCII characters.
This check is handled by the call to HtmlComponent::peek_type here:

yew/packages/yew-macro/src/html_tree/html_component.rs

Lines 262 to 276 in b694865

	impl PeekValue<Type> for HtmlComponentClose {
	fn peek(cursor: Cursor) -> Option<Type> {
	let (punct, cursor) = cursor.punct()?;
	(punct.as_char() == '<').as_option()?;
	let (punct, cursor) = cursor.punct()?;
	(punct.as_char() == '/').as_option()?;
	let (typ, cursor) = HtmlComponent::peek_type(cursor)?;
	let (punct, _) = cursor.punct()?;
	(punct.as_char() == '>').as_option()?;
	Some(typ)
	}

Keep in mind that this very peek_type function also enforces that the type must be a valid component name!

The function should handle even complex paths like ::std::some::Path<usize>::AssociatedType but there's an issue in how it handles type arguments (like <usize> in the previous example).
To parse the type within the angle brackets, it makes a call to our friend HtmlComponent::peek_type again :)

yew/packages/yew-macro/src/html_tree/html_component.rs

Lines 144 to 151 in b694865

	fn path_arguments(cursor: Cursor) -> Option<(PathArguments, Cursor)> {
	let (punct, cursor) = cursor.punct()?;
	(punct.as_char() == '<').as_option()?;
	let (ty, cursor) = Self::peek_type(cursor)?;
	let (punct, cursor) = cursor.punct()?;
	(punct.as_char() == '>').as_option()?;

And there we have it. The function for parsing type arguments unfortunately enforces that said type arguments must consist of valid component types which means that they must start with an uppercase letter.

Why do the self-closing tags still work?

I found this quite confusing at first but the answer is rather simple. The peek logic for an opening tag (which includes self-closing tags) doesn't check whether the tag ends with /> or >, it only checks for < and the type name.
Now the type name definitely isn't handled correctly in this case either as it still calls the peek_type function. It only parses up to <A , ignores the type arguments (it naively assumes that there are none because it failed), and then just returns.
If it were to check that the next character is a > or / this would fail much like it does with the closing tag.

Solution

Ideally we should get rid of the peeking logic entirely. It's basically a re-implementation of the parsing logic which can obviously lead to small but nonetheless significant differences. There are likely more issues like this waiting to be discovered.
I think there's even another bug hidden in there because it looks like the path_arguments function can only handle a single type argument...

Since that would take quite a bit of effort and time though, here's a much easier way to solve the issue:

1. Remove the following lines - which are responsible for checking whether a type is a valid component name - from the peek_type function:
   

   yew/packages/yew-macro/src/html_tree/html_component.rs

   Lines 200 to 202 in b694865

   	let type_str = last_ident?.to_string();
   	type_str.is_ascii().as_option()?;
   	type_str.bytes().next()?.is_ascii_uppercase().as_option()?;

2. Add a peek_component_type function which calls Self::peek_type() under the hood and then runs the ascii and uppercase checks we removed prior.
3. Have the peek implementations of HtmlComponentOpen and HtmlComponentClose call this function instead of peek_type

Of course a fix for this should also include a test case.

[lukechu10]

@siku2

Ideally we should get rid of the peeking logic entirely. It's basically a re-implementation of the parsing logic which can obviously lead to small but nonetheless significant differences.

I am not familiar (at all) with syn but would it be possible to just reuse syn's default .parse() for syn::TypePath?

Does syn support infinite lookahead or LL(k) grammars? I previously used a parser combinator library, nom before which seems pretty similar (the slice start seems pretty similar to syn's Cursor). Couldn't we just try to parse a html element and if it fails, backtrack and try to parse a component and so on...

Meanwhile, I'll do some more reading on proc macros and syn.