Add support for generating class reference descriptions from C# documentation comments

[Mantissa-23]

Describe the project you are working on

Currently working on a movement shooter, but this applies to any project using C#.

Describe the problem or limitation you are having in your project

C# docstrings do not apply to inspector property documentation-on-hover, unlike GDScript documentation comments.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Seeing docstrings in the inspector is valuable for both solo devs, and particularly for teams, where nontechnical team members often need to interact with classes generated by programmers, and the name of a property alone may not fully describe its functionality.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

While I'm unclear on the implementation details, the end-goal is fairly straightforward- Bring C# documentation comments to feature parity with GDScript documentation comments.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Not that I'm aware of.

Is there a reason why this should be core and not an add-on in the asset library?

For the same reason that both C# and GDScript documentation comments are core.

[ttencate]

I went looking for the same just now, and was surprised to find that (a) this functionality doesn't exist (or is broken in 4.2-beta3?), (b) there's no issue about it, and (c) there wasn't even a proposal until 5 hours ago (unless my search-fu failed me). So big thumbs up from me.

[Calinou]

I went looking for the same just now, and was surprised to find that (a) this functionality doesn't exist (or is broken in 4.2-beta3?), (b) there's no issue about it, and (c) there wasn't even a proposal until 5 hours ago (unless my search-fu failed me). So big thumbs up from me.

This is already an open discussion: #5675

This feature was never implemented (not in 3.x either), so it isn't broken.

[ttencate]

@Calinou Given the number of upvotes on that discussion, would it make sense to convert it (back) to an issue and close this one as a duplicate?

[Calinou]

@Calinou Given the number of upvotes on that discussion, would it make sense to convert it (back) to an issue and close this one as a duplicate?

Discussions can't be converted to issues (only the other way around is possible). You can create a new issue from a discussion, but this will just quote the discussion and won't create a redirect or transfer votes/reactions.

For reference, that discussion was originally an issue but it was converted to a discussion because it lacked a concrete technical implementation (see rationale). This particular issue would need to be edited to contain one. A concrete technical implementation doesn't have to be (pseudo)code, it can be an API design (e.g. what would the syntax look like for general use) but we need something. This prevents the discussion from being too vague and not focused enough.

Do we expect the BBCode syntax to be 100% identical to the one used in GDScript, including for references to other properties ([member something]) or functions ([method something])?

[ttencate]

Alright, let me take a stab at detailing the proposal.

Besides inspector tooltips (which would indeed be a major use case), GDScript doc comments also trigger generation of in-editor help pages. There are probably more places where doc comments are, or could be, used, e.g. in autocomplete tooltips. Since the rename of the issue, that's also in scope for this proposal, but it will just come along for free.

Documentation comment syntax

Reference:

• Documentation comments in C#

• Documentation comments in GDScript
  It's worth noting that the documentation for script members doesn't support the distinction between summary, description and tutorial links. It's just all concatenated into a single string, which may contain BBCode.

Of course, we wouldn't want to implement a parser for C# doc comments – Microsoft already did that. We should simply piggy-back on that, and convert the XML to BBCode that the editor understands. As a first attempt, just passing all XML through verbatim would already support doc comments that are just plain text, although many IDEs automatically insert the (required?) <summary></summary> tags so those would look messy.

Fortunately the list of common tags is fairly small. The following translations could be applied:

XML	BBCode	Notes
<c>...</c>	[code]...[/code]
<code>...</code>	[codeblock]...[/codeblock]
<example>...</example>	[b]Example:[/b][br]...
<exception cref="NAME">...</exception>	[b]Throws [NAME]:[/b] ...	Godot doesn't have proper exception handling, but showing explicitly documented exceptions in the docs would be good.
<include/>		Not supported (include verbatim)
<list>		Not supported (include verbatim)
<para>...</para>	...[br]
<param name="NAME">...</param>	[param NAME]: ...[br]
`	[param NAME]
<permission/>		Not supported (include verbatim)
<remarks>...</remarks>		Content appended (after [br]) to description
<returns>...</returns>	[b]Returns:[/b] ...
<see cref="MEMBER" href="URL" langword="KEYWORD"/>	[member CURRENT_CLASS.MEMBER]	href and langword not supported
<seealso cref="MEMBER" href="URL"/>	See [member CURRENT_CLASS.MEMBER][b]	href not supported
<summary>...</summary>		For classes, appended to brief_description, otherwise to description
<typeparam name="NAME">...</typeparam>	[code]NAME[/code]: ...[br]	Not supported in Godot but still belongs in the documentation
<typeparamref name="NAME"/>	[code]NAME[/code]	Not supported in Godot but still belongs in the documentation
<value>...</value>	[b]Value[/b]: ...[br]

Alternatively, a distinction could be made between "inline" and "block" elements, and we could have a template that puts all the block elements of the same type together. For example, it could group all method parameters under a single heading [b]Parameters[/b]:[br]....

Implementation

Internally, documentation data is stored in the DocData class (technically a class but used only as a namespace) in core/doc_data.h. The inspector uses this to create its tooltips, and the in-editor help uses it too. The entry point is DocData::ClassDoc, which contains vectors of the documentation of its members.

Both the inspector and the in-editor help obtain this ClassDoc (or rather, a vector of them, because inner classes) by calling the virtual function Script::get_documentation(), so the main task would be to implement this for CSharpScript, aptly marked TODO.

In GDScript, the documentation is stored in a private member variable docs, which is populated through _add_doc, which is called from GDScriptDocGen::generate_docs. It receives the parse tree from the GDScript parser, which already parsed the doc comments and attached them to the parse tree. This is where the C# implementation would diverge.

By setting GenerateDocumentationFile in the .csproj file, the compiler will emit an XML file containing all the documentation for the assembly. This ends up in .godot/mono/temp/bin/Debug/PROJECTNAME.xml alongside the PROJECTNAME.dll. (By default, the compiler also spits out a warning for every undocumented public field. That should be disabled or made opt-in.)

After successfully (re)loading the assembly, the mono module would attempt to load and parse an XML file next to the DLL with the same name, apply the above transformations, and add the documentation to each class that it knows about. This code would of course be guarded by #ifdef TOOLS_ENABLED because the documentation only exists in editor builds.

[Mantissa-23]

Whoops, didn't realize that existed. I swore I dug pretty hard for this. Feel free to close this issue, or reuse it for the implementation of the proposal.

[ttencate]

Do we expect the BBCode syntax to be 100% identical to the one used in GDScript, including for references to other properties ([member something]) or functions ([method something])?

Using BBCode in the C# doc comments directly didn't even occur to me. It would make implementation simpler, but it would

• feel weird for C# developers,
• not have IDE suppport,
• not play nice if the doc comments are also to be used in the .NET ecosystem.

So I don't think this is the right path.

[geekley]

Using BBCode in the C# doc comments directly didn't even occur to me.

I expect that BBCode inside C# doc comments is undesired and must not be supported at all. Any part with characters that BBCode would interpret (e.g. [, ]) must be properly escaped. This is doubly important inside <c> and <code> tags, but also everywhere else.

It's also best to define how any other tags not listed are handled. Should they just be removed along with contents? Or included verbatim, maybe as code? e.g: [code]<my-tag>[/code]...[code]</my-tag>[/code]

Gotta also consider how "open documentation" (in Godot Script docs view) command (e.g. when right-clicking an exported variable) should likely support C# docs too after this, and it may have clickable links to other types/fields/etc.

Likely there will need to be some conversions (e.g. PascalCase to underscore_case and maybe other equivalences) when referring to engine types/fields/etc. I don't mean in the displayed text necessarily, just in links - so they work properly.
E.g. <see cref="GD.PrintS"/> -> [method @GlobalScope.prints]
([member ...] doesn't work to go directly to the method; so you need to know that PrintS is a method).
This makes me wonder if converting C# docs to GDScript docs syntax is a good approach. Some context-dependent analysis of the referenced code might be unavoidable if this is going to be implemented properly for full support.

Also, it is possible to support href through URLs. URLs do work in GDScript if you use BBCode syntax like [url=https://docs.godotengine.org/en/stable/tutorials/scripting/gdscript/gdscript_basics.html#keywords]self[/url].

[Mantissa-23]

I expect that BBCode inside C# doc comments is undesired and must not be supported at all.

Is there a good reason for this other than "because it would be mixing two different markup syntaxes"? The only reason I can think of this being an issue is if we were to parse non-user generated docstrings, i.e. those in libraries, which I have to imagine we'd want to avoid at all costs for performance and conciseness reasons.

As far as I'm aware, C#'s official documentation doesn't mention any of the patterns in BBCode being used for anything. At first blush they seem compatible?

[geekley]

Mixing 2 different markup systems is enough good reason for not doing it. If you really wanted to do it, the implementation would have to be done right (think cases like <c>[b]...[/b]</c> and [code]<see.../>[/code]; what do they mean?). Doing it right would complicate the implementation and even so it would only cause confusion.
Not to mention the reasons already pointed in the comment above mine.

If you're using xmldoc, you expect xmldoc syntax, consistently. If this proposal wants to support BBCode syntax, then would be better to completely separate it from xmldoc and just implement it with attributes, like:
[Tooltip("[b]BBCode here[/b] etc")]
But this would probably not be what people want. They want xmldoc compatibility.

[geekley]

The following translations could be applied: [...] apply the above transformations

Not sure if I misunderstood the proposed solution, but honestly, converting xmldoc to BBCode to reuse GDScript's implementation seems like a good idea at first, but it might end up biting us in the leg later. If you're gonna take the effort to properly handle all cases and escapes correctly, then it's better to instead use this effort to just do it properly and parse the XML directly into whatever is used to "low-level" render the formatted docs.

Even if it's inevitable to use BBCode under the hood after parsing the XML, C# docs implementation should have no relation to GDScript docs implementation. I mean it shouldn't generate things like [method something], instead both C# and GDScript (and theoretically any other language) should be parsed into whatever links and formatting are needed to give it a similar look and behavior, in a language-agnostic way. Even if a ClassDoc ends up having to use vanilla BBCode under the hood for rendering (I have no idea if that's the case) - you should not need to parse any BBCode at all to create a ClassDoc. That would be just asking for trouble and bugs.

Again, sorry if misunderstood the proposed solution, just thought this might need clarification.

---

It's just all concatenated into a single string, which may contain BBCode.

It think this might be where the problem lies. If ClassDoc's string fields are actually BBCode instead of plain-text (I dunno if that's the case), then it would be better to refactor it to some FormattedString or whatever if possible, so that any BBCode serialization is decoupled from the documentation implementation, and all formatting is resolved before ClassDoc is created, in a language-agnostic way.
Otherwise you need to worry about properly escaping all BBCode from xmldoc.

[ttencate]

Good point. I tend to agree; however I'm not sure this can be implemented in a backwards compatible way. If any of the docs machinery is accessible from outside the engine, like from GDScript or GDExtension, and those APIs expect to send and receive BBCode, it may get difficult to refactor it to make it markup language agnostic. Would need to check.