x/sys/windows: use win32metadata?

[bradfitz]

Microsoft released this win32 metadata project: https://github.com/microsoft/win32metadata

It or its output is basically this 9.5MB machine-readable file describing all the Win32 APIs: https://github.com/microsoft/windows-rs/blob/master/crates/winmd/default/Windows.Win32.winmd

Maybe x/sys/windows could use it to auto-generate bindings more easily/accurately.

Might be fun. Might be tedious. Might be worth it.

/cc @alexbrainman @ianlancetaylor @zx2c4

[zx2c4]

That's a great idea. https://docs.microsoft.com/en-us/uwp/winrt-cref/winmd-files is the file format.

I'd be in favor of making x/win32 or x/windows/win32 or similar for this -- something totally separate from the incredibly cluttered and disorganized x/sys/windows. It'll probably take us a few tries to get it right, and I assume that Microsoft will periodically issue corrections to the metadata, which we'll use to regenerate. In that sense, we wouldn't want to strive at an API compatibility guarantee beyond what Microsoft provides. If Microsoft fixes a function signature, then we'd want that to reflect in x/win32 without additional hassle or compatibility burden. This in turn means there might be a little bit more churn than the usual x/ repos, at least in the beginning, but in exchange, we get a complete API picture that's much better than what we have now.

[zx2c4]

CC @jstarks in case you're interested too.

[bcmills]

@zx2c4

we wouldn't want to strive at an API compatibility guarantee beyond what Microsoft provides. If Microsoft fixes a function signature, then we'd want that to reflect in x/win32 without additional hassle or compatibility burden.

At that point would it make sense to rewrite x/sys/windows in terms of small compatibility shims over x/sys/win32?

[jstarks]

Yes, this sounds like a great idea. I agree that starting fresh, rather than trying to replat x/sys/windows (at least initially), is the right approach.

Note that there are fidelity issues with win32metadata today--for example, C unions are not yet supported correctly. But I'm told the team is working on it.

I bet @sotteson1 and @kennykerr would be happy to give advice and take feedback.

[zx2c4]

At that point would it make sense to rewrite x/sys/windows in terms of small compatibility shims over x/sys/win32?

That would potentially be a lot of work but wouldn't be impossible. It'd probably easier, though, to just deprecate x/sys/windows and encourage people to gradually move over to x/windows/win32 (or x/windows/winrt).

[zx2c4]

Note that there are fidelity issues with win32metadata today--for example, C unions are not yet supported correctly. But I'm told the team is working on it.

C unions are the big annoying thing for Go too, and we'd probably want to come up with an overall approach at handling these, whether it's just using unsafe.Pointer when possible, or exploding one struct into multiple, or something else. A lot of fun decisions to make, I guess. C# doesn't quite have unions either, I don't think, which perhaps is where the trouble is coming from on the win32metadata side. I'll be interested to learn what you guys come up with there, as perhaps it's a model we could copy for Go.

[kennykerr]

Note that there are fidelity issues with win32metadata today--for example, C unions are not yet supported correctly. But I'm told the team is working on it.

Note that the metadata fully supports C-style unions. Some of the early language support don't yet support them, but the metadata is ready for consumption/trial.

[jstarks]

Ah, good, I misunderstood the source of the union bug in the Rust crate. Thanks Kenny.

[staticgc]

Does this mean golang's CGO will link with MSVC compiler in future? or I am missing something.

If I am correct, golang on windows works with MingGW & not MSVC

[zx2c4]

Does this mean golang's CGO will link with MSVC compiler in future? or I am missing something.

If I am correct, golang on windows works with MingGW & not MSVC

CGO and MSVC are unrelated concerns to this one, as far as I can see here.

[zx2c4]

Yes, I agree with most of what you said. We'll try not to repeat the mistakes of x/sys/windows, of which there are many many, when generating from winmd. Fully generating it means we can easily tweak the output format until it looks like something acceptable. And relaxing the compatibility requirements -- mentioned in #43838 (comment) -- means that we'd be able to actually perfect the output, rather than getting stuck with inconsistent signatures, like the one you mentioned.

[deadprogram]

I cannot begin to tell you how much I want this feature. 💯