Skip to content

Conformance testing of ML APIs for the Web #80

Open
@anssiko

Description

@anssiko

@wchao1115 mentions in his talk on DirectML the importance of interoperability across a broad variety of hardware:

The goal of Direct ML is to provide best performance by leveraging the latest hardware features in modern PC while providing an implementation that work reliably in our different hardware platforms, old and new.

We put together a robust conformance test and driver certification process to ensure a high degree of consistency.

So a model works the same way on any windows PC.

The web is known for its focus on interoperability that gives browsers confidence that they are shipping software which is compatible with other implementations, and that later implementations will be compatible with their implementations.

The web interoperability means browsers must behave in predictable ways across different hardware, platforms, and OSes. The web is arguably the platform with the most diverse client installed base, which in turn has motivated approaches such as Progressive Enhancement / Graceful degradation discussed in #68.

Couple of questions:

  • Are there learnings or best practices from DirectML conformance testing efforts or elsewhere in native ecosystem that could be beneficial in the web-platform-tests for related Web APIs?
  • Can some of the existing infrastructure be reused or repurposed for web-platform-tests?
  • What are the special considerations in testing a graph API vs model loader API?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Developer's PerspectiveMachine Learning Experiences on the Web: A Developer's PerspectiveDiscussion topicTopic discussed at the workshopWeb Platform FoundationsWeb Platform Foundations for Machine Learning

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions