Skip to content

Support migrating old recorded tests using bulk facet recorder #2390

Open
@pokey

Description

@pokey

Once #2383 is merged, we should simplify the process of migrating old recorded tests to scope fixture tests using the new bulk fixture recorder.

We proceed as follows:

  1. Find all recorded tests with the given language id

  2. Canonicalize and inspect the command payload. We may want to filter to tests that use a specific set of actions and particular targets. We may also only want to migrate a specific subset of scopes, eg based on if they're marked as supported for the given language

  3. For each chosen test, get initial document state

  4. Skip if we've already processed a fixture with the same document state (or if there already exists a fixture with same document state?)

  5. Generate an entry in the bulk recorder file with a candidate header for each possible facet that has the given scope. For example:

    [[python]]
    
    [branch.if]
    [branch.if.iteration]
    [branch.try]
    [branch.try.iteration]
    [branch.loop]
    if aaa:
        pass
    ---
    
    [branch.if]
    [branch.if.iteration]
    [branch.try]
    [branch.try.iteration]
    [branch.loop]
    try:
        pass
    except:
        pass
    ---
    
  6. The user will then delete the unnecessary lines and clone the block as necessary eg to test both scope and iteration scope

  7. When the user saves, we should probably delete the recorded tests that they actually ended up migrating

To discuss: which tests to migrate?

We need to figure out how to decide which recorded tests to migrate. Some ideas:

  • All fixtures for a given language, regardless of whether they're marked as supported
  • Only ones marked as supported
  • User picks specific scope type

We also need to figure out if we want to drop recorded tests if their command payload doesn't match a particular shape. Eg skip if multiple scope types are mentioned, or of the command isn't one of a few expected commands

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions