Update Fiddle docs and GitHub CI configs

shoyer · copybara-github · commit 4956f24776a8 · 2025-11-03T14:36:30.000-08:00
The docs now build successfully again on ReadTheDocs: https://fiddle--595.org.readthedocs.build/en/595/index.html I've switched GitHub CI builds to use Python 3.11 or 3.12, which are much more modern than the ancient versions of Python that were previously being tested. Python 3.13 would be great, but some of the test dependencies aren't available. PiperOrigin-RevId: 827654128
diff --git a/.github/workflows/testing.yaml b/.github/workflows/testing.yaml
@@ -10,8 +10,10 @@ jobs:
   build:
     runs-on: ubuntu-latest
     strategy:
+      fail-fast: false
       matrix:
-        python-version: ["3.8", "3.9", "3.10"]
+        # tensorflow-text (and hence seqio) is not available for python 3.13
+        python-version: ["3.11", "3.12"]
 
     steps:
       - uses: actions/checkout@v3
diff --git a/.readthedocs.yml b/.readthedocs.yml
@@ -7,9 +7,9 @@ version: 2
 
 # Set the version of Python and other tools you might need
 build:
-  os: ubuntu-20.04
+  os: ubuntu-24.04
   tools:
-    python: "3.9"
+    python: "3.12"
 
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
diff --git a/docs/api_reference/core.rst b/docs/api_reference/core.rst
@@ -50,13 +50,3 @@ Advanced Functionality
 ~~~~~~~~~~~~~
 .. autoclass:: Buildable
     :members: __init__, __build__, __flatten__, __unflatten__, __path_elements__, __setattr__, __delattr__, __dir__, __repr__, __copy__, __deepcopy__, __eq__, __getstate__, __setstate__
-
-
-ArgFactory Internals
-~~~~~~~~~~~~~~~~~~~~
-.. currentModule:: fiddle._src.config
-.. autoclass:: _BuiltArgFactory
-
-Casting
-~~~~~~~
-.. autofunction:: register_supported_cast
diff --git a/docs/colabs.md b/docs/colabs.md
@@ -11,7 +11,6 @@ Please send us feedback if you have any suggestions for improvements! Or if you
 built something cool and want to share it out, we'd love to link it up at the
 "Where to go from here" section.
 
-[TOC]
 
 ## General colab advice
 
diff --git a/docs/conf.py b/docs/conf.py
@@ -99,3 +99,7 @@
 
 # Enable auto-generation of anchors for headings in markdown files.
 myst_heading_anchors = 3
+
+# Allow non-consecutive header level increase:
+# https://stackoverflow.com/questions/73074870/myst-allowing-non-consecutive-header-level-increase
+suppress_warnings = ['myst.header']
diff --git a/docs/flags_code_lab.md b/docs/flags_code_lab.md
@@ -7,48 +7,33 @@ to Fiddle's absl_flags integration! This code lab will describe how it all
 works. Configuration of fiddle config objects via command line arguments is
 supported using 3 APIs:
 
-| API        | Purpose                                                        |
-| ---------- | -------------------------------------------------------------- |
-| New API,   | Defines custom configurations per binary using the API         |
-: single     : `DEFINE_fiddle_config()` which returns a config object after   :
-: config     : applying all the command line overrides in order.              :
-:            : The usage of this API is more intuitive than the legacy API,   :
-:            : and it provides the ability to define custom overrides per     :
-:            : binary as well as read serialized configs from a file or       :
-:            : strings on the command line. Additionally, the overrides are   :
-:            : applied in order, which is a more intuitive user experience    :
-:            : than the current order followed by the legacy API.             :
-| New API,   | Defines a 'sweep' of multiple custom configurations per binary |
-: multiple   : using the API `DEFINE_fiddle_sweep()`. This is similar to      :
-: configs    : `DEFINE_fiddle_config` but returns a sequence of multiple      :
-:            : configs each with some metadata. It allows specifying them     :
-:            : via a sequence of overrides to config attributes, and/or       :
-:            : overrides to arguments of the function that generates configs. :
-:            : This is intended mainly for use in launch binaries of ML       :
-:            : experiments which perform hyperparameter sweeps.               :
-| Legacy API | Invoked via `create_buildable_from_flags()` that returns a     |
-:            : config object. Command line overrides are NOT applied in       :
-:            : order; all fiddlers are applied first, followed by all tags,   :
-:            : followed by all overrides.                                     :
+*   **New API, single config**: Defines custom configurations per binary using
+    the API `DEFINE_fiddle_config()` which returns a config object after
+    applying all the command line overrides in order. The usage of this API is
+    more intuitive than the legacy API, and it provides the ability to define
+    custom overrides per binary as well as read serialized configs from a file
+    or strings on the command line. Additionally, the overrides are applied in
+    order, which is a more intuitive user experience than the current order
+    followed by the legacy API.
+*   **New API, multiple configs**: Defines a 'sweep' of multiple custom
+    configurations per binary using the API `DEFINE_fiddle_sweep()`. This is
+    similar to `DEFINE_fiddle_config` but returns a sequence of multiple configs
+    each with some metadata. It allows specifying them via a sequence of
+    overrides to config attributes, and/or overrides to arguments of the
+    function that generates configs. This is intended mainly for use in launch
+    binaries of ML experiments which perform hyperparameter sweeps.
+*   **Legacy API**: Invoked via `create_buildable_from_flags()` that returns a
+    config object. Command line overrides are NOT applied in order; all fiddlers
+    are applied first, followed by all tags, followed by all overrides.
 
 > NOTE: New usages of the legacy flags API are discouraged and users should
 > migrate their legacy usage to the new API.
 
-See some working examples of the APIs below.
+See some working examples of the APIs below:
 
-<section class="tabs" markdown=1>
-
-### New API {.new-tab}
-
-[example](http://github.com/google/fiddle/tree/main/fiddle/_src/absl_flags/sample_test_binary.py)
-
-### Legacy API {.new-tab}
-
-[example](http://github.com/google/fiddle/tree/main/fiddle/absl_flags/example)
+-   [New API](http://github.com/google/fiddle/tree/main/fiddle/_src/absl_flags/sample_test_binary.py)
+-   [Legacy API](http://github.com/google/fiddle/tree/main/fiddle/absl_flags/example)
 
-</section>
-
-[TOC]
 
 ## How to structure your code
 
@@ -88,7 +73,7 @@ When our example program is executed, the following steps occur:
 
 <section class="tabs" markdown=1>
 
-### New API (single config) {.new-tab}
+### New API (single config)
 
 1.  **Launch**: We run our program on the command line:
 
@@ -109,7 +94,7 @@ When our example program is executed, the following steps occur:
     arbitrary functions on the built objects to carry out whatever task your
     application has been designed to do.
 
-### New API (multiple configs) {.new-tab}
+### New API (multiple configs)
 
 1.  **Launch**: We run our launcher program on the command line to launch
     multiple configs:
@@ -124,13 +109,11 @@ When our example program is executed, the following steps occur:
 
 2.  **Flag Parsing**: The custom Fiddle flag parser parses Fiddle-related flags
     from the command line, and applies any overrides specified in the sweeps to
-    create a sweep of one or more configs.
-    If multiple sweeps are specified, the cartesian product of the sweeps is
-    taken before applying them.
-    Any fiddler: or set: commands are then applied to all configs in the sweep,
-    in the order specified.
-    `_SAMPLE_FLAG.value` returns a sequence of SweepItem dataclasses, each of
-    which has a `.config` property of type `fdl.Buildable`, and an
+    create a sweep of one or more configs. If multiple sweeps are specified, the
+    cartesian product of the sweeps is taken before applying them. Any fiddler:
+    or set: commands are then applied to all configs in the sweep, in the order
+    specified. `_SAMPLE_FLAG.value` returns a sequence of SweepItem dataclasses,
+    each of which has a `.config` property of type `fdl.Buildable`, and an
     `overrides_applied` property which is the dict of overrides and can be
     useful to log as metadata attached to each experiment launched.
 
@@ -142,7 +125,7 @@ When our example program is executed, the following steps occur:
 4.  **Business logic in the main binary**: Is the same as in the single-config
     case.
 
-### Legacy API {.new-tab}
+### Legacy API
 
 1.  **Launch**: We run our program on the command line:
 
@@ -196,114 +179,88 @@ When our example program is executed, the following steps occur:
 
 ## Flag Syntax
 
-The Fiddle flag integration supports the following flag syntax:
-
--   **Base Config**: The base configuration function is specified with the flag:
+The Fiddle flag integration supports the following flag syntax.
 
 <section class="tabs" markdown=1>
 
-### New API {.new-tab}
-
-that was set as the `name` argument for `DEFINE_fiddle_config` or
-`DEFINE_fiddle_sweep`, and the command `config`. For example, if the flag object
-was instantiated as
-`DEFINE_fiddle_config(name="my_flag", ...)`, then the base config is specified
-by using `--my_flag
-config:some_function_returning_fiddle_config_to_be_overridden()`.
-With `DEFINE_fiddle_config` one can also use the command `config_file` to read
-from a JSON serialized config written to a file, or the command `config_str` to
-read from a JSON serialized config in encoded string form (the additional
-encoding involves zlib compression followed by base64 encoding).
-
-### Legacy API {.new-tab}
+### New API
+
+-   **Base Config**: The base configuration function is specified with the
+    `config` command, following the flag name provided to `DEFINE_fiddle_config`
+    or `DEFINE_fiddle_sweep`. For example, if the flag object was instantiated
+    as `DEFINE_fiddle_config(name="my_flag", ...)`, then the base config is
+    specified by using `--my_flag
+    config:some_function_returning_fiddle_config_to_be_overridden()`. With
+    `DEFINE_fiddle_config` one can also use the command `config_file` to read
+    from a JSON serialized config written to a file, or the command `config_str`
+    to read from a JSON serialized config in encoded string form (the additional
+    encoding involves zlib compression followed by base64 encoding).
+
+-   **Fiddlers**: Fiddlers are specified on the command line with the `fiddler`
+    command after the `name` argument for `DEFINE_fiddle_config` or
+    `DEFINE_fiddle_sweep`. For example, if the flag object was instantiated as
+    `DEFINE_fiddle_config(name="my_flag", ...)` then the fiddlers would be
+    invoked like `--my_flag fiddler:name_of_fiddler(value="new_value")`.
 
-`--fdl_config`. Example: `--fdl_config=base`. Alternatively, a JSON-serialized
-configuration can be read from a file via with the flag `--fdl_config_file`.
-Example: `--fdl_config_file='/some/path/config.json'`.
-
-</section>
+-   **Specific Overrides**: Specific overrides allow you to specify specific
+    values to arbitrary fields on the command line. The syntax is the `set`
+    command after the `name` argument for `DEFINE_fiddle_config` or
+    `DEFINE_fiddle_sweep`. For example, if the flag object was instantiated as
+    `DEFINE_fiddle_config(name="my_flag", ...)`, then the specific overrides are
+    specified using `--my_flag set:some_attr.some_sub_attr=some_value`.
+
+-   **Sweeps**: Sweeps allow you to specify multiple dicts of overrides to
+    apply, to generate a sweep of one or more configs. The syntax is the `sweep`
+    command after the `name` argument for `DEFINE_fiddle_sweep`. For example, if
+    the flag object was instantiated as `DEFINE_fiddle_sweep(name="my_flag",
+    ...)`, then one or more sweep functions can be specified using `--my_flag
+    sweep:name_of_sweep(arguments=123)` or just `--my_flag sweep:name_of_sweep`
+    if no arguments are required to the sweep function.
+
+    A `sweep:` command should specify a function call returning a list of
+    dictionaries, where each dictionary represents a single item in the sweep.
+    The entries in the dictionary are the overrides to apply, where keys can be
+    of the form:
+
+    *   `kwarg:foo` -- to specify or override a keyword argument to the base
+        config function specified by the `config:` command.
+    *   `arg:0` -- to specify or override a positional argument to the base
+        config function specified by the `config:` command.
+    *   `path.to.some.attr` -- to specify an override to an attribute in the
+        config returned by the base config function. These paths follow the same
+        format as is accepted by `set:` commands and can take quite general
+        forms like `foo.bar['baz'][0].boz`.
+
+    Multiple `sweep:` commands can be specified, which will result in taking the
+    cartesian product of the separate sweeps before applying them.
+
+### Legacy API
+
+-   **Base Config**: The base configuration function is specified with the
+    `--fdl_config` flag. Example: `--fdl_config=base`. Alternatively, a
+    JSON-serialized configuration can be read from a file with the flag
+    `--fdl_config_file`. Example: `--fdl_config_file='/some/path/config.json'`.
 
 -   **Fiddlers**: Fiddlers are specified on the command line with the
-
-<section class="tabs" markdown=1>
-
-### New API {.new-tab}
-
-command `fiddler` after the `name` argument for `DEFINE_fiddle_config` or
-`DEFINE_fiddle_sweep`. For example, if the flag object was instantiated as
-`DEFINE_fiddle_config(name="my_flag", ...)` then the fiddlers would be invoked
-like `--my_flag fiddler:name_of_fiddler(value="new_value")`.
-
-### Legacy API {.new-tab}
-
-`--fiddler=` flag. This flag can be specified multiple times. Example:
-`--fiddler=swap_weight_and_biases --fiddler=other_fiddler`.
-
-</section>
+    `--fiddler=` flag. This flag can be specified multiple times. Example:
+    `--fiddler=swap_weight_and_biases --fiddler=other_fiddler`.
 
 -   **Specific Overrides**: Specific overrides allow you to specify specific
     values to arbitrary fields on the command line. The syntax is
-
-<section class="tabs" markdown=1>
-
-### New API {.new-tab}
-
-the command `set` after the `name` argument for `DEFINE_fiddle_config` or
-`DEFINE_fiddle_sweep`. For example, if the flag object was instantiated as
-`DEFINE_fiddle_config(name="my_flag", ...)`, then the specific overrides are
-specified using `--my_flag set:some_attr.some_sub_attr=some_value`.
-
-### Legacy API {.new-tab}
-
-`--fdl.dotted.path.of.fields=3`, where everything after the `fdl.` prefix
-corresponds to the name of a field in the Fiddle configuration object
-corresponding to exactly the same Python code. For example, if (in Python) we
-wanted to set the value of a nested field to 15, I might write:
-`cfg.model.dense1.parameters = 15`. The corresponding syntax on the command line
-would be: `--fdl.model.dense1.parameters=15`. Due to shell escaping, to specify
-a string value, you often need to use two nested quotes, or escape the outer
-quotes (depending on your shell). For example:
-`--fdl.data.filename='"other.txt"'` (or equivalently:
-`--fdl.data.filename=\"other.txt\"`). Only "literal" values may be specified on
-the command line like this; if you'd like to set a complex value, please write a
-fiddler and invoke it with the previous Fiddlers syntax.
-
-</section>
-
--   **Sweeps**: Allow you to specify multiple dicts of overrides to apply, to
-    generate a sweep of one or more configs. The syntax is
-
-<section class="tabs" markdown=1>
-
-### New API {.new-tab}
-
-the command `sweep` after the `name` argument for `DEFINE_fiddle_sweep`. For
-example, if the flag object was instantiated as
-`DEFINE_fiddle_sweep(name="my_flag", ...)`, then one or more sweep functions can
-be specified using `--my_flag sweep:name_of_sweep(arguments=123)` or just
-`--my_flag sweep:name_of_sweep` if no arguments are required to the sweep
-function.
-
-A sweep: command should specify a function call returning a list of
-dictionaries, where each dictionary represents a single item in the sweep.
-The entries in the dictionary are the overrides to apply, where keys can be of
-the form:
-
- * `kwarg:foo` -- to specify or override a keyword argument to the base config
-   function specified by the `config:` command.
- * `arg:0` -- to specify or override a positional argument to the base config
-   function specified by the `config:` command.
- * `path.to.some.attr` -- to specify an override to an attribute in the
-   config returned by the base config function. These paths follow the same
-   format as is accepted by `set:` commands and can take quite general forms
-   like `foo.bar['baz'][0].boz`.
-
-Multiple `sweep:` commands can be specified, which will result in taking the
-cartesian product of the separate sweeps before applying them.
-
-### Old API {.new-tab}
-
-This is not supported in the old API.
+    `--fdl.dotted.path.of.fields=3`, where everything after the `fdl.` prefix
+    corresponds to the name of a field in the Fiddle configuration object
+    corresponding to exactly the same Python code. For example, if (in Python)
+    we wanted to set the value of a nested field to 15, I might write:
+    `cfg.model.dense1.parameters = 15`. The corresponding syntax on the command
+    line would be: `--fdl.model.dense1.parameters=15`. Due to shell escaping, to
+    specify a string value, you often need to use two nested quotes, or escape
+    the outer quotes (depending on your shell). For example:
+    `--fdl.data.filename='"other.txt"'` (or equivalently:
+    `--fdl.data.filename=\"other.txt\"`). Only "literal" values may be specified
+    on the command line like this; if you'd like to set a complex value, please
+    write a fiddler and invoke it with the previous Fiddlers syntax.
+
+-   **Sweeps**: This is not supported in the legacy API.
 
 </section>
 
diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -1,10 +1,10 @@
-sphinx >4, <5
+sphinx
 sphinx-autodoc-typehints
-sphinx-book-theme>=0.3.3
-sphinx-copybutton>=0.5.0
+sphinx-book-theme
+sphinx-copybutton
 sphinx-design
 sphinx-remove-toctrees
-jupyter-sphinx>=0.3.2
+jupyter-sphinx
 myst-nb
 
 # Install the Fiddle package itself.
diff --git a/setup.py b/setup.py
@@ -94,9 +94,6 @@
         # Specify the Python versions you support here. In particular, ensure
         # that you indicate whether you support Python 2, Python 3 or both.
         'Programming Language :: Python :: 3',
-        'Programming Language :: Python :: 3.8',
-        'Programming Language :: Python :: 3.9',
-        'Programming Language :: Python :: 3.10',
 
         'Topic :: Scientific/Engineering',
         'Topic :: Scientific/Engineering :: Mathematics',
