feat: ImageBuf::merge_metadata and oiiotool --mergemeta (AcademySoftwareFoundation#4672)

Clarify that IB::copy_metadata and oiiotool --pastemea fully replace all
existing metadata with that of the other IB.

Add a new IB::merge_metadata() and oiiotool --mergemeta that merges in
metadata from the other IB, preserving metadata in `this` that was not
in the other IB. Optionally, metadata that existed in both can be
overridden. Optionally, a subset of metadata to be merged can be
selected via a regex.

Signed-off-by: Larry Gritz <[email protected]>
Signed-off-by: Scott Wilson <[email protected]>

Author: lgritz

src/doc/imagebuf.rst

@@ -149,8 +149,9 @@ Copying ImageBuf's and blocks of pixels
 .. doxygenfunction:: OIIO::ImageBuf::operator=(ImageBuf &&src)
 .. doxygenfunction:: OIIO::ImageBuf::copy(const ImageBuf &src, TypeDesc format = TypeUnknown)
 .. doxygenfunction:: OIIO::ImageBuf::copy(TypeDesc format) const
-.. doxygenfunction:: OIIO::ImageBuf::copy_metadata
 .. doxygenfunction:: OIIO::ImageBuf::copy_pixels
+.. doxygenfunction:: OIIO::ImageBuf::copy_metadata
+.. doxygenfunction:: OIIO::ImageBuf::merge_metadata
 .. doxygenfunction:: OIIO::ImageBuf::swap
 
 

src/doc/oiiotool.rst

@@ -2998,19 +2998,46 @@ current top image.
 
     Takes two images -- the first will be a source of metadata only, and the
     second the source of pixels -- and produces a new copy of the second
-    image with the metadata from the first image added.
+    image with the metadata from the first image.
 
-    The output image's pixels will come only from the second input. Metadata
-    from the second input will be preserved if no identically-named metadata
-    was present in the first input image.
-
-    Examples::
+    Example::
 
         # Add all the metadata from meta.exr to pixels.exr and write the
         # combined image to out.exr.
         oiiotool meta.exr pixels.exr --pastemeta -o out.exr
 
 
+.. option:: --mergemeta <location>
+
+    Takes two images -- the first will be a source of metadata only, and the
+    second the source of pixels -- and produces a new copy of the second
+    image with the metadata from the first image added added to the second
+    image, item by item (i.e. not wholly removing the metadata that was
+    there before).
+
+    Optional appended modifiers include:
+
+    - `override=` *int* : If zero (the default), no
+      existing metadata in the destination will be altered, i.e., the only
+      metadata copied from source to destination will be those that did not
+      previously exist in the destination. If the override value is nonzero,
+      then all metadata items in the source will *replace* any existing
+      correspondingly named items in the destination.
+
+    - `pattern=` *regex* : If supplied, only copies metadata whose name
+      matches has a substring matching the regular expression. The special
+      character `^` indicates the beginning of the string and `$` indicates
+      the end of the string. 
+
+    Example::
+
+        # Add all of meta.exr's metadata whose name begins with "camera:"
+        # to pixels.exr, replacing any similarly named items, and write
+        # the combined image to out.exr.
+        oiiotool meta.exr pixels.exr --pastemeta:pattern="^camera:override=1" -o out.exr
+
+    The `--mergemeta` command  was added in OIIO 3.0.4.
+
 .. option:: --mosaic <size>
 
     Removes :math:`w \times h` images from the stack, dictated by the *size*,

src/doc/pythonbindings.rst

@@ -1966,19 +1966,6 @@ awaiting a call to `reset()` or `copy()` before it is useful.
     the ImageBuf.
 
 
-.. py:method:: ImageBuf.copy_metadata (other_imagebuf)
-
-    Replaces the metadata (all ImageSpec items, except for the data format
-    and pixel data window size) with the corresponding metadata from the
-    other ImageBuf.
-
-
-.. py:method:: ImageBuf.copy_pixels (other_imagebuf)
-
-    Replace the pixels in this ImageBuf with the values from the other
-    ImageBuf.
-
-
 .. py:method:: ImageBuf ImageBuf.copy (format=TypeUnknown)
 
     Return a full copy of this ImageBuf (with optional data format
@@ -2020,6 +2007,40 @@ awaiting a call to `reset()` or `copy()` before it is useful.
 
 
 
+.. py:method:: ImageBuf.copy_pixels (other_imagebuf)
+
+    Replace the pixels in this ImageBuf with the values from the other
+    ImageBuf.
+
+
+.. py:method:: ImageBuf.copy_metadata (other_imagebuf)
+
+    Replace the metadata of `Self` (all ImageSpec items, except for the data
+    format and pixel data window size) with the metadata from the other
+    ImageBuf.
+
+
+.. py:method:: ImageBuf.merge_metadata (src, override : bool = False, pattern : str = "")
+
+    Merge metadata from `src` into the metadata of `Self` (except for the data
+    format and pixel data window size). Metadata in `Self` that is not in
+    `src` will not be altered. Metadata in `Self` that also is in `src` will
+    be replaced only if `override` is True. If `pattern` is not empty, only
+    metadata having a substring that matches the regex pattern will be merged.
+
+    @version 3.0.5+
+
+    Example:
+
+    .. code-block:: python
+
+        A = ImageBuf("A.exr")
+        B = ImageBuf("B.exr")
+        A.merge_metadata(B, True, "^camera:")
+        # Now A contains all of B's metadata whose name starts with the
+        # substring "camera:"
+
+
 .. py:method:: ImageBuf.swap (other_imagebuf)
 
     Swaps the content of this ImageBuf and the other ImageBuf.

src/include/OpenImageIO/imagebuf.h

@@ -669,10 +669,36 @@ class OIIO_API ImageBuf {
     /// Move assignment.
     const ImageBuf& operator=(ImageBuf&& src);
 
-    /// Copy all the metadata from `src` to `*this` (except for pixel data
-    /// resolution, channel types and names, and data format).
+    /// Copy all the metadata from `src` to `*this`, replacing all named
+    /// metadata that was previously in `*this`. The "full" size and desired
+    /// tile size will also be replaced by the corresponding values from
+    /// `src`, but the pixel data resolution, channel types and names, and
+    /// data format of `*this` will not be altered.
     void copy_metadata(const ImageBuf& src);
 
+    /// Merge metadata from `src` into the metadata of `*this` (except for the
+    /// data format and pixel data window size). Metadata in `*this` that is
+    /// not in `src` will not be altered. Metadata in `*this` that also is in
+    /// `src` will be replaced only if `override` is True. If `pattern` is not
+    /// empty, only metadata having a substring that matches the regex pattern
+    /// will be merged.
+    ///
+    /// @param src
+    ///     The source ImageBuf supplying the metadata (but not pixel
+    ///     values).
+    /// @param override
+    ///     If true, `src` attributes will replace any identically-named
+    ///     attributes already in `*this`. If false (the default), only
+    ///     attributes whose names are not already in this list will be
+    ///     appended.
+    /// @param pattern
+    ///     If not empty, only copy metadata from `src` whose name contains
+    ///     a substring matching the regex `pattern`.
+    ///
+    /// @version 3.0.5+
+    void merge_metadata(const ImageBuf& src, bool override = false,
+                        string_view pattern = {});
+
     /// Copy the pixel data from `src` to `*this`, automatically converting
     /// to the existing data format of `*this`.  It only copies pixels in
     /// the overlap regions (and channels) of the two images; pixel data in

src/libOpenImageIO/imagebuf.cpp

@@ -5,6 +5,7 @@
 
 #include <iostream>
 #include <memory>
+#include <regex>
 
 #include <OpenImageIO/half.h>
 
@@ -169,6 +170,8 @@ class ImageBufImpl {
               void* progress_callback_data       = nullptr,
               DoLock do_lock                     = DoLock(true));
     void copy_metadata(const ImageBufImpl& src);
+    void merge_metadata(const ImageBufImpl& src, bool override = false,
+                        string_view pattern = {});
 
     // At least one of bufspan or buforigin is supplied. Set this->m_bufspan
     // and this->m_localpixels appropriately.
@@ -1774,6 +1777,55 @@ ImageBuf::copy_metadata(const ImageBuf& src)
 
 
 
+void
+ImageBufImpl::merge_metadata(const ImageBufImpl& src, bool override,
+                             string_view pattern)
+{
+    ImageSpec& myspec(this->specmod());
+    const ImageSpec& srcspec(src.spec());
+    std::regex re(std::string(pattern), std::regex_constants::basic);
+    for (const auto& a : srcspec.extra_attribs) {
+        if ((pattern.empty() || std::regex_search(a.name().string(), re))
+            && (override || !myspec.extra_attribs.contains(a.name())))
+            myspec.attribute(a.name(), a.type(), a.data());
+    }
+
+    if (override) {
+        if (pattern.empty() || std::regex_search("full_geom", re)) {
+            m_spec.full_x      = srcspec.full_x;
+            m_spec.full_y      = srcspec.full_y;
+            m_spec.full_z      = srcspec.full_z;
+            m_spec.full_width  = srcspec.full_width;
+            m_spec.full_height = srcspec.full_height;
+            m_spec.full_depth  = srcspec.full_depth;
+        }
+        if (pattern.empty() || std::regex_search("tile_size", re)) {
+            if (src.storage() == ImageBuf::IMAGECACHE) {
+                // If we're copying metadata from a cached image, be sure to
+                // get the file's tile size, not the cache's tile size.
+                m_spec.tile_width  = src.nativespec().tile_width;
+                m_spec.tile_height = src.nativespec().tile_height;
+                m_spec.tile_depth  = src.nativespec().tile_depth;
+            } else {
+                m_spec.tile_width  = srcspec.tile_width;
+                m_spec.tile_height = srcspec.tile_height;
+                m_spec.tile_depth  = srcspec.tile_depth;
+            }
+        }
+    }
+}
+
+
+
+void
+ImageBuf::merge_metadata(const ImageBuf& src, bool override,
+                         string_view pattern)
+{
+    m_impl->merge_metadata(*src.m_impl, override, pattern);
+}
+
+
+
 const ImageSpec&
 ImageBuf::spec() const
 {

src/oiiotool/oiiotool.cpp

@@ -4765,6 +4765,18 @@ OIIOTOOL_OP(pastemeta, 2, [&](OiiotoolOp& op, span<ImageBuf*> img) {
 
 
 
+// --mergemeta
+OIIOTOOL_OP(mergemeta, 2, [&](OiiotoolOp& op, span<ImageBuf*> img) {
+    std::string pattern = op.options("pattern");
+    bool override       = op.options("override").get<int>();
+
+    *img[0] = *img[2];
+    img[0]->merge_metadata(*img[1], override, pattern);
+    return true;
+});
+
+
+
 // --mosaic
 static void
 action_mosaic(Oiiotool& ot, cspan<const char*> argv)
@@ -6787,8 +6799,11 @@ Oiiotool::getargs(int argc, char* argv[])
       .help("Paste fg over bg at the given position (e.g., +100+50; '-' or 'auto' indicates using the data window position as-is; options: all=%d, mergeroi=%d)")
       .OTACTION(action_paste);
     ap.arg("--pastemeta")
-      .help("Copy the metadata from the first image to the second image and write the combined result.")
+      .help("Copy the metadata from the first image to the second image and keep the combined result")
       .OTACTION(action_pastemeta);
+    ap.arg("--mergemeta")
+      .help("Merge the metadata from the first image into the second image and keep the combined result (options: pattern=REGEX, override=%d)")
+      .OTACTION(action_mergemeta);
     ap.arg("--mosaic %s:WxH")
       .help("Assemble images into a mosaic (arg: WxH; options: pad=0, fit=WxH)")
       .OTACTION(action_mosaic);

src/python/py_imagebuf.cpp

@@ -426,8 +426,6 @@ declare_imagebuf(py::module& m)
 
         .def("pixelindex", &ImageBuf::pixelindex, "x"_a, "y"_a, "z"_a,
              "check_range"_a = false)
-        .def("copy_metadata", &ImageBuf::copy_metadata)
-        .def("copy_pixels", &ImageBuf::copy_pixels)
         .def(
             "copy",
             [](ImageBuf& self, const ImageBuf& src, TypeDesc format) {
@@ -442,6 +440,15 @@ declare_imagebuf(py::module& m)
                 return src.copy(format);
             },
             "format"_a = TypeUnknown)
+        .def("copy_pixels", &ImageBuf::copy_pixels)
+        .def("copy_metadata", &ImageBuf::copy_metadata)
+        .def(
+            "merge_metadata",
+            [](ImageBuf& self, const ImageBuf& src, bool override,
+               const std::string& pattern) {
+                self.merge_metadata(src, override, pattern);
+            },
+            "src"_a, "override"_a = false, "pattern"_a = "")
         .def("swap", &ImageBuf::swap)
         .def("getchannel", &ImageBuf::getchannel, "x"_a, "y"_a, "z"_a, "c"_a,
              "wrap"_a = "black")

testsuite/oiiotool-copy/ref/out.txt

@@ -76,6 +76,7 @@ green.exr            :   64 x   64, 3 channel, half openexr
     SHA-1: 8B61993247469F3C208CA894D71856727B11606A
     channel list: R, G, B
     compression: "zip"
+    hair: "black"
     PixelAspectRatio: 1
     screenWindowCenter: 0, 0
     screenWindowWidth: 1
@@ -93,6 +94,54 @@ greenmeta.exr        :   64 x   64, 3 channel, half openexr
     screenWindowCenter: 0, 0
     screenWindowWidth: 1
     weight: 20.5
+    camera:lens: "AX30"
+    camera:shutter: 0.0125
+    oiio:ColorSpace: "lin_rec709"
+    oiio:subimages: 1
+    openexr:lineOrder: "increasingY"
+Reading greenmeta-merge.exr
+greenmeta-merge.exr  :   64 x   64, 3 channel, half openexr
+    SHA-1: 8B61993247469F3C208CA894D71856727B11606A
+    channel list: R, G, B
+    compression: "zip"
+    eyes: 2
+    hair: "black"
+    PixelAspectRatio: 1
+    screenWindowCenter: 0, 0
+    screenWindowWidth: 1
+    weight: 20.5
+    camera:lens: "AX30"
+    camera:shutter: 0.0125
+    oiio:ColorSpace: "lin_rec709"
+    oiio:subimages: 1
+    openexr:lineOrder: "increasingY"
+Reading greenmeta-merge-override.exr
+greenmeta-merge-override.exr :   64 x   64, 3 channel, half openexr
+    SHA-1: 8B61993247469F3C208CA894D71856727B11606A
+    channel list: R, G, B
+    compression: "zip"
+    eyes: 2
+    hair: "brown"
+    PixelAspectRatio: 1
+    screenWindowCenter: 0, 0
+    screenWindowWidth: 1
+    weight: 20.5
+    camera:lens: "AX30"
+    camera:shutter: 0.0125
+    oiio:ColorSpace: "lin_rec709"
+    oiio:subimages: 1
+    openexr:lineOrder: "increasingY"
+Reading greenmeta-merge-camera.exr
+greenmeta-merge-camera.exr :   64 x   64, 3 channel, half openexr
+    SHA-1: 8B61993247469F3C208CA894D71856727B11606A
+    channel list: R, G, B
+    compression: "zip"
+    hair: "black"
+    PixelAspectRatio: 1
+    screenWindowCenter: 0, 0
+    screenWindowWidth: 1
+    camera:lens: "AX30"
+    camera:shutter: 0.0125
     oiio:ColorSpace: "lin_rec709"
     oiio:subimages: 1
     openexr:lineOrder: "increasingY"

testsuite/oiiotool-copy/run.py

@@ -93,11 +93,17 @@
             + "--pattern checker 256x256 3 --paste +150+75 -o pasted.tif")
 
 # test --pastemeta
-command += oiiotool ("--pattern:type=half constant:color=0,1,0 64x64 3 -o green.exr")
-command += oiiotool ("--pattern:type=half constant:color=1,0,0 64x64 3 -attrib hair brown -attrib eyes 2 -attrib weight 20.5 -o redmeta.exr")
-command += oiiotool ("redmeta.exr green.exr --pastemeta -o greenmeta.exr")
+command += oiiotool ("--pattern:type=half constant:color=0,1,0 64x64 3 -attrib hair black -o green.exr")
+command += oiiotool ("--pattern:type=half constant:color=1,0,0 64x64 3 -attrib hair brown -attrib eyes 2 -attrib weight 20.5 -attrib camera:lens AX30 --attrib camera:shutter 0.0125  -o meta.exr")
+command += oiiotool ("meta.exr green.exr --pastemeta -o greenmeta.exr")
+command += oiiotool ("meta.exr green.exr --mergemeta -o greenmeta-merge.exr")
+command += oiiotool ("meta.exr green.exr --mergemeta:override=1 -o greenmeta-merge-override.exr")
+command += oiiotool ("meta.exr green.exr --mergemeta:pattern=\"^camera:\" -o greenmeta-merge-camera.exr")
 command += info_command ("green.exr", safematch=True)
 command += info_command ("greenmeta.exr", safematch=True)
+command += info_command ("greenmeta-merge.exr", safematch=True)
+command += info_command ("greenmeta-merge-override.exr", safematch=True)
+command += info_command ("greenmeta-merge-camera.exr", safematch=True)
 
 # test mosaic
 # Purposely test with fewer images than the mosaic array size