Regressions in TIFF support on 32-bit PowerPC (from "Corrected passing TIFF_LONG to libtiff")

[mgorny]

What did you do?

python -m pytest Tests/test_file_libtiff.py on a 32-bit PowerPC system.

What did you expect to happen?

Tests passing.

What actually happened?

======================================================= short test summary info =======================================================
SKIPPED [1] Tests/test_file_libtiff.py:848: LZMA compression support is not configured
SKIPPED [1] Tests/test_file_libtiff.py:863: WEBP compression support is not configured
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_g4_tiff - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_g4_large - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_g4_tiff_file - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_g4_tiff_bytesio - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_g4_non_disk_file_object - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_g4_write - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_write_metadata[False] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_write_metadata[True] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_additional_metadata - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_custom_metadata - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_subifd - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_xmlpacket_tag - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_int_dpi - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_g3_compression - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_little_endian - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_big_endian - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_g4_string_info - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_blur - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_compressions - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_tiff_jpeg_compression - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_tiff_deflate_compression - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_quality - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_cmyk_save - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_palette_save[im0] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_palette_save[im1] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_bw_compression_w_rgb[tiff_ccitt] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_bw_compression_w_rgb[group3] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_bw_compression_w_rgb[group4] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_bytesio - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_ycbcr - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_exif_ifd - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_crashing_metadata - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_page_number_x_0 - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_write_icc - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_tiff_with_jpegtables - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_sampleformat_write - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_block_tile_tags[None] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_block_tile_tags[jpeg] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_sampleformat_not_corrupted - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_multistrip[tiff_adobe_deflate] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_multistrip[jpeg] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_single_strip[True] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_single_strip[False] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_zero[tiff_adobe_deflate] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_zero[None] - RuntimeError: Error setting from dictionary
FAILED Tests/test_file_libtiff.py::TestFileLibTiff::test_save_many_compressed - RuntimeError: Error setting from dictionary
============================================== 46 failed, 44 passed, 2 skipped in 29.40s ==============================================

A sample backtrace:

____________________________________________________ TestFileLibTiff.test_g4_tiff _____________________________________________________

self = <Tests.test_file_libtiff.TestFileLibTiff object at 0xf2fd54c8>
tmp_path = PosixPath('/tmp/pytest-of-mgorny/pytest-39/test_g4_tiff0')

    def test_g4_tiff(self, tmp_path):
        """Test the ordinary file path load path"""
    
        test_file = "Tests/images/hopper_g4_500.tif"
        with Image.open(test_file) as im:
            assert im.size == (500, 500)
>           self._assert_noerr(tmp_path, im)

Tests/test_file_libtiff.py:60: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Tests/test_file_libtiff.py:47: in _assert_noerr
    im.save(out_bytes, format="tiff", compression="group4")
src/PIL/Image.py:2432: in save
    save_handler(self, fp, filename)
src/PIL/TiffImagePlugin.py:1847: in _save
    e = Image._getencoder(im.mode, "libtiff", a, encoderconfig)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

mode = '1', encoder_name = 'libtiff'
args = ('1', 'group4', 0, '', [(256, 500), (257, 500), (258, 1), (259, 4), (262, 1), (266, 1), ...], {273: 4, 279: 4}), extra = ()

    def _getencoder(mode, encoder_name, args, extra=()):
        # tweak arguments
        if args is None:
            args = ()
        elif not isinstance(args, tuple):
            args = (args,)
    
        try:
            encoder = ENCODERS[encoder_name]
        except KeyError:
            pass
        else:
            return encoder(mode, *args + extra)
    
        try:
            # get encoder
            encoder = getattr(core, encoder_name + "_encoder")
        except AttributeError as e:
            msg = f"encoder {encoder_name} not available"
            raise OSError(msg) from e
>       return encoder(mode, *args + extra)
E       RuntimeError: Error setting from dictionary

src/PIL/Image.py:437: RuntimeError
-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
_TIFFVSetField: : Bad value 0 for "FillOrder" tag.

Original bug report, with detailed logs: https://bugs.gentoo.org/942034

What are your OS, Python and Pillow versions?

• OS: Gentoo Linux
• Python: 3.12.7
• Pillow: reproduced with 11.0.0 and 48a4ee9

I've been able to bisect it to b1b0353.

Report from the bisected commit:

$ python3 -m PIL --report
--------------------------------------------------------------------
Pillow 9.5.0.dev0
Python 3.12.7 (main, Oct 26 2024, 14:16:21) [GCC 13.3.1 20240614]
--------------------------------------------------------------------
Python modules loaded from /home/mgorny/Pillow/src/PIL
Binary modules loaded from /home/mgorny/Pillow/src/PIL
--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 9.5.0.dev0
*** TKINTER support not installed
--- FREETYPE2 support ok, loaded 2.13.2
--- LITTLECMS2 support ok, loaded 2.16
--- WEBP support ok, loaded 1.4.0
--- WEBP Transparency support ok
--- WEBPMUX support ok
--- WEBP Animation support ok
--- JPEG support ok, compiled for libjpeg-turbo 3.0.3
--- OPENJPEG (JPEG2000) support ok, loaded 2.5.2
--- ZLIB (PNG/ZIP) support ok, loaded 1.3.1
--- LIBTIFF support ok, loaded 4.6.0
*** RAQM (Bidirectional Text) support not installed
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------
BLP
Extensions: .blp
Features: open, save, encode
--------------------------------------------------------------------
BMP image/bmp
Extensions: .bmp
Features: open, save
--------------------------------------------------------------------
BUFR
Extensions: .bufr
Features: open, save
--------------------------------------------------------------------
CUR
Extensions: .cur
Features: open
--------------------------------------------------------------------
DCX
Extensions: .dcx
Features: open
--------------------------------------------------------------------
DDS
Extensions: .dds
Features: open, save
--------------------------------------------------------------------
DIB image/bmp
Extensions: .dib
Features: open, save
--------------------------------------------------------------------
EPS application/postscript
Extensions: .eps, .ps
Features: open, save
--------------------------------------------------------------------
FITS
Extensions: .fit, .fits
Features: open, save
--------------------------------------------------------------------
FLI
Extensions: .flc, .fli
Features: open
--------------------------------------------------------------------
FPX
Extensions: .fpx
Features: open
--------------------------------------------------------------------
FTEX
Extensions: .ftc, .ftu
Features: open
--------------------------------------------------------------------
GBR
Extensions: .gbr
Features: open
--------------------------------------------------------------------
GIF image/gif
Extensions: .gif
Features: open, save, save_all
--------------------------------------------------------------------
GRIB
Extensions: .grib
Features: open, save
--------------------------------------------------------------------
HDF5
Extensions: .h5, .hdf
Features: open, save
--------------------------------------------------------------------
ICNS image/icns
Extensions: .icns
Features: open, save
--------------------------------------------------------------------
ICO image/x-icon
Extensions: .ico
Features: open, save
--------------------------------------------------------------------
IM
Extensions: .im
Features: open, save
--------------------------------------------------------------------
IMT
Features: open
--------------------------------------------------------------------
IPTC
Extensions: .iim
Features: open
--------------------------------------------------------------------
JPEG image/jpeg
Extensions: .jfif, .jpe, .jpeg, .jpg
Features: open, save
--------------------------------------------------------------------
JPEG2000 image/jp2
Extensions: .j2c, .j2k, .jp2, .jpc, .jpf, .jpx
Features: open, save
--------------------------------------------------------------------
MCIDAS
Features: open
--------------------------------------------------------------------
MIC
Extensions: .mic
Features: open
--------------------------------------------------------------------
MPEG video/mpeg
Extensions: .mpeg, .mpg
Features: open
--------------------------------------------------------------------
MSP
Extensions: .msp
Features: open, save, decode
--------------------------------------------------------------------
PCD
Extensions: .pcd
Features: open
--------------------------------------------------------------------
PCX image/x-pcx
Extensions: .pcx
Features: open, save
--------------------------------------------------------------------
PIXAR
Extensions: .pxr
Features: open
--------------------------------------------------------------------
PNG image/png
Extensions: .apng, .png
Features: open, save, save_all
--------------------------------------------------------------------
PPM image/x-portable-anymap
Extensions: .pbm, .pgm, .pnm, .ppm
Features: open, save
--------------------------------------------------------------------
PSD image/vnd.adobe.photoshop
Extensions: .psd
Features: open
--------------------------------------------------------------------
QOI
Extensions: .qoi
Features: open
--------------------------------------------------------------------
SGI image/sgi
Extensions: .bw, .rgb, .rgba, .sgi
Features: open, save
--------------------------------------------------------------------
SPIDER
Features: open, save
--------------------------------------------------------------------
SUN
Extensions: .ras
Features: open
--------------------------------------------------------------------
TGA image/x-tga
Extensions: .icb, .tga, .vda, .vst
Features: open, save
--------------------------------------------------------------------
TIFF image/tiff
Extensions: .tif, .tiff
Features: open, save, save_all
--------------------------------------------------------------------
WEBP image/webp
Extensions: .webp
Features: open, save, save_all
--------------------------------------------------------------------
WMF
Extensions: .emf, .wmf
Features: open, save
--------------------------------------------------------------------
XBM image/xbm
Extensions: .xbm
Features: open, save
--------------------------------------------------------------------
XPM image/xpm
Extensions: .xpm
Features: open
--------------------------------------------------------------------
XVTHUMB
Features: open
--------------------------------------------------------------------

[mgorny]

As I've noted on #7053, that pull request changed Pillow to start passing int64_t via variable argument list when libtiff expects TIFF_LONG (i.e. a uint32_t), which is clearly wrong.

[mgorny]

In my opinion, the best way to deal with this would be to revert the change to the C code, reopen #7008 and address it properly. The current solution is worse than the problem, as it breaks TIFF support entirely where variable argument passing isn't incidentally "close enough" for 32-bit/64-bit integers. The correct solution would be to pass these EXIF tags as TIFF_IFD8 (perhaps TIFF_LONG8 will work too), rather than modify the behavior for TIFF_LONG.

[radarhere]

I've created #8529 to resolve this.

[mgorny]

Thank you!