JPEG2000 irreversible option misleading

[scaramallion]

The documentation for the J2K plugin says:

irreversible
If True, use the lossy Irreversible Color Transformation followed by DWT 9-7. Defaults to False, which means to use the Reversible Color Transformation with DWT 5-3.

I think the "color transformation" here is referring to the multiple component transformation (MCT), however looking at the source code, MCT isn't being applied, so no color transform is occurring (as I understand it irreversible in openjpeg only controls which DWT is used). This is visible in the output codestream:

from io import BytesIO
import numpy as np
from PIL import Image
from struct import unpack

# https://github.com/python-pillow/Pillow/blob/master/Tests/images/xmp_test.jpg
im = Image.open("xmp_test.jpg")
assert im.mode == "RGB"
ref = np.asarray(im)
b = BytesIO()
b.name = "foo.j2k"  # workaround so no JP2
im.save(b, format="JPEG2000", irreversible=False)

# Parse J2K codestream
codestream = b.getvalue()
print(f"len: {len(codestream)}")
if codestream[0:2] == b'\xff\x4f':
    if codestream[2:4] == b'\xff\x51':
        lsiz = unpack('>H', codestream[4:6])[0]
        ssiz = codestream[42]
        if ssiz & 0x80:
            print(f"precision: {(ssiz & 0x7F) + 1}")
            print("signed: true")
        else:
            print(f"precision: {ssiz + 1}")
            print("signed: false")

        cod_offset = lsiz + 4
        if codestream[cod_offset:cod_offset + 2] == b'\xff\x52':
            # 0 for none, 1 for applied to components 0, 1, 2
            print(f"MCT: {codestream[cod_offset + 8]}")
            # 0 for DWT 9-7 (irreversible), 1 for DWT 5-3 (reversible)
            print(f"DWT: {codestream[cod_offset + 13]}")

im = Image.open(b)
assert np.array_equal(ref, np.asarray(im))

Produces:

len: 3506536
precision: 8
signed: false
MCT: 0
DWT: 1

For comparison, with Jpeg2KEncode.c modified to use MCT:

len: 2682646
precision: 8
signed: false
MCT: 1
DWT: 1

With the transform into YCbCr the compression ratio is higher, as expected, while still being reversible.

What are your OS, Python and Pillow versions?

• OS: Ubuntu 20.04
• Python: 3.9
• Pillow: current master d393cfb

[radarhere]

Looking at the Pillow source code, we're not doing anything beyond just passing the irreversible option onto OpenJPEG.

Searching for documentation about this option in OpenJPEG, I found https://github.com/uclouvain/openjpeg/wiki/DocJ2KCodec

Irreversible compression (ICT + DWT 9-7). This option offers the possibility to work with the Irreversible Color Transformation (ICT) in place of the Reversible Color Transformation (RCT) and with the irreversible DWT 9-7 in place of the 5-3 filter. Default: off.

So our documentation pretty closely matches that official description.

[scaramallion]

Well no, the ICT/RCT colour transforms are only applied if MCT is 1. Since you don't use MCT then no colour transform is used. The docs literally says

use the lossy Irreversible Color Transformation followed by DWT 9-7

Which is only half true, because there's no colour transform.

What openjpeg are saying is that having an irreversible DWT then allows ICT if MCT is enabled (and a reversible one allows RCT if MCT is enabled). But MCT itself is completely optional, you're free to perform the compression on whatever colour space your input data is in (even though doing so would result in a lower compression ratio).

From the ISO/IEC 15444-1 J2K Standard:

G.2 Reversible multiple component transformation (RCT)
The use of the reversible multiple component transformation is signaled in the COD marker segment (see Annex A.6.1).
The RCT shall be used only with the 5-3 reversible filter.

The signal in the COD segment is the MCT value I flagged in the code example, which shows it disabled.

[scaramallion]

I do have a branch which adds MCT as an option when saving. Not sure if it's suitable for a PR.