Reconcile IJ1 <-> IJ2 synchronization and object conversion

[imagejan]

There are currently multiple open issues related to IJ1 <-> IJ2 image conversion:

• (Verify overlay translation code happening everywhere it needs to #85) Verify overlay translation code happening everywhere it needs to
• (Converting a Dataset to ImagePlus implicitly shows it #134) Converting a Dataset to ImagePlus implicitly shows it
• (# @ImgPlus img Loads VirtualStack into RAM #144) #@ImgPlus img Loads VirtualStack into RAM
• (Be more proactive syncing ImagePlus to ImageDisplay #184) Be more proactive syncing ImagePlus to ImageDisplay
• (Use legacy layer to lookup active IJ1 objects indirectly #185) Use legacy layer to lookup active IJ1 objects indirectly
• (Converters between Dataset and ImagePlus do too much #186) Converters between Dataset and ImagePlus do too much
• (Add converters between ImgLib2 RAI and ImageJ1 ImagePlus #229) Add converters between ImgLib2 RAI and ImageJ1 ImagePlus
• (ImagePlus conversion leads to multiple compatible inputs being listed #230) ImagePlus conversion leads to multiple compatible inputs being listed
• (Scripts calling IJ1 plugins can fail on ImageStack.getProcessor upon second run #240) Scripts calling IJ1 plugins can fail on ImageStack.getProcessor upon second run
• (Conversion ImagePlus > Dataset > ImagePlus throws UnsupportedOperationException #241) Conversion ImagePlus > Dataset > ImagePlus throws UnsupportedOperationException
• (Fix ColorTableHarmonizer #259) Fix ColorTableHarmonizer

As mentioned by @ctrueden in #186 and #229 (comment), we should possibly separate conversion and (lazy) synchronization better and create thorough tests for all (reported and anticipated) use cases.

@ctrueden do you think it would be feasible to change behavior such that:

• Conversion between ImagePlus <=> Dataset,Img,ImageDisplay etc. does not change the LegacyImageMap
• Synchronization (i.e. update of the LegacyImageMap via LegacyService) happens only when new UI events happen (i.e. UIService#show() called, or ImagePlus object created/shown)

?

[ctrueden]

@imagejan Thanks for summarizing these. We can (and should) change the converter behavior not to touch the LegacyImageMap. The second point—changing when we synchronize—would require careful consideration and testing.

[imagejan]

An important point to test is if conversion/synchronization also works correctly on a higher level that allows interacting with how the current image is displayed (current slice, color map, display settings, etc.):

• ImagePlus <=> ImageDisplay
• ImagePlus <=> DatasetView

See also this discussion on the forum: Analyze the current slice of an ImgPlus in ImageJ2

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/analyze-the-current-slice-of-an-imgplus-in-imagej2/38238/3

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/get-active-image-index-in-an-imagej2-plugin-called-in-fiji/39764/3

[imagejan]

I think the comment in imagej/pyimagej#155 (comment) by @ctrueden is relevant here, also without the Python-related parts:

• Someone creates an ImageJ2/ImgLib2 object such as Dataset/Img/RandomAccessibleInterval (e.g. reading a file with SCIFIO, backed by a DiskCachedCellImg)
• The ImageJ2/ImgLib2 image object gets wrapped to an ImagePlus
• Some ImageJ-(1.x)-based plugin processes the ImagePlus which might change pixels in place
• You call a Command taking the original ImageJ2/ImgLib2 object as an input, expecting that the changes to the ImagePlus are synced.

The question is when (and at which performance cost) synchronization should be performed. Maybe a boolean option flag to distinguish between an "eager" and a "lazy" mode would make sense here, similar to auto_sync that @elevans suggests for pyimagej?

I am posting this here for reference, in case someone will have time to focus on this issue for one week during a future hackathon 😄

[maarzt]

Is somebody working on this image or should I maybe do it? @imagejan @ctrueden @hinerm

[imagejan]

I'm not sure if @hinerm or @elevans have this on their radar currently, but I guess that you'd be most welcome to work on it, @maarzt! 🙏

[elevans]

@maarzt @imagejan This issue is on our radar and in the queue. We're working on finishing up the PyImageJ publication, getting the PyImageJ documentation/examples in order and releasing a new version of PyImageJ (1.1.0). The way things are currently planned we will address this issue in the following release (not 1.1.0 as there is not enough time).