Add LockingFileListFilter to allow multiple processes to watch the same directory [INT-768]

[spring-operator]

Iwein Fuld opened INT-768* and commented

In some usecases (see forum post) it is needed to prevent files from being picked up by another process. This can be done using lock files.

---

Affects: 1.0.3

Reference URL: http://forum.springsource.org/showthread.php?p=256330

Issue Links:

• Add namespace support for File locking [INT-821] #4827 Add namespace support for File locking

• Merge scanner and filter in FileReadingMessageSource [INT-947] #4931 Merge scanner and filter in FileReadingMessageSource

[spring-operator]

Iwein Fuld commented

Changed fix version (as the changes are in HEAD now)

[spring-operator]

Iwein Fuld commented

I've added a draft for nio support. I also added a deleteOnExit on the file lock variant. Maybe we should drop the lock file variant altogether, nio seems cleaner, although it does hog some more memory in case there are a lot of files to lock.

[spring-operator]

Mark Fisher commented

I agree that the FileChannel based version is nice. I do have two questions:

1. Who would be expected to call unlock()?
2. Do we need a FileListFilter implementation or should we consider just adding the fileLocker as a property of the FileReadingMessageSource directly?

Ultimately we will want to add namespace support for this but only after we feel comfortable answering those questions. Since what we have in place is ready for use, perhaps we should release this in M1 to let people try it. Then in M2 we could add namespace support. My only concern is that people might start using it, and if we switch to just the "fileLocker" property, that would require changes. However, in a milestone release, these types of changes should be expected... that type of experimentation and feedback is necessary to get it right before the RC and Final releases.

[spring-operator]

Iwein Fuld commented

ad 1)
I'd expect the lock to be released either when the process ends, or when the file is processed (which would mean the file message is routed back to the source). There is no namespace support for it, but we do have an onMessageProcessed method.

Another (better) strategy is to pass the lock as a message header. Then the downstream component can unlock, delete, move as appropriate. All these things are open to experiments now, we can add more official support for them based on the feedback we get.

ad 2)
I'm thinking the nicest way to do it is to react implicitly to FileListFilter's implementing FileLocker that would avoid namespace changes and keep this feature a little out of your face if you're not interested (which should be the preferred option imo).

[spring-operator]

Iwein Fuld commented

After some discussion it has been decided that this is ready for M1. Subsequent issues will follow from feedback.

[spring-operator]

Mark Fisher commented

I added #4827 as a follow-up to this issue in the M2 timeframe.