dragonfly is not threadsafe

[tolsen]

I tried running my Rails app using JRuby in threadsafe mode but saw what appeared to be a race condition. Basically if two requests came in around the same time, both resulting in dragonfly jobs for resizing different images, one image would be resized to the size that the other image was supposed to be resized to. For example, if a requests came in to resize image A to X by Y and image B to U by V, then sometimes image A would be resized to U by V!

The problem probably stems from a Dragonfly app not being itself threadsafe. If two threads request Dragonfly[:images], they will get back the same app and will try to run jobs on it simultaneously.

I started to look at the Dragonfly code and quickly came across an instance of non-threadsafe code on Dragonfly::App at https://github.com/markevans/dragonfly/blob/master/lib/dragonfly/app.rb#L21

      def apps
        @apps ||= {}
      end

And there's similar instances of the non threadsafe pattern @var ||= foo in other parts of the code. That pattern is probably not what's causing the race condition I'm seeing. That's just the start of a threadsafety audit.

Being able to run in threadsafe mode in jruby setup results in substantially decreased memory usage. It would be great to be able to use dragonfly in such a setup. Furthermore, with Rails 4.0 turning on threadsafe mode by default, threadsafety is becoming increasingly important in general in the Rails community.

[markevans]

Thanks for the heads-up

the pattern @var ||= 'blah' is neither thread-safe or thread-unsafe in itself as far as I can tell (though I'm no threading expert!) - it's more to do with sharing global variables across requests (the Dragonfly apps themselves being an example)

However I have no idea why your specific example would happen

If you or anyone else would be able to highlight any specific places that are not threadsafe I'd be grateful - I would like it to be, though I'm no expert and I haven't had time or needed to deal with it yet. I will have a look when I get the chance though

On 13 Sep 2012, at 16:57, Tim Olsen wrote:

I tried running my Rails app using JRuby in threadsafe mode but saw what appeared to be a race condition. Basically if two requests came in around the same time, both resulting in dragonfly jobs for resizing different images, one image would be resized to the size that the other image was supposed to be resized to. For example, if a requests came in to resize image A to X by Y and image B to U by V, then sometimes image A would be resized to U by V!

The problem probably stems from a Dragonfly app not being itself threadsafe. If two threads request Dragonfly[:images], they will get back the same app and will try to run jobs on it simultaneously.

I started to look at the Dragonfly code and quickly came across an instance of non-threadsafe code on Dragonfly::App at https://github.com/markevans/dragonfly/blob/master/lib/dragonfly/app.rb#L21

  def apps
    @apps ||= {}
  end

And there's similar instances of the non threadsafe pattern @var ||= foo in other parts of the code. That pattern is probably not what's causing the race condition I'm seeing. That's just the start of a threadsafety audit.

Being able to run in threadsafe mode in jruby setup results in substantially decreased memory usage. It would be great to be able to use dragonfly in such a setup. Furthermore, with Rails 4.0 turning on threadsafe mode by default, threadsafety is becoming increasingly important in general in the Rails community.

—
Reply to this email directly or view it on GitHub.

[tolsen]

Right, @var ||= 'blah' is not a problem if that code is only executed by one thread for any object instance. In the example I give, apps() is a singleton method on Dragonfly::App so that code could be executed simultaneously by two different threads.

[markevans]

I'm going to close this for now as it's not been updated in a while. feel free to reopen if it's still a problem and can be reproduced

[jf]

I don't know if this is the source of the problems I'm seeing, but I have to concur: after a lot of troubleshooting, it seems like there are race conditions in Dragonfly.

If I get the chance to fix it, I'll be sure to send in a patch. But for the moment, pls consider that there may be problems that are not resolved yet. I have an app using Dragonfly right now, and all I have to do is hit it consistently with a respectable load (100+ requests), and you can be sure that somewhere along the way, Dragonfly will fail (with Errno::ENOENTs, etc.). I've tried switching ruby versions (between MRI, and Rubinius), and also checking other bits of the app.... It's only in the image handling bits by Dragonfly that I get race conditions.