SIGFPE on specially crafted jpeg - b6029d31

[jodiecunningham]

Hi Mozilla team,

I am running from the master branch here.

The following file causes a SIGFPE in lt-cjpeg on my platform:
https://www.dropbox.com/s/a03ddlno6v6ubzz/b6029d31?dl=0

To reproduce:
cjpeg -outfile /dev/null b6029d31

Output:
Aborted

GDB output, (not very useful at the moment, I am going to work on that):

Program terminated with signal SIGFPE, Arithmetic exception.
#0  0x00007ffff7b9e598 in alloc_sarray () from /home/jodicun/opt/mozjpeg-master-clean/.libs/libjpeg.so.62
#0  0x00007ffff7b9e598 in alloc_sarray () from /home/jodicun/opt/mozjpeg-master-clean/.libs/libjpeg.so.62
#1  0x000000000040581d in start_input_bmp ()
#2  0x0000000000401df8 in main ()
exe = '/home/jodicun/opt/mozjpeg-master-clean/.libs/lt-cjpeg -outfile /dev/null /home/'

System Details:
AMD64
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty

Found with the fuzzer American Fuzzy Lop ( http://lcamtuf.coredump.cx/afl/ )

[CoolOppo]

Good find.

[dcommander]

The root of this problem is that rdbmp.c uses the ambiguous INT32 type. Windows bitmap headers are defined with very specific types:

https://msdn.microsoft.com/en-us/library/windows/desktop/dd183376%28v=vs.85%29.aspx

(DWORD = unsigned int, LONG = int, WORD = unsigned short)

libjpeg-turbo/libjpeg-turbo@3a6cadf fixes this by using the appropriate types that match the Windows bitmap header spec.