Is there a known regression in 0.17 ?

Submitted by lock on

Hello, I'm using libraw since a while in my opensource software.
However, since 0.17 I have a crash I did not have before.
gdb gives me:

#2 0x0000000000432b75 in readraw (name=0x16c95b0 "/home/lock/Bureau/test.CR2", fit=0x703680 ) at io/image_formats_libraries.c:747
747 ret = libraw_dcraw_process(raw);
(gdb) p raw
$1 = (libraw_data_t *) 0x1cdb008
(gdb) p *raw
value of type `libraw_data_t' requires 299696 bytes, which is more than max-value-size
(gdb) l
742 libraw_recycle(raw);
743 libraw_close(raw);
744 return -1;
745 }
746
747 ret = libraw_dcraw_process(raw);
748 if (ret) {
749 printf("Error in libraw %s\n", libraw_strerror(ret));
750 free(data);
751 libraw_recycle(raw);

The whole code can be seen here:
https://free-astro.org/svn/siril/trunk/src/io/image_formats_libraries.c

Could you explain me the problem ?

Best Regards,
Cyril

Forums: 

Your backtrace is not

Submitted by lexa on

Your backtrace is not informative, sorry.

Is the crash repeatable? Is it occurs with libraw own examples too? If both 'yes', could you please share problematic raw file?

-- Alex Tutubalin @LibRaw LLC

Sorry,

Submitted by lock on

Sorry,
the crash is repeatable yes. It only occurs with 0.17.2 because I have no problem with 0.17.1. I use the same code for at least one year with no problem or crash. So this is the reason I think it could be a regression.

The file I tested is here: https://www.dropbox.com/s/nfpdqt9cqvmlfu4/test.CR2?dl=0

The backtrace:
#0 0x00007ffff58d8158 in LibRaw::vng_interpolate() () from /usr/lib/x86_64-linux-gnu/libraw.so.15
#1 0x00007ffff590f738 in LibRaw::dcraw_process() () from /usr/lib/x86_64-linux-gnu/libraw.so.15
#2 0x0000000000432b75 in readraw (name=0x1be3850 "/home/lock/Bureau/test.CR2", fit=0x703680 ) at io/image_formats_libraries.c:747
#3 0x00000000004334d4 in open_raw_files (name=0x1be3850 "/home/lock/Bureau/test.CR2", fit=0x703680 , type=0)
at io/image_formats_libraries.c:900
#4 0x000000000042cb13 in any_to_fits (imagetype=TYPERAW, source=0x1be3850 "/home/lock/Bureau/test.CR2", dest=0x703680 ) at io/conversion.c:797
#5 0x0000000000441cc3 in read_single_image (filename=0x1c0eaf0 "/home/lock/Bureau/test.CR2", dest=0x703680 , realname_out=0x7fffffffd3e8)
at io/single_image.c:120
#6 0x0000000000441d6e in open_single_image (filename=0x1c0eaf0 "/home/lock/Bureau/test.CR2") at io/single_image.c:145
#7 0x0000000000447cc7 in opendial () at gui/callbacks.c:1684
#8 0x0000000000447de2 in on_open1_activate (menuitem=0x13f3f80, user_data=0x0) at gui/callbacks.c:1721
#9 0x00007ffff5e91fa5 in g_closure_invoke () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#10 0x00007ffff5ea3fc1 in ?? () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#11 0x00007ffff5eacd5c in g_signal_emit_valist () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#12 0x00007ffff5ead08f in g_signal_emit () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#13 0x00007ffff785b0be in gtk_widget_activate () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#14 0x00007ffff7732376 in gtk_menu_shell_activate_item () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#15 0x00007ffff77326ab in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#16 0x00007ffff7716991 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#17 0x00007ffff5e921d4 in ?? () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#18 0x00007ffff5eac4b8 in g_signal_emit_valist () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#19 0x00007ffff5ead08f in g_signal_emit () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#20 0x00007ffff7858cbc in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#21 0x00007ffff7713abe in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#22 0x00007ffff7715a42 in gtk_main_do_event () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#23 0x00007ffff724b795 in ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
#24 0x00007ffff7278762 in ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
#25 0x00007ffff5bbb1a7 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#26 0x00007ffff5bbb400 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#27 0x00007ffff5bbb722 in g_main_loop_run () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#28 0x00007ffff7714b75 in gtk_main () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#29 0x000000000041d909 in main (argc=1, argv=0x7fffffffe198) at main.c:351

The only difference between 0

Submitted by lexa on

The only difference between 0.17.1 and 0.17.2 is strncpy usage: https://github.com/LibRaw/LibRaw/commit/c4551c868d6986477071315e9e847de5...

This change affects only Sony files processing, but your sample looks like it canon file (.CR2)

So, two questions:
1)what version of gcc compiler do you use (have you switched to gcc6?)
2) do you really use 0.17.2 or 0.18-stable from github (look in libraw\libraw_version.h)

-- Alex Tutubalin @LibRaw LLC

Hello.

Submitted by lock on

Hello.

That is strange indeed.
I use the libraw in debian testing repository : https://tracker.debian.org/pkg/libraw, 0.17.2-4

I've tested my software on ubuntu 16.04 (with libraw 0.17.1) with no errors.
But in testing the crash seems to occur only with VNG interpolation. (not AHD)

My gcc is gcc-6 indeed but same error with gcc-5

Best regards,
Cyril

could you please provide

Submitted by lexa on

could you please provide exact git command line to fetch exactly the branch you use?
Debian repo (tag system) is unknown for me, so I do not know what tag i need to fetch.

Most likely, debian people has used bad patch from libraw-0.18 (for gcc6 compatibility). This patch (circulating around internet) broke vng interpolation code (but makes libraw source compatible with gcc6)

-- Alex Tutubalin @LibRaw LLC

LibRaw 0.17 (and before)

Submitted by lexa on

LibRaw 0.17 (and before) known to be not compilable by gcc6. So if you're using gcc6 and it builds OK, it means you use some modified source.

The patch proposed by community brokes 4-color processing in vng_interpolate. meanwhile, it looks like Debian community use this patch for debian repo: https://github.com/LibRaw/LibRaw/issues/56

(And, yes, they know that the patch is broken, see discussion in github issue thread)

-- Alex Tutubalin @LibRaw LLC

followup: I'm unable to find

Submitted by lexa on

followup: I'm unable to find right branch/tag in debian repo. Master branch is unchanged LibRaw 0.17.2, so it should work with DNG, but will not compile with gcc6.

Looks like you use some specific tag/branch?

-- Alex Tutubalin @LibRaw LLC

And followup again.

Submitted by lexa on

And followup again.
Here is the patch: https://github.com/LibRaw/LibRaw/commit/eba7a66e23f2f62cc4c8115af8066f2d...

So, if in your copy of libraw/internal/dcraw_common.cpp
vng_interolate() code starts with:

  struct interpolate_terms {
    signed char y1, x1, y2, x2, weight;
    unsigned char grads;
  };

than your libraw copy is indeed patched by this wrong patch.
Do not use VNG (and use patched library), or do not use gcc6 (and use unpatched library), or wait for autumn LibRaw-0.18 public snapshot (or, may be, 0.18-release)

-- Alex Tutubalin @LibRaw LLC

Thank you so much for your

Submitted by lock on

Thank you so much for your answer.

The most important thing I do not want is that the patched library is released on a long term support system.
But it seems ok for that because Ubuntu 16.04 is using 0.17.1. However, next Ubuntu will have this problem I think: https://launchpad.net/ubuntu/yakkety/+source/libraw because they apparently use same sources than testing.

0.18 should fix it ? If yes no problem for me, I will wait. But users with Ubuntu 16.10 will probably crash, except if someone patch the 0.17.2 to fix the bug. So after that we can ask to Ubuntu dev's team to fix the package.

Thanks again

LibRaw 0.17.2 (official

Submitted by lexa on

LibRaw 0.17.2 (official release from this site) do not have this bug (not crashes on VNG), but do not compile with gcc6.
If someone uses (broken) 3rd party patch - we're not responsible for that.

LibRaw 0.18-201610xx (i expect next public snapshot in october or so) will both compile by gcc6 and will do VNG :)

Current libraw 0.18 public snapshot (this is not release!) from https://github.com/LibRaw/LibRaw is the same as LibRaw 0.17.2: no patch, no bug, no gcc6 compile

-- Alex Tutubalin @LibRaw LLC

Sorry, current public

Submitted by lexa on

Sorry, current public snapshot contains this broken patch. To be reverted today

-- Alex Tutubalin @LibRaw LLC

OK.

Submitted by lock on

OK.

Glad to see that it will be fixed.

Thank you.

and checked: dcraw_emu -q 1

Submitted by lexa on

and checked: dcraw_emu -q 1 works fine with your sample

-- Alex Tutubalin @LibRaw LLC