deejjjaaaa: yes, you're right, that link was one of about 20+ ongoing posts since December 2019 on the Affinity Photo support forum.
Alex: I apologize, as I can only trust what the authoritative people each tell me, and Affinity (as clearly as they seem to be able to) say they have escalated this to LibRaw, and that it is a LibRaw issue, having nothing to do with them. Pretty well the same as LibRaw feels it is not in their power to remedy this, but rather Affinity's.
Since I seem to have a "he said - he said' issue, would you be kind enough to provide any constructive steps to isolate whom I can work with to resolve this?
There's a good number of users for these 5 cameras with the same issues, and since this sensor's image processing works similarly to prior models, it might hopefully be something reproducible. I've lost nearly a year and a half unsuccessfully trying to obtain support on this one issue from Affinity, so anything that 'moves the needle forward' will be hugely appreciated!
Please also note that LibRaw will typically scale your raw values between the black and white levels during the linearization step (e.g. 65535*(x - black)/(white-black)), so you might get larger values even for gamma=1.0.
Unfortunately I found that if you set no_auto_scale as well to disable this, you also kill LibRaw's white balance for some reason, which is typically not good for the demosaicing step, see https://github.com/letmaik/rawpy/issues/101
I can’t understand what kind of additional support you require from our library.
LibRaw provides access to RAW data for calling application (e.g. Affinity). Calling application is responsible for this data interpretation and processing (e.g. channel mixing if one want to treat Foveon image as BW).
I think I found the issue (mix of versions in my lib folder). Sorry for that.
Sorry to ask you that but do you have any idea about the date of the next stable release ?
Attached one file, but I have same issue with other CR3,
Maybe I'm doing something wrong, but my code is working for all other formats. So I don't know.
I have generated the backtrace from valgrind. Here you go:
==23395== Process terminating with default action of signal 11 (SIGSEGV)
==23395== Bad permissions for mapped region at address 0x5185000
==23395== at 0x4EC4DBB: LibRaw::adobe_coeff(unsigned int, char const*, int) (colordata.cpp:1697)
==23395== by 0x4EAC172: LibRaw::GetNormalizedModel() (normalize_model.cpp:1157)
==23395== by 0x4E9C07D: LibRaw::identify() (identify.cpp:1071)
==23395== by 0x4EC7C95: LibRaw::open_datastream(LibRaw_abstract_datastream*) (open.cpp:376)
==23395== by 0x4EC94F0: LibRaw::open_file(char const*, long long) (open.cpp:52)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== HEAP SUMMARY:
==23395== in use at exit: 298,973 bytes in 9 blocks
==23395== total heap usage: 27 allocs, 18 frees, 343,743 bytes allocated
==23395==
==23395== Searching for pointers to 9 not-freed blocks
==23395== Checked 1,382,688 bytes
==23395==
==23395== 8 bytes in 1 blocks are still reachable in loss record 1 of 9
==23395== at 0x4C29E96: malloc (vg_replace_malloc.c:309)
==23395== by 0x5E1A868: ??? (in /usr/lib64/libgomp.so.1.0.0)
==23395== by 0x5E2968B: ??? (in /usr/lib64/libgomp.so.1.0.0)
==23395== by 0x5E18EC6: ??? (in /usr/lib64/libgomp.so.1.0.0)
==23395== by 0x400F4C2: _dl_init (in /usr/lib64/ld-2.17.so)
==23395== by 0x40011A9: ??? (in /usr/lib64/ld-2.17.so)
==23395== by 0x2: ???
==23395== by 0x1FFF0003D2: ???
==23395== by 0x1FFF0003DF: ???
==23395== by 0x1FFF0003E2: ???
==23395==
==23395== 37 bytes in 1 blocks are still reachable in loss record 2 of 9
==23395== at 0x4C2A4B6: operator new(unsigned long) (vg_replace_malloc.c:344)
==23395== by 0x58C4CF8: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x58C6580: char* std::string::_S_construct(char const*, char const*, std::allocator const&, std::forward_iterator_tag) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x58C69B7: std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x4E56AA5: LibRaw_file_datastream::LibRaw_file_datastream(char const*) (libraw_datastream.cpp:56)
==23395== by 0x4EC948D: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 40 bytes in 1 blocks are still reachable in loss record 3 of 9
==23395== at 0x4C2A4B6: operator new(unsigned long) (vg_replace_malloc.c:344)
==23395== by 0x4EC947F: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 240 bytes in 1 blocks are still reachable in loss record 4 of 9
==23395== at 0x4C2A4B6: operator new(unsigned long) (vg_replace_malloc.c:344)
==23395== by 0x4E56AEA: LibRaw_file_datastream::LibRaw_file_datastream(char const*) (libraw_datastream.cpp:70)
==23395== by 0x4EC948D: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 568 bytes in 1 blocks are still reachable in loss record 5 of 9
==23395== at 0x4C29E96: malloc (vg_replace_malloc.c:309)
==23395== by 0x64D458C: __fopen_internal (in /usr/lib64/libc-2.17.so)
==23395== by 0x58834FF: std::__basic_file::open(char const*, std::_Ios_Openmode, int) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x58BE7F9: std::basic_filebuf >::open(char const*, std::_Ios_Openmode) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x4E56B06: LibRaw_file_datastream::LibRaw_file_datastream(char const*) (libraw_datastream.cpp:74)
==23395== by 0x4EC948D: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 2,560 bytes in 1 blocks are still reachable in loss record 6 of 9
==23395== at 0x4C29E96: malloc (vg_replace_malloc.c:309)
==23395== by 0x4ECC062: malloc (libraw_alloc.h:49)
==23395== by 0x4ECC062: LibRaw::malloc(unsigned long) (utils_libraw.cpp:239)
==23395== by 0x4EA59D4: LibRaw::parse_makernote(int, int) (makernotes.cpp:527)
==23395== by 0x4E93820: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:503)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9DC41: LibRaw::identify() (identify.cpp:719)
==23395== by 0x4EC7C95: LibRaw::open_datastream(LibRaw_abstract_datastream*) (open.cpp:376)
==23395== by 0x4EC94F0: LibRaw::open_file(char const*, long long) (open.cpp:52)
==23395==
==23395== 4,096 bytes in 1 blocks are still reachable in loss record 7 of 9
==23395== at 0x4C29E96: malloc (vg_replace_malloc.c:309)
==23395== by 0x4EC670C: libraw_memmgr (libraw_alloc.h:36)
==23395== by 0x4EC670C: LibRaw::LibRaw(unsigned int) (init_close_utils.cpp:26)
==23395== by 0x400D75: main (simple_dcraw.cpp:64)
==23395==
==23395== 8,192 bytes in 1 blocks are still reachable in loss record 8 of 9
==23395== at 0x4C2AB5B: operator new[](unsigned long) (vg_replace_malloc.c:433)
==23395== by 0x58BE1CB: std::basic_filebuf >::_M_allocate_internal_buffer() (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x58BE811: std::basic_filebuf >::open(char const*, std::_Ios_Openmode) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x4E56B06: LibRaw_file_datastream::LibRaw_file_datastream(char const*) (libraw_datastream.cpp:74)
==23395== by 0x4EC948D: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 283,232 bytes in 1 blocks are definitely lost in loss record 9 of 9
==23395== at 0x4C2A4B6: operator new(unsigned long) (vg_replace_malloc.c:344)
==23395== by 0x4EC6BFF: LibRaw::LibRaw(unsigned int) (init_close_utils.cpp:104)
==23395== by 0x400D75: main (simple_dcraw.cpp:64)
==23395==
==23395== LEAK SUMMARY:
==23395== definitely lost: 283,232 bytes in 1 blocks
==23395== indirectly lost: 0 bytes in 0 blocks
==23395== possibly lost: 0 bytes in 0 blocks
==23395== still reachable: 15,741 bytes in 8 blocks
==23395== of which reachable via heuristic:
==23395== stdstring : 37 bytes in 1 blocks
==23395== suppressed: 0 bytes in 0 blocks
==23395==
==23395== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault
[siddaharth.suman@sid samples]$
deejjjaaaa: yes, you're right, that link was one of about 20+ ongoing posts since December 2019 on the Affinity Photo support forum.
Alex: I apologize, as I can only trust what the authoritative people each tell me, and Affinity (as clearly as they seem to be able to) say they have escalated this to LibRaw, and that it is a LibRaw issue, having nothing to do with them. Pretty well the same as LibRaw feels it is not in their power to remedy this, but rather Affinity's.
Since I seem to have a "he said - he said' issue, would you be kind enough to provide any constructive steps to isolate whom I can work with to resolve this?
There's a good number of users for these 5 cameras with the same issues, and since this sensor's image processing works similarly to prior models, it might hopefully be something reproducible. I've lost nearly a year and a half unsuccessfully trying to obtain support on this one issue from Affinity, so anything that 'moves the needle forward' will be hugely appreciated!
All my best,
DLJ
Please also note that LibRaw will typically scale your raw values between the black and white levels during the linearization step (e.g. 65535*(x - black)/(white-black)), so you might get larger values even for gamma=1.0.
Unfortunately I found that if you set no_auto_scale as well to disable this, you also kill LibRaw's white balance for some reason, which is typically not good for the demosaicing step, see https://github.com/letmaik/rawpy/issues/101
dcraw_ppm_tiff_writer() applies gamma curve on write (imgdata.image[] is linear, while output is usually not).
To set gamma curve to linear use
imgdata.params.gamm[0] = imgdata.params.gamm[1] = 1.0;
Thanx.
I still do not see anything that should be *fixed* on our side (hope Affinity postprocessing is own, not demo code provided by our library)
I think he refers to this thread = https://forum.affinity.serif.com/index.php?/topic/93631-outstanding-bw-o...
Also, you wrote:
> This issue was escalated to the LibRaw dev team by the Affinity (Serif) devs about a year ago
Sure? Never heard of something like this
I can’t understand what kind of additional support you require from our library.
LibRaw provides access to RAW data for calling application (e.g. Affinity). Calling application is responsible for this data interpretation and processing (e.g. channel mixing if one want to treat Foveon image as BW).
That's right.
But package are built only on stable release :).
Cheers,
current master is stable enough for production usage.
I think I found the issue (mix of versions in my lib folder). Sorry for that.
Sorry to ask you that but do you have any idea about the date of the next stable release ?
Cheers,
I also have a good tiff file.
But for a reason a I don't understand, libraw_open_file returns an error.
OK thanks. I've no doubt the issue is on my system then, thank you so much.
Stay safe,
Cyril
I recloned the git repo.
I wrote:
printf("version: %s\n", LIBRAW_VERSION_STR);
return(libraw_open_file(rawdata, name));
And I have:
version: 0.20.0-WorkInProgress
Error in libraw Unsupported file format or not RAW file
I need to investigate then.
I've tested with fresh git clone:
git clone https://github.com/LibRaw/LibRaw.git LibRaw.gh && cd LibRaw.gh && make -f Makefile.devel CXX=clang++ CC=clang -j4 && ./bin/dcraw_emu -T -w ~/1/canon_eos_m50_07.cr3
This results in ~/1/canon_eos_m50_07.cr3.tiff as expected
(note CXX=.. CC=.. is for my system with clang installed as c++ compiler).
Thank you for the sample. I do not see any problems with this file.
Please make sure you're using up-to-date master branch from Github,
Attached one file, but I have same issue with other CR3,
Maybe I'm doing something wrong, but my code is working for all other formats. So I don't know.
https://www.dropbox.com/s/38yf9tcqtvnihll/canon_eos_m50_07.cr3?dl=0
Could you please share sample file with us
Hello, with the master version I have the following error with m50 cr3:
Unsupported file format or not RAW file
Cheers,
CR3 support is published in Oct 2019.
How's the support for CR3 coming along? Thanks!
OK.
We'll disable gcc version less than 4.9 for 0.20 release.
Thanks Alex,
I updated my gcc to 7.3.1 and the issue was resolved!
--Siddaharth Suman
You're correct! My gcc version is at 4.8.5. Let me update and try again. I'll post the end result here.
--Siddaharth Suman
Looks like (known) gcc 4.8x problem with this code: https://github.com/LibRaw/LibRaw/issues/264
I have generated the backtrace from valgrind. Here you go:
==23395== Process terminating with default action of signal 11 (SIGSEGV)
==23395== Bad permissions for mapped region at address 0x5185000
==23395== at 0x4EC4DBB: LibRaw::adobe_coeff(unsigned int, char const*, int) (colordata.cpp:1697)
==23395== by 0x4EAC172: LibRaw::GetNormalizedModel() (normalize_model.cpp:1157)
==23395== by 0x4E9C07D: LibRaw::identify() (identify.cpp:1071)
==23395== by 0x4EC7C95: LibRaw::open_datastream(LibRaw_abstract_datastream*) (open.cpp:376)
==23395== by 0x4EC94F0: LibRaw::open_file(char const*, long long) (open.cpp:52)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== HEAP SUMMARY:
==23395== in use at exit: 298,973 bytes in 9 blocks
==23395== total heap usage: 27 allocs, 18 frees, 343,743 bytes allocated
==23395==
==23395== Searching for pointers to 9 not-freed blocks
==23395== Checked 1,382,688 bytes
==23395==
==23395== 8 bytes in 1 blocks are still reachable in loss record 1 of 9
==23395== at 0x4C29E96: malloc (vg_replace_malloc.c:309)
==23395== by 0x5E1A868: ??? (in /usr/lib64/libgomp.so.1.0.0)
==23395== by 0x5E2968B: ??? (in /usr/lib64/libgomp.so.1.0.0)
==23395== by 0x5E18EC6: ??? (in /usr/lib64/libgomp.so.1.0.0)
==23395== by 0x400F4C2: _dl_init (in /usr/lib64/ld-2.17.so)
==23395== by 0x40011A9: ??? (in /usr/lib64/ld-2.17.so)
==23395== by 0x2: ???
==23395== by 0x1FFF0003D2: ???
==23395== by 0x1FFF0003DF: ???
==23395== by 0x1FFF0003E2: ???
==23395==
==23395== 37 bytes in 1 blocks are still reachable in loss record 2 of 9
==23395== at 0x4C2A4B6: operator new(unsigned long) (vg_replace_malloc.c:344)
==23395== by 0x58C4CF8: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator const&) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x58C6580: char* std::string::_S_construct(char const*, char const*, std::allocator const&, std::forward_iterator_tag) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x58C69B7: std::basic_string, std::allocator >::basic_string(char const*, std::allocator const&) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x4E56AA5: LibRaw_file_datastream::LibRaw_file_datastream(char const*) (libraw_datastream.cpp:56)
==23395== by 0x4EC948D: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 40 bytes in 1 blocks are still reachable in loss record 3 of 9
==23395== at 0x4C2A4B6: operator new(unsigned long) (vg_replace_malloc.c:344)
==23395== by 0x4EC947F: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 240 bytes in 1 blocks are still reachable in loss record 4 of 9
==23395== at 0x4C2A4B6: operator new(unsigned long) (vg_replace_malloc.c:344)
==23395== by 0x4E56AEA: LibRaw_file_datastream::LibRaw_file_datastream(char const*) (libraw_datastream.cpp:70)
==23395== by 0x4EC948D: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 568 bytes in 1 blocks are still reachable in loss record 5 of 9
==23395== at 0x4C29E96: malloc (vg_replace_malloc.c:309)
==23395== by 0x64D458C: __fopen_internal (in /usr/lib64/libc-2.17.so)
==23395== by 0x58834FF: std::__basic_file::open(char const*, std::_Ios_Openmode, int) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x58BE7F9: std::basic_filebuf >::open(char const*, std::_Ios_Openmode) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x4E56B06: LibRaw_file_datastream::LibRaw_file_datastream(char const*) (libraw_datastream.cpp:74)
==23395== by 0x4EC948D: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 2,560 bytes in 1 blocks are still reachable in loss record 6 of 9
==23395== at 0x4C29E96: malloc (vg_replace_malloc.c:309)
==23395== by 0x4ECC062: malloc (libraw_alloc.h:49)
==23395== by 0x4ECC062: LibRaw::malloc(unsigned long) (utils_libraw.cpp:239)
==23395== by 0x4EA59D4: LibRaw::parse_makernote(int, int) (makernotes.cpp:527)
==23395== by 0x4E93820: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:503)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9331D: LibRaw::parseCR3(unsigned long long, unsigned long long, short&, char*, short&, short&) (cr3_parser.cpp:519)
==23395== by 0x4E9DC41: LibRaw::identify() (identify.cpp:719)
==23395== by 0x4EC7C95: LibRaw::open_datastream(LibRaw_abstract_datastream*) (open.cpp:376)
==23395== by 0x4EC94F0: LibRaw::open_file(char const*, long long) (open.cpp:52)
==23395==
==23395== 4,096 bytes in 1 blocks are still reachable in loss record 7 of 9
==23395== at 0x4C29E96: malloc (vg_replace_malloc.c:309)
==23395== by 0x4EC670C: libraw_memmgr (libraw_alloc.h:36)
==23395== by 0x4EC670C: LibRaw::LibRaw(unsigned int) (init_close_utils.cpp:26)
==23395== by 0x400D75: main (simple_dcraw.cpp:64)
==23395==
==23395== 8,192 bytes in 1 blocks are still reachable in loss record 8 of 9
==23395== at 0x4C2AB5B: operator new[](unsigned long) (vg_replace_malloc.c:433)
==23395== by 0x58BE1CB: std::basic_filebuf >::_M_allocate_internal_buffer() (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x58BE811: std::basic_filebuf >::open(char const*, std::_Ios_Openmode) (in /usr/lib64/libstdc++.so.6.0.19)
==23395== by 0x4E56B06: LibRaw_file_datastream::LibRaw_file_datastream(char const*) (libraw_datastream.cpp:74)
==23395== by 0x4EC948D: LibRaw::open_file(char const*, long long) (open.cpp:38)
==23395== by 0x400DDC: main (simple_dcraw.cpp:122)
==23395==
==23395== 283,232 bytes in 1 blocks are definitely lost in loss record 9 of 9
==23395== at 0x4C2A4B6: operator new(unsigned long) (vg_replace_malloc.c:344)
==23395== by 0x4EC6BFF: LibRaw::LibRaw(unsigned int) (init_close_utils.cpp:104)
==23395== by 0x400D75: main (simple_dcraw.cpp:64)
==23395==
==23395== LEAK SUMMARY:
==23395== definitely lost: 283,232 bytes in 1 blocks
==23395== indirectly lost: 0 bytes in 0 blocks
==23395== possibly lost: 0 bytes in 0 blocks
==23395== still reachable: 15,741 bytes in 8 blocks
==23395== of which reachable via heuristic:
==23395== stdstring : 37 bytes in 1 blocks
==23395== suppressed: 0 bytes in 0 blocks
==23395==
==23395== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault
[siddaharth.suman@sid samples]$
Pages