A good place to start IMHO is "Image sensors and signal processingfor digital still cameras", edited by Junichi Nakamura and "Way Beyond Monochrome" by Ralph Lambrecht. One book covers the digital part, the other speaks of photographic process.
To get flatten output you may try samples/unprocessed_raw sample
B-channel pattern you show looks ok (one value in 2x2 square), while R-channel is not. Could you please upload raw somewhere and provide the link for in-depth analysis?
1) To disable auto-brightening of images:
- set params.no_auto_bright to 1
- set params.adjust_maximum_thr to 0.0 (in this case, for some cameras you may need to set params.user_sat /white point/ manually to avoid pink tint in highlights)
Afterh that, images (with same exposure) will be processed with same data maximum and no automatic brightening.
2) To adjust brightness (at output stage) use params.bright to some value above 1.0
Thank you so much! Is there any way that I can scale the colors manually, to make the above two images the same?
Also, what does auto_bright exactly do? You said in some post that it changes the histogram to make the image brighter. Is it histogram equalization on the output sRGB image?
I open one of my images in rawdigger and noticed that the min and max values for the colours are really different:
(values are min max avg and sigma)
R 2 2521 307 446
G 8 4045 716 1004
B 1 4046 683 1038
G2 5 4045 716 1004
This pef file can be found here: https://drive.google.com/file/d/1TcCcoIL-q7K85UAJC4FcwtpDdlihDuoP/view?u...
Is there anything I can do to correct this image (and about 50 other images I have with similar problem)? There are some wedding pictures among them and I'd like to get these fixed...
There is certain amount of flare, and possibly the exposure was too much, but I can't say for sure without the raw. You can download free trial of our RawDigger and check the exposure yourself of course: https://www.rawdigger.com/download
I have created a jpg from my raw file, if needed I can also share the raw file. The fact is that for all raw images from a certain day, this effect is present in the same are (left-hand side of each picture). Or is it then the intensity that I need to reduce for the raw image?
There is no problem with white balance on the image. Possibly you mean that the shot contains blown-out areas, but I can't say how much of those can be brought back as the image you've uploaded is in fact JPEG, not raw PEF.
It is unclear how do you mean saying "white balance set too high". Could you please upload a sample raw and provide a link to it, to better understand the issue?
for(int i = 0; i < 1000; i++)
gamma_curve (O.gamm[0], O.gamm[1], 2, (t_white << 3)/O.bright);
And run Canon 5Ds image though profiler (Intel VTune amplifier). Gamma curve (1000 repetitions) takes 0.619 seconds, so usual one-time run should be about than 0.6 msec
CPU: i7-7700k at base speed.
Other processing: 411ms for decode, 700ms for scale colors, 900ms for (fastest available) linear interpolation, so about 2sec total.
So, gamma_curve is less than 1/3000 of total processing time for such large files.
Either your profiler is wrong, or your math library (pow(), log(), or exp() functions) is incredibly slow.
Well, after some tests and thinking, it downed to me that I should just adjust the gamma curve, there is no point in multiplying everything with a constant multiplyer.
So my function is now like this, and it works (but not exactly how it does in RawTherapee)
I also want to pick a single gamma curve and apply those same remap value to hundreds or thousands of images. So having the ability to directly manipulate the libraw_colordata_t.curve[] data in lieu of defining gammas would be VERY nice. I find on 50mp canon cr2 images calculating the gamma curve remap per image is almost 15s per image on an ivy bridge xeon.
Hi Alex,
I'm the rawpy developer and just had a look at this. I'm using LibRaw::COLOR(row, col) to construct the color pattern matrix, which works fine for non-x-trans images. But for x-trans any value after the first row returns 0. so ::COLOR(1, 0) is 0, as well as ::COLOR(1,1) is 0. I'm using libraw 0.18.2 currently and noticed some x-trans related changes in newer versions: https://github.com/LibRaw/LibRaw/compare/0.18.2...0.18.5 But they seem to be related to security only.
My test image is: http://img.photographyblog.com/reviews/fujifilm_x_t1/photos/fujifilm_x_t...
Can you check whether the COLOR method works correctly here? I think there may be a problem.
Thanks!
Maik
Amazing, yes you are completely right, that's the reason it was always failing. In gcc, I added
LOCAL_CFLAGS += -fsigned-char
to the make file and it works perfectly and decodes like a charm! As I read, the default of a char is not really defined in c and depends on the architecture, so this error makes perfectly sens. But may there would be a way to explicitly init is as a signed char directly in code so that these things will not happen to others?
so after quite a lot of coffee, I found some kind of translation method to translate the stacktrace to real code lines. Here's the result:
********** Crash dump: **********
signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x7246192bb8
Routine fuji_decode_sample_odd(fuji_compressed_block*, fuji_compressed_params const*, unsigned short*, int, int_pair*) at libraw_fuji_compressed.cpp:490
Routine LibRaw::xtrans_decode_block(fuji_compressed_block*, fuji_compressed_params const*, int) at libraw_fuji_compressed.cpp:727
Routine LibRaw::fuji_decode_strip(fuji_compressed_params const*, int, long long, unsigned int) at libraw_fuji_compressed.cpp:919
Routine LibRaw::fuji_decode_loop(fuji_compressed_params const*, int, long long*, unsigned int*) at libraw_fuji_compressed.cpp:999
Routine LibRaw::fuji_compressed_load_raw() at libraw_fuji_compressed.cpp:983
Routine LibRaw::unpack() at libraw_cxx.cpp:2811 (discriminator 3)
Routine libraw_unpack at libraw_c_api.cpp:130
Routine Java_com_tssystems_Libraw_open at libraw.c:51
For me it seems like the access of the grads array via the gradient variable runs further than the array, but I'm not sure. What do you think? Does any output of the variables may help?
Just a note: The value of the gradient variable before it fails is 2272 which seems way higher than all others (they are around 250). Maybe there's something going wrong here?
// Edit: The really strange thing is that the Ra... Rg values change each time I run it and the crash finally ocurse. For instance, now I get
Ra 16383 Rb 0 Rc 0 Rd 0 Rg 0
or
Ra 0 Rb 0 Rc 0 Rd 0 Rg 849
before it will crash. Seems there is something really going wrong with the values?
For some other pretty strange reason it almost seems like something is running parallel, because I can't see the full var dump of the q_table in onces. But this could also be a transfering issue of the log in android since the buffer speed is somewhat limited? I don't have openmp enabled.
A good place to start IMHO is "Image sensors and signal processingfor digital still cameras", edited by Junichi Nakamura and "Way Beyond Monochrome" by Ralph Lambrecht. One book covers the digital part, the other speaks of photographic process.
Got it. Thanks for taking the time to reply.
If you had any book recommendations that help explain how to do this postprocessing, I'd be very interested.
Thanks again, and thanks for making this amazing library available!
FRV does not use LibRaw for raw image postprocessing, only raw decode phase is used (LibRaw::open_file/LibRaw::unpack).
'Variable contrast' curves in FRV are based on real photographic film contrast curves.
To get flatten output you may try samples/unprocessed_raw sample
B-channel pattern you show looks ok (one value in 2x2 square), while R-channel is not. Could you please upload raw somewhere and provide the link for in-depth analysis?
1) To disable auto-brightening of images:
- set params.no_auto_bright to 1
- set params.adjust_maximum_thr to 0.0 (in this case, for some cameras you may need to set params.user_sat /white point/ manually to avoid pink tint in highlights)
Afterh that, images (with same exposure) will be processed with same data maximum and no automatic brightening.
2) To adjust brightness (at output stage) use params.bright to some value above 1.0
Thank you so much! Is there any way that I can scale the colors manually, to make the above two images the same?
Also, what does auto_bright exactly do? You said in some post that it changes the histogram to make the image brighter. Is it histogram equalization on the output sRGB image?
Thanks!
Hi Iliah,
Rawdigger gave me new insight :-)
I open one of my images in rawdigger and noticed that the min and max values for the colours are really different:
(values are min max avg and sigma)
R 2 2521 307 446
G 8 4045 716 1004
B 1 4046 683 1038
G2 5 4045 716 1004
This pef file can be found here:
https://drive.google.com/file/d/1TcCcoIL-q7K85UAJC4FcwtpDdlihDuoP/view?u...
Is there anything I can do to correct this image (and about 50 other images I have with similar problem)? There are some wedding pictures among them and I'd like to get these fixed...
Regards, Ronald.
Dear Sir:
There is certain amount of flare, and possibly the exposure was too much, but I can't say for sure without the raw. You can download free trial of our RawDigger and check the exposure yourself of course: https://www.rawdigger.com/download
Hi Iliah,
I have created a jpg from my raw file, if needed I can also share the raw file. The fact is that for all raw images from a certain day, this effect is present in the same are (left-hand side of each picture). Or is it then the intensity that I need to reduce for the raw image?
Regards, Ronald.
Dear Sir:
There is no problem with white balance on the image. Possibly you mean that the shot contains blown-out areas, but I can't say how much of those can be brought back as the image you've uploaded is in fact JPEG, not raw PEF.
Hi Iliah,
if you look at the following picture https://drive.google.com/file/d/1autVVPZejMNJjeltv9E4_o8JqW266qukEw/view... you can see the effect. Does this help to understand the issue with the pictures and what I'd like to accomplish?
Regards, Ronald.
It is unclear how do you mean saying "white balance set too high". Could you please upload a sample raw and provide a link to it, to better understand the issue?
Just changed (in copy_mem_image code)
From:
to
And run Canon 5Ds image though profiler (Intel VTune amplifier). Gamma curve (1000 repetitions) takes 0.619 seconds, so usual one-time run should be about than 0.6 msec
CPU: i7-7700k at base speed.
Other processing: 411ms for decode, 700ms for scale colors, 900ms for (fastest available) linear interpolation, so about 2sec total.
So, gamma_curve is less than 1/3000 of total processing time for such large files.
Either your profiler is wrong, or your math library (pow(), log(), or exp() functions) is incredibly slow.
Thank you.
Corrected on site (/documentation) and in current 0.19-branch (to be published on next update)
Yes, this will effectively map [0...max_value/exposure] to full output range.
Well, after some tests and thinking, it downed to me that I should just adjust the gamma curve, there is no point in multiplying everything with a constant multiplyer.
So my function is now like this, and it works (but not exactly how it does in RawTherapee)
void do_color_curve() { int i; double resulting_gamma; double max_color=original_exposure_max_channels*(1.0f/exposure); for(i=0;i<65535;i++) { double i_double=i; resulting_gamma=pow(i_double/(double)max_color,1.0f/gamma); if(resulting_gamma>1)resulting_gamma=1; color_curve[i]=resulting_gamma*255; } }I also want to pick a single gamma curve and apply those same remap value to hundreds or thousands of images. So having the ability to directly manipulate the libraw_colordata_t.curve[] data in lieu of defining gammas would be VERY nice. I find on 50mp canon cr2 images calculating the gamma curve remap per image is almost 15s per image on an ivy bridge xeon.
OK, looks like over-optimization in COLOR:
int COLOR(int row, int col) { return libraw_internal_data.internal_output_params.fuji_width ? FCF(row, col) : FC(row, col); } int FC(int row, int col) { return (imgdata.idata.filters >> (((row << 1 & 14) | (col & 1)) << 1) & 3); } int fcol(int row, int col);where FCF() is for 'fuji-rotated' images (S5 Pro, etc).
It should be
int COLOR(int row, int col) { return imgdata.idata.filters < 1000 ? fcol(row,col): (libraw_internal_data.internal_output_params.fuji_width ? FCF(row, col) : FC(row, col)); }To be fixed in 0.19 (0.18 is security-only-patches now)
Hi Alex,
I'm the rawpy developer and just had a look at this. I'm using LibRaw::COLOR(row, col) to construct the color pattern matrix, which works fine for non-x-trans images. But for x-trans any value after the first row returns 0. so ::COLOR(1, 0) is 0, as well as ::COLOR(1,1) is 0. I'm using libraw 0.18.2 currently and noticed some x-trans related changes in newer versions: https://github.com/LibRaw/LibRaw/compare/0.18.2...0.18.5 But they seem to be related to security only.
My test image is: http://img.photographyblog.com/reviews/fujifilm_x_t1/photos/fujifilm_x_t...
Can you check whether the COLOR method works correctly here? I think there may be a problem.
Thanks!
Maik
That was it, thank you very much!
I changed my code to:
libraw_raw2image(raw_struct);
libraw_subtract_black(raw_struct);
And it works very nicely :) (well, still have to do other things now, but the results are much better).
Yes, default char type (signed/unsigned) is undefined (although it is signed in all our test/production environments).
We'll change char definition to signed char (or, maybe, to int8_t, will see)
Thank you for reporting the bug.
Amazing, yes you are completely right, that's the reason it was always failing. In gcc, I added
LOCAL_CFLAGS += -fsigned-char
to the make file and it works perfectly and decodes like a charm! As I read, the default of a char is not really defined in c and depends on the architecture, so this error makes perfectly sens. But may there would be a way to explicitly init is as a signed char directly in code so that these things will not happen to others?
Thanks again!
it looks like I know the answer:
q_table is defined as:
char *q_table; /* quantization table */
So, signed char, from -127 to 128.
It is initialized with ..-4...+4 values at lines 110-130.
Is there any chance that your char type is by default unsigned char?
Hi Alex,
so after quite a lot of coffee, I found some kind of translation method to translate the stacktrace to real code lines. Here's the result:
For me it seems like the access of the grads array via the gradient variable runs further than the array, but I'm not sure. What do you think? Does any output of the variables may help?
Just a note: The value of the gradient variable before it fails is 2272 which seems way higher than all others (they are around 250). Maybe there's something going wrong here?
Thanks
Torsten
Hi,
here are some dumps:
// Edit: The really strange thing is that the Ra... Rg values change each time I run it and the crash finally ocurse. For instance, now I get
before it will crash. Seems there is something really going wrong with the values?
For some other pretty strange reason it almost seems like something is running parallel, because I can't see the full var dump of the q_table in onces. But this could also be a transfering issue of the log in android since the buffer speed is somewhat limited? I don't have openmp enabled.
Pages