1st: We do not have donation link (yet). One could buy our software (RawDigger or FastRawVewer or bundle)
2nd: does dcraw_emu -r 2 1 3 1 results in different WB? If so, please start with samples/half_mt.c sample (it also uses C-API) with this additional lines below line ~80:
Ok, I think the issue was the gamma correction. Setting both gamm parameters to 1.0 solved it. Now it seems like I am getting color separated and interpolated raw values!
Interpolation I would like to have, for sure, and color conversion to a single G value is also appreciated. Data scaling only if the ratios between the colors are kept. White balance is probably something I would play with to see what works better, although I probably want that as it has been calibrated by the camera company. Gamma is a no however. So, I would want processed data.
BTW, I also noticed that the final image was mirrored vertically; I had to place a flip switch in there.
LibRaw::dcraw_process() (plus dcraw_make_mem_image()) do full processing steps that include
- white balance
- interpolation
- data scaling to full range (assuming maximum value scaled to 65535 if auto-bright is turned off)
- color conversion
- gamma conversion (on make_mem_image step)
What data you want to use, processed one or original raw values? If you need raw values, you do not need processing step, just use imgdata.rawdata...image pointers to access unprocessed/unaltered raw values after LibRaw::unpack()
Thanks again for the quick response. I am not using men_image_sample, so that's ok. I am converting the data array into a fits image for astronomical image processing. And I saw the malloc also.
I also managed to get it to work now as, example
rgb[0] = (ushort)img->data[i]
This is why you should not code when exhausted!
Additional Qs:
1)The peak intensities are near ~36000 in all three color channels, while I know that the detector in the Canon Rp is only 14 bit. So something is scaling the images. I don't believe I have any options enabled that would scale the image (wb=1, no auto brightening, etc). What could it be? When converting to DNG with Lightroom peak intensities are saturated at 16384 correctly.
2) I tried all interpolation methods and I am still getting some pixels with values of zero. Why is that?
Yes, data member is [1] in size just to make it an array.
In real code output is allocated via malloc(sizeof(libraw_processed_image_t)+widh*height*bytes-per-pixel, so it is possible to access data[] beyond data[0]).
mem_image sample supports 16-bit output for ppm files, but please note that ppm format is recorded in 'network byte order', so byte swap is used (not needed for internal access to data casted to ushort)
Thanks for the response, much appreciated. I read this above and tried the explicit recast, but it did not work. I noticed that it is a single element point array to its first element. I am certain that I am being the idiot here. Could you give a concrete syntax example? It would be nice if this were documented in the libraw description; most people that want to work with raw data do so because they want the full 16 bit data range. Thanks!
libraw_processed_image_t.data (char [1] array) is just a pointer to actual data (to avoid pointer here plus additional allocation). One need to recast it to uint16_t to use w/ 16-bit output.
Well, I hacked the code and rewrote the structure to define the "data" variable as ushort (libraw/libraw_types.h lines #177: "unsigned char" changed to "unsigned short"). The only other mod I had to do was to rewrite one of the sample programs (men_image_sample.cpp line #69) to recast row_pointer[0] to be = (uchar *) &img->data ... The entire package compiles perfectly fine and works as intended now.
I may be misunderstanding something here. An unsigned character, by def is 1 byte and can only take values between 0 and 255. Even if you recast the output to be ushort, the procedure placed only an unsigned character into the processed_image_t->data array, as its type is cast in the header to be only an unsigned char. How do we actually get 16 bit outputs from this function?
I post an example code snippet here. It is hard to find an example online for me:
void exif_callback(void *context, int tag, int type, int len, unsigned int ord, void *ifp) {
LibRaw_abstract_datastream* data = static_cast(ifp);
MyContext *mycontext = static_cast(context);
// read noiseProfile tag as an example
tag &= 0x0fffff; // Undo (ifdN + 1) << 20)
if (tag == 0xc761) {
double values[2];
data->read(values, sizeof(double), 2);
std::cout << values[0] << " " << values[1] << std::endl;
// store values into MyContext if you need
}
}
static LibRaw lr;
lr.open_file(filename);
MyContext context; // a custom struct just to save data
lr.set_exifparser_handler(exif_callback, &context);
lr.unpack();
FWIW I (and a number of others) don't believe it is the "correct" solution, however I do understand that from the perspective of the Libraw developers it is the most expedient solution for general images that don't require calibration.
The only reasonable option you have is to adopt the new behavior because it's the correct one, and most probably it's here to stay. Nobody guarantees you the next firmware version will not change the margins, for example because the fab was changed, and thus the order channels in the CFA will change. Your applications shouldn't be designed to relate on mutables to begin with. Now code a converter from old flats to a new format, satisfy your thousands of users, and be done.
I don't think we have samples from Flextight X5, so sending us a link to a scan of a shot with known colours, like ColorChecker or a scanner profiling target, would be a good start ;)
Hi David, this is Jasem from INDI. We have the same issues as well, and this could potentially affect thousands of users. I think at this point, there are two ways:
1. Fork libraw and disable these changes instead of subjecting end-users to deal with this?
2. Add a patch to disable them in the package management scripts?
One of the labs I use has a Hasselblad (formerly Imacon) Flextight X5 Scanner. It produces RAW files that need to be decoded.
My current workflow is to decode them on Mac, but I would like to bring them directly into libRaw/Digikam. Is there anything I can do to help get the format supported by libRaw?
1st: We do not have donation link (yet). One could buy our software (RawDigger or FastRawVewer or bundle)
2nd: does dcraw_emu -r 2 1 3 1 results in different WB? If so, please start with samples/half_mt.c sample (it also uses C-API) with this additional lines below line ~80:
iprc->params.user_mul[0] = 2;
iprc->params.user_mul[2] = 3;
iprc->params.user_mul[1] =
iprc->params.user_mul[3] = 1;
This should result into bluish output.
(Note: you do not need libraw_raw2image if libraw_dcraw_process is used)
3rd: to use 'raw colors' set params.output_color to 0, this will disable camera color matrix use.
Just a correction. The code snippet I put there was incorrect.
Recasting should be simply
ushort *image = (ushort *) processed->data;
-A
Yes, (good) color profiles created for specific light source works better than generic (D65 or D50) ones.
These files are floating-point DNG files compressed with deflate compression.
To decode such files by LibRaw (without DNG SDK) you need to build LibRaw with USE_ZLIB defined and libz linked.
Here is tiff file produced from your sample by dcraw_emu -T (built w/ options mentioned above): https://www.dropbox.com/s/b2tpar7n395ee2j/_1010194-hdr.dng.tiff?dl=0
Hi Alex,
Ok, I think the issue was the gamma correction. Setting both gamm parameters to 1.0 solved it. Now it seems like I am getting color separated and interpolated raw values!
Thanks
Hi Alex!
You are fantastic for responding so quickly!
Interpolation I would like to have, for sure, and color conversion to a single G value is also appreciated. Data scaling only if the ratios between the colors are kept. White balance is probably something I would play with to see what works better, although I probably want that as it has been calibrated by the camera company. Gamma is a no however. So, I would want processed data.
BTW, I also noticed that the final image was mirrored vertically; I had to place a flip switch in there.
Thanks again!
LibRaw::dcraw_process() (plus dcraw_make_mem_image()) do full processing steps that include
- white balance
- interpolation
- data scaling to full range (assuming maximum value scaled to 65535 if auto-bright is turned off)
- color conversion
- gamma conversion (on make_mem_image step)
What data you want to use, processed one or original raw values? If you need raw values, you do not need processing step, just use imgdata.rawdata...image pointers to access unprocessed/unaltered raw values after LibRaw::unpack()
Hi Alex,
Thanks again for the quick response. I am not using men_image_sample, so that's ok. I am converting the data array into a fits image for astronomical image processing. And I saw the malloc also.
I also managed to get it to work now as, example
rgb[0] = (ushort)img->data[i]
This is why you should not code when exhausted!
Additional Qs:
1)The peak intensities are near ~36000 in all three color channels, while I know that the detector in the Canon Rp is only 14 bit. So something is scaling the images. I don't believe I have any options enabled that would scale the image (wb=1, no auto brightening, etc). What could it be? When converting to DNG with Lightroom peak intensities are saturated at 16384 correctly.
2) I tried all interpolation methods and I am still getting some pixels with values of zero. Why is that?
Thanks!
Yes, data member is [1] in size just to make it an array.
In real code output is allocated via malloc(sizeof(libraw_processed_image_t)+widh*height*bytes-per-pixel, so it is possible to access data[] beyond data[0]).
mem_image sample supports 16-bit output for ppm files, but please note that ppm format is recorded in 'network byte order', so byte swap is used (not needed for internal access to data casted to ushort)
Hi Alex,
Thanks for the response, much appreciated. I read this above and tried the explicit recast, but it did not work. I noticed that it is a single element point array to its first element. I am certain that I am being the idiot here. Could you give a concrete syntax example? It would be nice if this were documented in the libraw description; most people that want to work with raw data do so because they want the full 16 bit data range. Thanks!
Could you please check with dcraw_emu -6 -T ?
libraw_processed_image_t.data (char [1] array) is just a pointer to actual data (to avoid pointer here plus additional allocation). One need to recast it to uint16_t to use w/ 16-bit output.
Well, I hacked the code and rewrote the structure to define the "data" variable as ushort (libraw/libraw_types.h lines #177: "unsigned char" changed to "unsigned short"). The only other mod I had to do was to rewrite one of the sample programs (men_image_sample.cpp line #69) to recast row_pointer[0] to be = (uchar *) &img->data ... The entire package compiles perfectly fine and works as intended now.
Hi,
I may be misunderstanding something here. An unsigned character, by def is 1 byte and can only take values between 0 and 255. Even if you recast the output to be ushort, the procedure placed only an unsigned character into the processed_image_t->data array, as its type is cast in the header to be only an unsigned char. How do we actually get 16 bit outputs from this function?
Hi Alex,
I'd like to say thank you, my program is now generating the correct colour based on your suggestion.
I'm moving forward to understand the tonal curves, hopefully I can reproduce the same results as "copy_mem_image".
Hi there,
Nikon Z cameras present themselves as Z 7, Z 6, Z 5, Z 50, not Z7, Z6, ...
Regards
Klaus
I post an example code snippet here. It is hard to find an example online for me:
void exif_callback(void *context, int tag, int type, int len, unsigned int ord, void *ifp) {
LibRaw_abstract_datastream* data = static_cast(ifp);
MyContext *mycontext = static_cast(context);
// read noiseProfile tag as an example
tag &= 0x0fffff; // Undo (ifdN + 1) << 20)
if (tag == 0xc761) {
double values[2];
data->read(values, sizeof(double), 2);
std::cout << values[0] << " " << values[1] << std::endl;
// store values into MyContext if you need
}
}
static LibRaw lr;
lr.open_file(filename);
MyContext context; // a custom struct just to save data
lr.set_exifparser_handler(exif_callback, &context);
lr.unpack();
I will try that once I upgrade to 0.20 release.
This patch fixes linear_max for XZ1: https://github.com/LibRaw/LibRaw/commit/87792af903fc21d5157ee2f2ba194ff1...
(by not providing this value)
Oh, don't speak for number of others. Myself, I'm happy LibRaw developers switched to correct and future-proof solution.
FWIW I (and a number of others) don't believe it is the "correct" solution, however I do understand that from the perspective of the Libraw developers it is the most expedient solution for general images that don't require calibration.
The only reasonable option you have is to adopt the new behavior because it's the correct one, and most probably it's here to stay. Nobody guarantees you the next firmware version will not change the margins, for example because the fab was changed, and thus the order channels in the CFA will change. Your applications shouldn't be designed to relate on mutables to begin with. Now code a converter from old flats to a new format, satisfy your thousands of users, and be done.
I don't think we have samples from Flextight X5, so sending us a link to a scan of a shot with known colours, like ColorChecker or a scanner profiling target, would be a good start ;)
Hi David, this is Jasem from INDI. We have the same issues as well, and this could potentially affect thousands of users. I think at this point, there are two ways:
1. Fork libraw and disable these changes instead of subjecting end-users to deal with this?
2. Add a patch to disable them in the package management scripts?
Hi All —
One of the labs I use has a Hasselblad (formerly Imacon) Flextight X5 Scanner. It produces RAW files that need to be decoded.
My current workflow is to decode them on Mac, but I would like to bring them directly into libRaw/Digikam. Is there anything I can do to help get the format supported by libRaw?
Pages