Inverting raw_image and getting magenta output

Submitted by amaschas on

Hi all, I'm rather new to using libraw, so I apologize if this question is somewhat naive. I'm trying to build a set of tools for handling DLSR captures of film negatives, and I'm attempting some tests to verify my guess that performing several steps in the negative inversion process using the RAW sensor data rather than interpolated data that has been converted to a color space will provide some advantages. I'm using a RAW capture of a color film negative from an A7r IV that I've converted to a DNG for this test.

My first step is a simple attempt to invert the color data in a RAW capture of a film negative. I did this by determining the black level and maximum value of the sensor data in the raw_image, and then iterating over the the data in raw_image and setting ImageProcessor.imgdata.rawdata.raw_image[i] = ImageProcessor.imgdata.color.maximum - ImageProcessor.imgdata.rawdata.raw_image[i];. I figured I could just take every value in the array and subtract it from the maximum value to get the inverted image. I then ran dcraw_process() and dcraw_ppm_tiff_writer() to generate a TIFF.

The approach does in fact generate an image, but the image in question is almost entirely magenta. I feel like I'm missing something very fundamental about what's going on here, and I'm hoping someone can provide me with some guidance. Thanks!

Forums: 

1st:

Submitted by lexa on

1st:
raw_image[] values are 'as decoded from RAW file', so black level is not subtracted.

If you're making 'in house' application, so the only camera you plan to use is A7R-4, the exact black level for this camera is 512.
so inverted_value = maximum - original_value + 512 (maximum is also has black level within it).

If you plan to use multiple cameras, look into src/preprocessing/subtract_black.cpp source for black subtraction code and into src/utils/utils_libraw.cpp:adjust_bl() function for additional details.

2nd: If you're processing color negatives, inversion in linear domain will not result in acceptable image color because of color masking used in that negatives ( http://www.brianpritchard.com/why_colour_negative_is_orange.htm )
Also, for both color and BW negatives you'll need to adjust contrast.

-- Alex Tutubalin @LibRaw LLC

Hey Alex, thanks for the

Submitted by amaschas on

Hey Alex, thanks for the reply! I'm aware that I will have to adjust for the mask and for the low contrast in the negative. At this point I just want to get the inversion step right so that I can be a little more certain that my overall approach will work. I updated my code as you suggested, but I'm still getting an entirely magenta image. Here's my loop:

for(int i = 0;i < ImageProcessor.imgdata.sizes.iwidth *  ImageProcessor.imgdata.sizes.iheight; i++) {
    int inverted_value = ImageProcessor.imgdata.color.maximum - ImageProcessor.imgdata.rawdata.raw_image[i] + 512;
    ImageProcessor.imgdata.rawdata.raw_image[i] = inverted_value;
}

This is the original images: http://ur.sine.com/temp/original.png
And this is the output of that code: http://ur.sine.com/temp/output.png

Could you please provide link

Submitted by lexa on

Could you please provide link to the RAW file too, it is very interesting to play w/ it.

-- Alex Tutubalin @LibRaw LLC

Black point is same for all

Submitted by lexa on

Black point is same for all channels in Sony files (and, yes, it is 512 in the file you shared).

The problem is white balance. Here is your image in 'raw composite' view: https://www.dropbox.com/s/iowl9jkhgzhxqy0/screenshot%202020-04-22%2019.3... (not white balanced, green channel(s) is strongest as expected)

So, I see two possible ways:
1) If you'll go 'raw data inversion' way: Invert white balance coefficients too.

2) Generate proper negative image in linear space (so, no raw data inversion, normal processing with linear gamma output), than invert it.

-- Alex Tutubalin @LibRaw LLC

Getting closer...

Submitted by amaschas on

Thank you, I think that got me closer! I'm still a little puzzled about how the white balance coefficients work (they appear to be relative, rather than absolute?) but I'll see if I can figure it out. Thanks again!

Getting Confusing Output from White Balance

Submitted by amaschas on

So I've done a bit of work trying to figure out how to invert the white balance of my image, but I'm seeing some counterintuitive behavior from the white balance process, and I'm hoping you can provide me some insight into what's going on. I compiled libRaw with DCRAW_VERBOSE enabled to get some transparency into what white balance coefficients were actually getting used, and this is what I'm observing.

If I make no change to any white balance settings, I get: multipliers 2.858569 1.000000 1.338857 1.000000

If I run this loop, which inverts the white balance value for each channel:

for(int i = 0;i < 3; i++) {
    if (ImageProcessor.imgdata.color.cam_mul[i] != 0) {
        ImageProcessor.imgdata.params.user_mul[i] = 1 / ImageProcessor.imgdata.color.cam_mul[i];
    }
}

I can see that my user_mul values are set correctly, but the draw_process uses different values for some reason.

user_mul: 0.418985 1.000000 0.599532 0.000000
multipliers 1.000000 2.386720 1.430915 2.386720

I even tried manually setting the user_mul values, but got the same result. I'm sort of at a loss for where the multipliers that get used are coming from.

1/0.418985 == 2.38672/1.0 (G

Submitted by lexa on

1/0.418985 == 2.38672/1.0 (G/R coeffs ratio). Or, in other words, dcraw_process scales WB coeffs to get full data range after WB applied (and scaled at some time).

BTW, I still not sure that RAW value inversion leads in the right direction. Why not invert (w/ mask subtraction) image after raw processing applied?

-- Alex Tutubalin @LibRaw LLC

Building RAW Negative Inversion Tools

Submitted by amaschas on

Ah, ok, that makes sense, thanks for the explanation. I guess I'll have to do something more sophisticated than just inverting the wb coefficients, because the output channels are getting cut off on the high end. Thanks for explaining that!

I want to try to invert the RAW data, rather than the processed data, for a few reasons. The first is that I eventually want to make a tool that outputs a RAW file (using the DNG SDK). While it is possible to invert a negative in most RAW editors by inverting the tone curve, it makes the resulting image difficult to work with, and being able to work with the inverted image in RAW would make things a lot easier.

The second reason is a bit more abstract, and is somewhat of an experiment. The way that the different color emulsion layers work in a negative—specifically the orange mask that negative the color leakage in the magenta and cyan channels—means that for any operation on those channel to work correct, you need to operate in the negative's native color space. The matrix multiplication that converts the RAW data into a color space alters the curves in the individual color channels in a way that makes inverting and cancelling the effect of the orange mask difficult. I'm guessing that the demosaic process might screw with the color channels from the negative as well.

Finally, the ultimate goal of all of this is building a tool that will take 3 RAW exposures—one of each color channel—created by taking a photo of the negative with red, green and blue light, and compositing them together. The tool will then invert the image, white balance and output to RAW.

A lot of this process is derived from the way film scanners work. Most high end film scanners user RGB light sources and black and white sensors to read the three color channels and account for the orange mask by adjusting the analog gain per channel.

1) imgdata.params.output

Submitted by lexa on

1) imgdata.params.output_color=0 will output resulting demosaiced image in 'camera' color space.

3) highlight clipping may occur because
a) red or blue channel is clipped due to WB
b) or auto-brightness applied in ppm_tiff_writer or make_mem_image

-- Alex Tutubalin @LibRaw LLC

I tried auto white balancing

Submitted by amaschas on

I tried auto white balancing and it went a long way toward fixing the issues. I did have to manually reduce the brightness though. Is there a way to control the brightness adjustment in ppm_tiff_writer?

I'm also noticing that the red and blue channels appear very compressed in the output image relative to the green channel. I'm not sure what would cause that.

[edit] I set no_auto_bright = 1 and no_auto_scale = 1 and that seems to have helped. Not sure why one of those would have compressed one channel and not the others. I think I'll probably need to do some custom analysis of the negative to set the white balance and scale of the resulting image. Something about inverting the values seems to throw off the automatic processes.

no_auto_scale=1 is very

Submitted by lexa on

no_auto_scale=1 is very special use parameter. If set, it will instruct dcraw_process() to skip scale_colors() step (this step performs WB and data scaling). This is not intended to use with standard LibRaw's interpolation methods, this parameter is targeted if one implements own interpolation (demosaic) that wants to deal with unchanged (not scaled) source data.

It looks like we need to document no_auto_scale better in LibRaw docs :)

To manual control output brightness, use no_auto_bright=1 and change imgdata.params.bright from 1 to something.

-- Alex Tutubalin @LibRaw LLC

Strange Scaling of Colors

Submitted by amaschas on

So I've made some progress by setting:

use_auto_wb = 1
no_auto_bright = 1
bright = 0.8

But I think the auto scaling is having some issues scaling the channels correctly. I think it's probably because the upper and lower bounds of each channel were calculated with the un-inverted negative?

This is the original histogram from RawDigger: http://ur.sine.com/temp/tether8387-Full-8000x5320.png

...and this is the histogram from the generated TIFF from Rawtherapee: http://ur.sine.com/temp/post-process-channels.png

Is there a good way to have libRaw recalculate the per-channel min and max, or alternatively, to set them manually?

Is this histogram is from

Submitted by lexa on

Is this histogram is from inverted data?

If so, I could not answer the question, I simply do not know what happens w/ processing if channels are inverted.

Here are histograms of your tether8387 file processed with
1) dcraw_emu -T -W -a -b 1 (Tiff, no_auto_bright,auto-wb, bright=1): https://www.dropbox.com/s/6l67ry7y03xlj7p/screenshot%202020-05-04%2009.0...

2) same, but -b 0.8: https://www.dropbox.com/s/7o1w581gamxy4nz/screenshot%202020-05-04%2009.0...

I do not see anything not expected here.

I

-- Alex Tutubalin @LibRaw LLC

Yeah, the weird histogram is

Submitted by amaschas on

Yeah, the weird histogram is from the inverted data. I think I'm going to have to deeply familiarize myself with what scale_colors does, because I think something in that process is relying on values that were generated prior to inverting the raw data.