Questions about color matrices in colordata

Submitted by dsi on

Hi,
I am trying to understand the behaviour/purpose of the various color conversion matrices that are present in the colordata struct. I was able to glean some information from the various forum posts dating back several years but I still have a bunch of clarifications. I am little new to raw processing and so I want to ensure that any erroneous results I am getting is not due to my misunderstanding of how the library returns these coefficients.

cam_xyz[4][3]
* This appears to be convert from XYZ colorspace to the camera colorspace (R G1 B G2). Does this do this conversion for the specific camera or for a reference camera? I noticed that DNG does AnalogBalance*CameraCalibrationN*CameraMatrixN to get this matrix. Is cam_xyz equivalent to this for DNG and for other formats?
* Do cam_xyz[1..4][] always correspond to the the channel order R G1 B G2?
* When you say cam_xyz has last all zeros on the last row for an RGB image, I am unclear on what that means. Is RGB image a Foveon image or does it mean for any Bayer CFA?

cam_mul[4]
* Are they the R G1 B G2 values that correspond to white as set by the camera?
* Is the order always R G1 B G2 irrespective of the layout?
* I noticed that WB_coeffs has coefficients for standard illuminants for some formats. So will XYZ = inv(cam_xyz)*WB_coeffs[21][...] corresponds to the XYZ for d65 white point?

pre_mul[4]
* Does this equal WB_coeffs[21][...] if set?

cmatrix[3][4]. Not clear what this really does? From the dimensions, it appears to operate on the linear color space of the camera. But not clear what it transforms it to.

rgb_cam[3][4]. Am I correct in assuming this converts the linear camera values R G1 B G2 into RGB in the sRGB colorspace?

ccm[3][4]. Not clear what this really is doing?

In general, for non-demosaiced image, each pixel only has one color component? So how exactly can I use a matrix such as rgb_cam?

Thanks for all your help in advance. libraw is really powerful but kinda hard to understand (atleast for me personally!).

Dinesh

Forums: 

Thanks

Submitted by dsi on

I will take a look at that link. I will get back to you if I have any questions.

Dinesh

Some follow ups

Submitted by dsi on

Alex,
I am not clear if my earlier follow up was actually posted. So apologies for a repeated post.

The link you had sent me was really helpful. I think I now understand how the cam_xyz, rgb_cam, cam_mul and pre_mul matrices work.

However, the code referenced in the link appears to be older and hence does not contain any reference to ccm. Also, the link does not contain any information/comments about cmatrix.

I just noticed from the latest code that appears to populate cmatrix primarily through calls to cam_xyz_coeff.

Would appreciate any help about these two matrices (ccm and cmatrix)?

Dinesh

cmatrix is camera color data

Submitted by lexa on

cmatrix is camera color data similar to rgb_cam, but calculated from in-RAW color data.

ccm is also 'camera color matrix' retireved from RAW in 'as is' (excl. normalizing) form.

-- Alex Tutubalin @LibRaw LLC

Thanks!

Submitted by dsi on

Alex,
So it looks like cmatrix and ccm are also for converting from camera space to sRGB. Correct?

The ccm appears to be only read from the file and not computed and it appears to be normalized by sum(imgdata.colors.ccm).

cmatrix also appears to be either read from the file or derived from camera properties.

So, if these are not available in the file, I can assume this to be all zeros?

Is it safe to assume that cam_xyz and rgb_cam are sufficient for most color space transformations?
* rgb_cam for direct conversion to sRGB?
* pseudoinverse(cam_xyz) to move to XYZ space and then apply any suitable transformation to move to a colorspace such as Rec.2020 or ProPhoto of my choice?

Also, a quick question about wb_coeffs. Are the coeffs in the order of the CFA pattern OR are they always R G1 B G2?

Regards,
Dinesh

wb_coeffs are in channel

Submitted by lexa on

wb_coeffs are in channel numbering order (as returned by COLOR(row,col))

-- Alex Tutubalin @LibRaw LLC

ccm vs cmatrix

Submitted by dsi on

Alex,
A quick follow up. So based on your descriptions, cmatrix is the transformation from Camera Space to sRGB.

Is ccm similar to cam_xyz i.e. transformation between Camera Space and XYZ?

I have a DNG file shot from a Google Pixel 3 that has all-zeros for cam_xyz and ccm. The rgb_cam and cmatrix have the same values. The profile is also not available.

For such files, how do I convert from input space/camera space to XYZ?

ccm is not similar to cmatrix

Submitted by lexa on

ccm is not similar to cmatrix.

To convert to XYZ use cmatrix or rgb_cam, multiplied by srgb2xyz conversion matrix, see convert_to_rgb() source for details.

-- Alex Tutubalin @LibRaw LLC

Thanks!

Submitted by dsi on

That makes perfect sense. So, the matrix to convert from camera space to XYZ becomes xyz_rgb*rgb_cam. Does that impact the gamut of colors that can be represented as Camera Space -> sRGB is a narrowing conversion?

But, what exactly is ccm? I know in your earlier comment you mentioned

"ccm is also 'camera color matrix' retireved from RAW in 'as is' (excl. normalizing) form."

but could you provide more details?

Thanks,
Dinesh

Matrix conversion is not

Submitted by lexa on

Matrix conversion is not gamut limited (unless you cut negative values in some intermediate steps)

ccm is 'camera matrix parsed from metadata. period', it is preserved as is (w/ possible value normalization). Usually it 'just some color data with not known camera/vendor specific meaning'.

-- Alex Tutubalin @LibRaw LLC

Normalizing cam_rgb

Submitted by dsi on

Alex,
I noticed that during the computation of rgb_cam from cam_xyz in the cam_xyz_coeff function, there is a normalization that is applied when cam_rgb is computed and the pre_mul is updated. The annotated DCRAW site also does not provide much explanation about this.

The author's comment is "But I don’t fully understand what this tiny little section of code really does. I’d love to hear from someone who can explain it to me."

Given this I have some questions:
1. For camera's (like Google Pixel 3 above), where cam_xyz has to be derived from cmatrix (or rgb_cam) and xyz_rgb, how can I undo the normalization when calculating cam_xyz?

2. During cam_rgb normalization, the pre_mul is updated. So, I assume that the coeffs in pre_mul need to be applied to the RAW data before demosaicing and applying rgb_cam to convert from Camera Space -> sRGB?

3. When should I use cam_mul then?

Dinesh

As far as I understood from

Submitted by lexa on

As far as I understood from the previous discussion, you're creating your own postprocessing for RAW.

I do not thing you need to replicate LibRaw/dcraw color processing pipeline.

-- Alex Tutubalin @LibRaw LLC

Apologies

Submitted by dsi on

Alex,
Please forgive me if I gave the impression that I am trying to replicate DCRAW/libraw.

I am creating my own RAW processing pipeline but I am relying on libraw to read the CFA and metadata for the color transformation to implement this pipeline.

Due to the lack of documentation, I am just trying to understand how these coefficients work or are meant to be used to make sure I am implementing my pipeline correctly.

For example, for the Google Pixel 3 camera, I had to derive the cam_xyz from rgb_cam. However, the resulting matrix resulted in a strong color cast probably because I did not undo the normalization after doing INV(rgb_cam).

Also, I assumed that cam_mul would be sufficient because they are the as shot WB multipliers but pre_mul appears to be needed because of the cam_rgb normalization.

I am just looking for more information on these.

Thanks for all your help thus far!

Dinesh

pre_mul is not used for

Submitted by lexa on

pre_mul is not used for normalization. It derived from cam_rgb data (to get daylight wb coeffs)

-- Alex Tutubalin @LibRaw LLC

You can start here: what does

Submitted by ib on

You can start here: what does cam_xyz represent, what kind of colour transform, what is it applied to, and what's the result? Before you know the answer the life will be not easy. Put it another way, if you are dealing with colour, it's only logical to start with a bit of study of colour engineering, and not to rely on guessing from the code or trial and error.

--
Iliah Borg

Thanks

Submitted by dsi on

Hi Iliah,
Thanks for the pointers. I do have an understanding of color science. Its just that with dcraw having multiple properties that describe very similar things, I was just trying to ensure I was using the right ones.

Regards,
Dinesh