Equivalent of dcraw -d option?

Submitted by David Partridge on

Should I just set output_params.no_interpolation = 1; to get the equivalent od trhe dcraw -d option?

Thanks

Forums: 

I do not know what exactly

Submitted by lexa on

I do not know what exactly you want to achieve (-d option behavior in dcraw has changed several times), but yes, no_interpolation = 1 will turn off interpolation and fbdd demoising step(s).

-- Alex Tutubalin @LibRaw LLC

It would appear there's a

Submitted by David Partridge on

It would appear there's a possible bug with libraw in this area.

If I run dcraw -d M103_L_0048_ISO800_60s__16C.CR2, it creates a PGM output file:

29/05/2019 12:01 18,024,947 M103_L_0048_ISO800_60s__16C.pgm

if I run dcraw_emu -disinterp M103_L_0048_ISO800_60s__16C.CR2, it creates a PPM output file:

29/05/2019 12:02 54,074,807 M103_L_0048_ISO800_60s__16C.CR2.ppm

I see two problems:

1) This is the trivial issue that the output fileid is slightly different (which I'm happy to forgive).

2) The major problem is that the output file is NOT a single colour PGM greyscale bitmap as was produced by dcraw, but a 3 colour PPM bitmap.

Cheers

David Partridge

As mentioned above, there is

Submitted by lexa on

As mentioned above, there is no direct equivalent for dcraw's -d option in LibRaw (esp. dcraw_emu sample).
If you need to get RAW data as is, you, most likely, need unprocessed_raw sample.

-- Alex Tutubalin @LibRaw LLC

OK I'm a bit disappointed

Submitted by David Partridge on

OK I'm a bit disappointed about that. Looking at the source in dcraw_common.cpp it appears that the "document mode" stuff has been #ifdef-ed out of existence.

I had specified equivalent options to those I used with dcraw and captured the output file by sub-classing libraw and over-riding just dcraw_ppm_tiff_writer() and write_ppm_tiff() which worked a treat except that it didn't produce the 16 bit PGM greyscale data I hoped for but instead produced a 16 bit PPM RGB bitmap (which I confirmed using dcraw_emu).

Advice and guidance in more detail of how to get at the grayscale data will be greatly welcome

Might I ask why you explicitly disabled the "document mode" code?

Here's what my code looks like right now:

			if (!m_bColorRAW)
			{
				// Document mode (i.e. don't demosaic the image in libraw).
				O.no_interpolation = 1;
			};
 
			// Don't stretch or rotate raw pixels (equivalent to dcraw -j)
			O.use_fuji_rotate = 0;
 
			// Don't flip the image (equivalent to dcraw -t 0)
			O.user_flip = 0;
 
			// Output color space : raw-> sRGB (default)
			/*
			argv[argc] = _T("-o");
			argc++;
			argv[argc] = _T("0");
			argc++;*/
 
			workspace.GetValue(REGENTRY_BASEKEY_RAWSETTINGS, _T("BlackPointTo0"), bBlackPointTo0);
			if (bBlackPointTo0)
			{
				// Set black point to 0 
				O.user_black = 0;
			};
 
			// Output is 16 bits (equivalent of dcraw flag -4)
			O.gamm[0] = O.gamm[1] = O.no_auto_bright = 1;
			O.output_bps = 16;
 
			lstrcpy(g_szInputFileName, (LPCTSTR)m_strFileName);
			g_Progress = pProgress;
 
			if ((ret = rawProcessor.unpack()) != LIBRAW_SUCCESS)
			{
				bResult = FALSE;
				ZTRACE_RUNTIME("Cannot unpack %s: %s", m_strFileName, libraw_strerror(ret));
			}
			if (!bResult) break;
 
			if (LIBRAW_SUCCESS != (ret = rawProcessor.dcraw_process()))
			{
				ZTRACE_RUNTIME("Cannot do postprocessing on %s: %s", m_strFileName, libraw_strerror(ret));
				if (LIBRAW_FATAL_ERROR(ret))
					bResult = FALSE;
			}
			if (!bResult) break;
			pFiller = new BitMapFiller(pBitmap, pProgress);
			pFiller->SetWhiteBalance(fRedScale, fGreenScale, fBlueScale);
			// Get the Colour Filter Array type and set into the bitmap filler
			m_CFAType = GetCurrentCFAType();
			pFiller->SetCFAType(m_CFAType);
 
			// Set up the intercept code to write the image data to our bitmap instead of 
			// to an external file, and invoke the overridden dcraw_ppm_tiff_writer()
			rawProcessor.setBitMapFiller(pFiller);
			if (LIBRAW_SUCCESS != (ret = rawProcessor.dcraw_ppm_tiff_writer("")))
			{
				bResult = FALSE;
				ZTRACE_RUNTIME("Cannot write image data to bitmap %s", libraw_strerror(ret));
			}

Thanks

David Partridge

Dear Sir:

Submitted by lexa on

Dear Sir:

Could you please specify what result you want to achiveve?
Is it unaltered pixel values, or something else?

-- Alex Tutubalin @LibRaw LLC

Sure - what I want is the 16

Submitted by David Partridge on

Sure - what I want is the 16 bit greyscale "raw" data so I can do my own de-bayer processing on it (except for the case of "full colour" raw files where I want the RGB image data). That's what the code that reads:

			if (!m_bColorRAW)
			{
				// Document mode (i.e. don't demosaic the image in libraw).
				O.no_interpolation = 1;
			};

was intended to achieve: m_bColorRaw is set thus: 

		m_bColorRAW	= P1.is_foveon || !(P1.filters);

Would that be accomplished by grabbing the "raw image data" from libraw_raw_data after running unpack()?

If so I think (hope) I'll try using that based on the "unprocessed_raw.cpp sample.

Thanks again

David Partridge

Unaltered raw pixels are

Submitted by lexa on

Unaltered raw pixels are stored in imgdata.rawdata:
raw_image: for bayer/x-trans/monochrome (single component per pixel)
color3_image, color4_image - for 3/4 full-color images.

-- Alex Tutubalin @LibRaw LLC

Followup: only one of

Submitted by lexa on

Followup: only one of pointers mentioned above is non-NULL after unpack()

-- Alex Tutubalin @LibRaw LLC

Got it - is the data in the

Submitted by David Partridge on

Got it - is the data in the ushort arrays in "network byte order" or do I need to do a byte-swap?

Thank you.

AFAICT, the image at (e.g.) raw_image is raw_width*raw_height.

How to actually skip the image frame/border - do I need to offset by top_margin/left margin? Do you have a neat bit of code to iterate through the rows and skip the margins (I don't want to reinvent the wheel) so I only extract the rectangle that is width*height and (I assume) centred in the frame?

Thanks again

David Partridge

Data is in native byte order.

Submitted by lexa on

Data is in native byte order.

Image rectangle may not be centered in frame, this is camera-specific, so one need to use left/top_margin. You may use LibRaw::copy_bayer() and it's calling code (raw2image) as an example (neat bit of code).

Note: it is safer to use imgdata.sizes.raw_pitch (in bytes!) to access rows. Usually it is just multiplication of raw_width and pixel size, but there are some exceptions (e.g. if LibRaw is compiled with RawSpeed library).

-- Alex Tutubalin @LibRaw LLC

Thanks yet again,

Submitted by David Partridge on

Thanks yet again,

Here's a shortened version of the code I'm proposing to use to process the data at raw_image

#define RAW(row,col) \
	Rawdata.raw_image[(row)*S.raw_width+(col)]
 
	BOOL littleEndian = htons(0x55aa) != 0x55aa;
	if (!m_bColorRAW)
	{
		// This is a regular RAW file, so we should have the "raw" 16-bit greyscale
		// pixel array hung off RawData.raw_image.
		ZASSERT(NULL != RawData.raw_image);
 
		// stuff omitted
		// Convert raw data to big-endian 
		if littleEndian
			_swab(
				(char*)(RawData.raw_image),
				(char*)(RawData.raw_image),
				S.raw_height*S.raw_pitch);		// Use number of rows times row width in BYTES!!
 
		for (int row = 0; row < S.height; row++)
		{
			// Write raw pixel data into our private bitmap format
			pFiller->Write(RAW(raw, S.left_margin), sizeof(ushort), S.width);
		}

Does that make sense? Is it correct?

Thanks

David Partridge

I still do not know what you

Submitted by lexa on

I still do not know what you want to achieve, so this may be correct, may be not.

Is this code does what you want or not?

-- Alex Tutubalin @LibRaw LLC

I want to extract the raw

Submitted by David Partridge on

I want to extract the raw image pixel data from the file loaded - I'm asking if that code I posted with a bug will extract the 16 bit raw image data without the frame that is width*height. Right now I'm not seeing what I expect when I look at RawData.raw_image as what DCRAW -t 0 -d -4 wrote to the PGM file including the header looks like (I've shown the binary data as hex):

P5
5202 3465
65535
04 fA 02 80 04 40

Correction to the bug

for (int row = 0; row < S.height; row++)
{
// Write raw pixel data into our private bitmap format
pFiller->Write(RAW(row+S.top_margin, S.left_margin), sizeof(ushort), S.width);
}

and I cannot find that data in the raw_image array at or near raw_image(row+S.top_margin, S.left_margin)

David Partridge

Raw data pixels are in raw

Submitted by lexa on

Raw data pixels are in raw_image array.
These pixels are unaltered (in particular, black level/bias is not subtracted), unscaled, uncropped and no other processing is made.

So, if you need *unaltered* pixels: here it is. If you need some type of processing (black subtraction, white balance, scale, crop) you need to do it in your code.

-- Alex Tutubalin @LibRaw LLC

AFAIK that's exactly what

Submitted by David Partridge on

AFAIK that's exactly what dcraw -d -4 does (i.e. no processing at all apart from unpacking the raw image, extracting the image from the frame, and writing the PGM file). Have I misunderstood this?

So I'm having a bit of difficulty understanding why I'm seeing a difference?

Cheers

David Partridge

AFAIK, -D is completely no

Submitted by lexa on

AFAIK, -D is completely no processing, while -d looks like black subtraction is performed.
Also, both -D/-d creates gamma corrected image (if no other flags)

-- Alex Tutubalin @LibRaw LLC

AFAICT using the -4 flag

Submitted by David Partridge on

AFAICT using the -4 flag means gamma coefficients both set to 1. So I *think* that means that gamma correction should make no changes. Is that correct?

Certainly -d and -D give quite different results - d is quite a bit brighter than the -D version - would that fit with black subtraction processing? Where in the code would I find the black subtraction (so I can do that processing myself on the RAW data?

Is it out of order to ask why the document mode code was removed?

Thanks again

David Partridge

Document mode is not

Submitted by lexa on

Document mode is not supported in our dcraw processing emulation (dcraw_process() call) because it makes things too complex.
Our unprocessed_raw.cpp and 4channels.cpp samples provides unprocessed raw data output (for bayer case), that is enough for most practical cases. It is also good starting point(s) for ones who want only unprocessed raw data access and do full processing.

Black subtraction: if LibRaw docs covering imgdata.color.black/imgdata.color.cblack[] is not enough, you may use LibRaw::subtract_black() source as an example. It operates with imgdata.image[] array, assuming that rawdata.raw_image[] values are already in place.

Also, LibRaw::raw2image_ex() do raw_image=>image population and black subtraction in single step.

-- Alex Tutubalin @LibRaw LLC