One more thing I dont quite understand.
I read tone curve from a cr2. This is linear.
I read it from a NEF and it looks quite strange. It grows exponentially from 0 to about 770 and then continues a linear grow. Dos these values also have to undergo a calculation?
I have one more question.
I read the cam_mul from a cr2.
I got 1987 - 1253 - 2675 - 1253.
From what I read about White Balance Multipliers I was expecting Values in a range of 1.0 - 3.0.
Do I have to devide cam_mul by 1000?
For Canon/Nikon cameras, camera profile is static and defined in adobe_coeff() function (in internal/dcraw_common.cpp).
For some cameras (Olympus, Samsung, Phase One), and formats (DNG) camera profile is provided in RAW file (and enabled via params.use_camera_matrix=1)
Thanks for the reply,
Is the camera color profile something static per model or is it something variable and an interpreter of the raw file needs?
Sorry, I am just providing a tool for data extraction. My knowledge about digital imaging is pretty limited..
cam_mul is camera WB multipliers. From camera.
cam_xyz, pre_mul and rgb_cam are from camera color profile, so from LibRaw.
curve is linear in Canon raws, and from camera for Nikon raws.
To get the same result you need to reproduce hidden exposure correction and tone curve from ViewNX.
I know nothing about ViewNX internals, so cannot recommend something meaningful (try LibRaw's exposure correction with highlights compression as in my StackExchange reply)
I cant figure out if I am doing something wrong, or if its even possible to get the same result as the output of the nikon viewnx converter tool.
I am happy to provide a raw file and a expected image converted with viewns, if its something really easy for someone good at libraw/dcraw to confirm or deconfirm that its possible.
I tried imgD.params.use_camera_wb = 1; and imgD.params.no_auto_bright = 1; which seems to be close. But the image is a bit to dark then. and if using no auto scale, it becomes almost black.
I gave up on just using the raw data without processing because I could not figure out how to make opencv do the demosaic.
Depending of what normalization you need, you may use
1) imgdata.params.no_auto_bright=1 to disable auto brighten
2) params.no_auto_scale=1 to disable data scaling to 65536/camera_data_maximum
3) or just use raw data as is from imgdata.rawdata.raw_image (for bayer data) without calls to LibRaw::dcraw_process()
I am looking for the simplest possible solution to a simple problem: to compute the average of a number of images stored as dng files, and save it as a dng file. I have not found any ready-made app and have just spent a frustrating afternoon failing to build Adobe's dng sdk. Can you suggest something practical?
Yes, Foveon color rendering is very approximate.
The field uneven-ess is stored in Foveon files metadata, but it is undocumented by Sigma, so no idea how to use it.
Hope, someone will contribute the right Foveon RGB processing code into LibRaw.
With default settings, LibRaw makes these adjustments on processing (dcraw_process()), very similar to dcraw processing:
0) Black level subtracted
1) White balance applied
2) Camera color profile applied
3) Range adjusted to 1% of pixels set to saturation
All these adjustments can be turned off via imgdata.params.* tuning
i can encapsulate everything needed to mimic that behavior in some mlv support code.
on open the mlv code will generate the index and spits out frame by frame if you call another routine.
This looks very complicated for me.
LibRaw API is file-oriented. One pass filename to LibRaw::open_file(), then operates with LibRaw object which hide all internals into it.
For .MLV files there are many complex entries:
- .idx file that contains fileIndex (but not file names)
- .mlv (?) files whitch contains fileIndex and fileCount.
From LibRaw API view, it's looks like we need to pass filename.idx to open_file(), but how to recover .mlv filenames from .idx? Is LibRaw need to read headers from all .mlv files from folder or what? What should it do if some files are missing? What should it do if several files with same fileIndex exists?
LibRaw (single-image oriented) API is simple:
LibRaw lr;
lr.imgdata.params.shot_select = N; // by default it is 0, generally shot_select=1 is used after 0th frame read to merge two sub-frames for Fuji cameras; that's all
lr.open_file(char *filename); // Opens file and reads all metadata
lr.unpack(); // unpack RAW data
lr.dcraw_process(); // to get RGB bitmap from raw
This is definitely not movie-oriented way. For movies one need other API.
BlackMagic is another story because separate DNG for each frame are stored. RedOne support is not used anywhere AFAIK.
Yeah, the MLV format is designed to have frames appear out-of-order to be able to optimize memory block and write sizes.
Nevertheless it is simple to overcome this - there is a .idx file which contains the file numbers and offsets to VIDF blocks.
These file can be generated using mlv_dump or with the in-camera player.
If you can send me the direct link to the API the mlv routines have to fulfill, i can prepare what is necessary to index the files on startup etc.
typedef struct {
uint16_t fileNumber; /* the logical file number as specified in header */
uint16_t empty; /* for future use. set to zero. */
uint64_t frameOffset; /* the file offset at which the frame is stored (VIDF/AUDF) */
} PACKED mlv_xref_t;
typedef struct {
uint8_t blockType[4]; /* can be added in post processing when out of order data is present */
uint32_t blockSize; /* this can also be placed in a separate file with only file header plus this block */
uint64_t timestamp;
uint32_t frameType; /* bitmask: 1=video, 2=audio */
uint32_t entryCount; /* number of xrefs that follow here */
//mlv_xref_t xrefEntries; /* this structure refers to the n'th video/audio frame offset in the files */
} PACKED mlv_xref_hdr_t;
Thanks for info.
I've take a quick look into the specs. It looks like, there is no way to locate Nth frame without reading all previous frames, right?
In LibRaw paradigm it will result into very inefficient applications: for movie apps one need API like get_next_frame(), while photography-oriented API (with one-two frame(s) per file) is entirely different.
Without stream-oriented API, support for MLV in LibRaw will be useless.
Ah great!
One more thing I dont quite understand.
I read tone curve from a cr2. This is linear.
I read it from a NEF and it looks quite strange. It grows exponentially from 0 to about 770 and then continues a linear grow. Dos these values also have to undergo a calculation?
You need to divide to G1 value (1253).
I have one more question.
I read the cam_mul from a cr2.
I got 1987 - 1253 - 2675 - 1253.
From what I read about White Balance Multipliers I was expecting Values in a range of 1.0 - 3.0.
Do I have to devide cam_mul by 1000?
For Canon/Nikon cameras, camera profile is static and defined in adobe_coeff() function (in internal/dcraw_common.cpp).
For some cameras (Olympus, Samsung, Phase One), and formats (DNG) camera profile is provided in RAW file (and enabled via params.use_camera_matrix=1)
Thanks for the reply,
Is the camera color profile something static per model or is it something variable and an interpreter of the raw file needs?
Sorry, I am just providing a tool for data extraction. My knowledge about digital imaging is pretty limited..
cam_mul is camera WB multipliers. From camera.
cam_xyz, pre_mul and rgb_cam are from camera color profile, so from LibRaw.
curve is linear in Canon raws, and from camera for Nikon raws.
To get the same result you need to reproduce hidden exposure correction and tone curve from ViewNX.
I know nothing about ViewNX internals, so cannot recommend something meaningful (try LibRaw's exposure correction with highlights compression as in my StackExchange reply)
I cant figure out if I am doing something wrong, or if its even possible to get the same result as the output of the nikon viewnx converter tool.
I am happy to provide a raw file and a expected image converted with viewns, if its something really easy for someone good at libraw/dcraw to confirm or deconfirm that its possible.
I tried imgD.params.use_camera_wb = 1; and imgD.params.no_auto_bright = 1; which seems to be close. But the image is a bit to dark then. and if using no auto scale, it becomes almost black.
I gave up on just using the raw data without processing because I could not figure out how to make opencv do the demosaic.
Depending of what normalization you need, you may use
1) imgdata.params.no_auto_bright=1 to disable auto brighten
2) params.no_auto_scale=1 to disable data scaling to 65536/camera_data_maximum
3) or just use raw data as is from imgdata.rawdata.raw_image (for bayer data) without calls to LibRaw::dcraw_process()
If possible can you help me disable these things?
my issue is posted here also:
http://stackoverflow.com/questions/22355491/libraw-is-making-my-images-t...
use -arch i386 -arch x86_64 (both flags) in compiler flags when building the library
DNG is just a TIFF file with custom tags. So, you may modify any tiff-writing code to write DNG.
I am looking for the simplest possible solution to a simple problem: to compute the average of a number of images stored as dng files, and save it as a dng file. I have not found any ready-made app and have just spent a frustrating afternoon failing to build Adobe's dng sdk. Can you suggest something practical?
Yes, Foveon color rendering is very approximate.
The field uneven-ess is stored in Foveon files metadata, but it is undocumented by Sigma, so no idea how to use it.
Hope, someone will contribute the right Foveon RGB processing code into LibRaw.
Hi, nice to see Sigmas Merrill sensor supported, i did a ICC file of my DP2m and posted a link and my result here:
http://www.dpreview.com/forums/post/53209141
Hope to see even more improvment on the green cast, for now only BW images of Merrills are good lokking, the noice is looking lovely like analog.
Cheers
Fixed in 0.16-stable branch https://github.com/LibRaw/LibRaw/tree/0.16-stable
With default settings, LibRaw makes these adjustments on processing (dcraw_process()), very similar to dcraw processing:
0) Black level subtracted
1) White balance applied
2) Camera color profile applied
3) Range adjusted to 1% of pixels set to saturation
All these adjustments can be turned off via imgdata.params.* tuning
Thanks. To be fixed in 0.16-stable and master branches.
Yes it seems that exiv2 gives me what I need. Thank you very much Alex!
Libraw not parse EXIF data for camera serial number.
Use other EXIF parsers (libexiv2 or exiftool)
i can encapsulate everything needed to mimic that behavior in some mlv support code.
on open the mlv code will generate the index and spits out frame by frame if you call another routine.
This looks very complicated for me.
LibRaw API is file-oriented. One pass filename to LibRaw::open_file(), then operates with LibRaw object which hide all internals into it.
For .MLV files there are many complex entries:
- .idx file that contains fileIndex (but not file names)
- .mlv (?) files whitch contains fileIndex and fileCount.
From LibRaw API view, it's looks like we need to pass filename.idx to open_file(), but how to recover .mlv filenames from .idx? Is LibRaw need to read headers from all .mlv files from folder or what? What should it do if some files are missing? What should it do if several files with same fileIndex exists?
LibRaw (single-image oriented) API is simple:
This is definitely not movie-oriented way. For movies one need other API.
BlackMagic is another story because separate DNG for each frame are stored. RedOne support is not used anywhere AFAIK.
Yeah, the MLV format is designed to have frames appear out-of-order to be able to optimize memory block and write sizes.
Nevertheless it is simple to overcome this - there is a .idx file which contains the file numbers and offsets to VIDF blocks.
These file can be generated using mlv_dump or with the in-camera player.
If you can send me the direct link to the API the mlv routines have to fulfill, i can prepare what is necessary to index the files on startup etc.
see https://bitbucket.org/hudson/magic-lantern/src/tip/modules/mlv_rec/mlv.h...
typedef struct {
uint16_t fileNumber; /* the logical file number as specified in header */
uint16_t empty; /* for future use. set to zero. */
uint64_t frameOffset; /* the file offset at which the frame is stored (VIDF/AUDF) */
} PACKED mlv_xref_t;
typedef struct {
uint8_t blockType[4]; /* can be added in post processing when out of order data is present */
uint32_t blockSize; /* this can also be placed in a separate file with only file header plus this block */
uint64_t timestamp;
uint32_t frameType; /* bitmask: 1=video, 2=audio */
uint32_t entryCount; /* number of xrefs that follow here */
//mlv_xref_t xrefEntries; /* this structure refers to the n'th video/audio frame offset in the files */
} PACKED mlv_xref_hdr_t;
Thanks for info.
I've take a quick look into the specs. It looks like, there is no way to locate Nth frame without reading all previous frames, right?
In LibRaw paradigm it will result into very inefficient applications: for movie apps one need API like get_next_frame(), while photography-oriented API (with one-two frame(s) per file) is entirely different.
Without stream-oriented API, support for MLV in LibRaw will be useless.
Pages