LibRaw 0.16 Release

Submitted by lexa on

LibRaw 0.16-Release changes (since 0.15.x):
  • Support for new cameras:
    • Baumer TXG14
    • Blackmagic Cinema
    • Canon EOS 70D, C500, S120, G16
    • Fujifilm X-M1, X-A1, XE2, XQ1
    • Hasselblad Lunar, Stellar
    • Leica C, X VARIO
    • Nikon D5200, P7800, D5300, D610, Df, 1 AW1
    • Nokia Lumia 1020, 1520
    • Olympus E-P5,E-M1, STYLUS1
    • OmniVision OV5647 (Raspberry Pi)
    • Panasonic LF1, GX7, GF6, GM1
    • Pentax K-50, K-500, Q7,K-3
    • Richon GR
    • Samsung NX300, NX1100, NX2000, Galaxy NX (EK-GN120)
    • Sony RX100II, RX1R, NEX-3N,NEX-5T, RX10, A3000 (ILCE-3000), A7, A7R
  • Support for Foveon sensor based on X3F code by Roland Karlsson

    BSD-like license, so included in main LibRaw code. No 'foveon intepolation', so no way to get good colors from old Sigma cameras (SD9, SD14, Polaroid x530). For modern Foveon cameras one may try to create ICC profile (not supplied).

    • Support for Medium resolution RAWs from DPx Merrill and SD1 cameras. RAW data extracted as is (4800x1600 pixels), aspect ratio is set to 0.5, so these RAWs are processed to full-size 4800x3200 RGB.
    • Support for Foveon thumbnail extraction. Only JPEG and bitmap thumbnails extracted, but 'foveon' (RAW) thumbnails are really not used in production cameras.
    • New imgdata.params.force_foveon_x3f flag Forces use of x3f-tools based code for Foveon processing if LibRaw compiled with demosaic-pack-GPL2 (and does nothing if LibRaw compiled without this pack). New flag -disadcf added to dcraw_emu sample to use this flag.
API Changes:
  • New parameters in imgdata.params:

    - imgdata.params.no_interpolation - disables interpolation step in LibRaw::dcraw_process() call.

    - imgdata.params.no_auto_scale - disables call to scale_colors() in LibRaw::dcraw_process() call.

    - imgdata.params.sraw_ycc - disables Canon sRAW YCbCr to RGB conversion in LibRaw::unpack() call (use for RAW analyzers

  • New Fuji X-Trans handling: - imgdata.iparams.filters value is now 9 for Fuji X-Trans (instead of 2) - imgdata.iparams.xtrans[6][6] matrix contains row/col to color mapping for Fuji X-Trans sensor.
  • LibRaw::setCancelFlag() - use for fast decoder termination
  • LibRaw_abstract_datastream::make_byte_buffer() call is not needed more.
Other Changes
  • New demosaic code: DHT Demosaic by Anton Petrusevich

    Set params.user_qual=11 to use.

  • New demosaic code: Modified AHD Demosaic by Anton Petrusevich

    Set params.user_qual=12 to use.

  • New C-API call libraw_COLOR(libraw_data_t *t, int row,int col) (so LibRaw::COLOR(row,col) exposed to C-API users)
  • Better EXIF parsing:
    • ISO values for new Nikon cameras (D4, D800)
    • black level extraction for Nikon D5300
    • correct Olympus color data conversion
    • Sony cameras model name is set by SonyModelID EXIF tag
    • Sony ARW2: black level and color matrix extracted from EXIF data
    • Samsung: black level and color matrix extracted from EXIF; Camera multipliers are now extracted correctly even if black is not 0
  • Removed faster lossless jpeg decoder ported from RawSpeed library some years ago. Build LibRaw with RawSpeed to get fast decoding.
  • Canon EOS model name normalized by unique ID table
  • DNG without Compression tag assumed uncompressed
  • Backported 0.15.4 input data checks
  • Support for CMake builds

Comments

Does it support multi-core?

Submitted by John Thomas (not verified) on

Hi Iliah,

I'm new here. :-) Some questions about LibRaw:

1. It has calling compatibility with DCRaw? (IOW can I switch from DCraw to LibRaw without any modifications in code?)

2.a. It uses all the cores of a multi-core CPU?

2.b. It is faster than DCRaw?

Thanks in advance

1. dcraw is utility, LibRaw

Submitted by lexa on

1. dcraw is utility, LibRaw is library. Generally, you need to change your source code
2a. Yes, if compiled with OpenMP support
2b. Yes in some cases

-- Alex Tutubalin @LibRaw LLC

Thanks for info.

Submitted by lexa on

Thanks for info.
I've take a quick look into the specs. It looks like, there is no way to locate Nth frame without reading all previous frames, right?
In LibRaw paradigm it will result into very inefficient applications: for movie apps one need API like get_next_frame(), while photography-oriented API (with one-two frame(s) per file) is entirely different.
Without stream-oriented API, support for MLV in LibRaw will be useless.

-- Alex Tutubalin @LibRaw LLC

MLV Format

Submitted by g3gg0 (not verified) on

Yeah, the MLV format is designed to have frames appear out-of-order to be able to optimize memory block and write sizes.
Nevertheless it is simple to overcome this - there is a .idx file which contains the file numbers and offsets to VIDF blocks.
These file can be generated using mlv_dump or with the in-camera player.

If you can send me the direct link to the API the mlv routines have to fulfill, i can prepare what is necessary to index the files on startup etc.

see https://bitbucket.org/hudson/magic-lantern/src/tip/modules/mlv_rec/mlv.h...

typedef struct {
uint16_t fileNumber; /* the logical file number as specified in header */
uint16_t empty; /* for future use. set to zero. */
uint64_t frameOffset; /* the file offset at which the frame is stored (VIDF/AUDF) */
} PACKED mlv_xref_t;

typedef struct {
uint8_t blockType[4]; /* can be added in post processing when out of order data is present */
uint32_t blockSize; /* this can also be placed in a separate file with only file header plus this block */
uint64_t timestamp;
uint32_t frameType; /* bitmask: 1=video, 2=audio */
uint32_t entryCount; /* number of xrefs that follow here */
//mlv_xref_t xrefEntries; /* this structure refers to the n'th video/audio frame offset in the files */
} PACKED mlv_xref_hdr_t;

This looks very complicated

Submitted by lexa on

This looks very complicated for me.
LibRaw API is file-oriented. One pass filename to LibRaw::open_file(), then operates with LibRaw object which hide all internals into it.
For .MLV files there are many complex entries:
- .idx file that contains fileIndex (but not file names)
- .mlv (?) files whitch contains fileIndex and fileCount.

From LibRaw API view, it's looks like we need to pass filename.idx to open_file(), but how to recover .mlv filenames from .idx? Is LibRaw need to read headers from all .mlv files from folder or what? What should it do if some files are missing? What should it do if several files with same fileIndex exists?

LibRaw (single-image oriented) API is simple:

LibRaw lr;
lr.imgdata.params.shot_select = N; // by default it is 0, generally shot_select=1 is used after 0th frame read to merge two sub-frames for Fuji cameras; that's all
lr.open_file(char *filename); // Opens file and reads all metadata
lr.unpack(); // unpack RAW data
lr.dcraw_process(); // to get RGB bitmap from raw

This is definitely not movie-oriented way. For movies one need other API.

BlackMagic is another story because separate DNG for each frame are stored. RedOne support is not used anywhere AFAIK.

-- Alex Tutubalin @LibRaw LLC

nah, not complicated at all

Submitted by g3gg0 (not verified) on

i can encapsulate everything needed to mimic that behavior in some mlv support code.
on open the mlv code will generate the index and spits out frame by frame if you call another routine.

Sigma Merrill

Submitted by Max (not verified) on

Hi, nice to see Sigmas Merrill sensor supported, i did a ICC file of my DP2m and posted a link and my result here:
http://www.dpreview.com/forums/post/53209141

Hope to see even more improvment on the green cast, for now only BW images of Merrills are good lokking, the noice is looking lovely like analog.

Cheers

Yes, Foveon color rendering

Submitted by lexa on

Yes, Foveon color rendering is very approximate.
The field uneven-ess is stored in Foveon files metadata, but it is undocumented by Sigma, so no idea how to use it.
Hope, someone will contribute the right Foveon RGB processing code into LibRaw.

-- Alex Tutubalin @LibRaw LLC

SONY ILCE-7R (aka A7R)

Submitted by robgrune on

My Digikam 3.5.0 can open Sony A7R RAW files, BUT the colours are strange and images appear as though under frosted glass.

Can you correct and add the A7R camera to your list of supported cameras?

Thank you.

A7R is supported in (internal

Submitted by lexa on

A7R is supported in (internal) development branch. It is under (heavy) internals change, so I do not push it to github (public repo) on regular schedule.
I can push it once, but this is *development* release, no guarantee for seamless work (and, sure, it is not binary compatible with 0.16).

-- Alex Tutubalin @LibRaw LLC