This is ~2 year old comment.
For now, we know *WHY* Nikon does this: this is camera calibration to make white balance coefficients (presets) the same for all cameras (of given model)
So gamma correction is applied after interpolation? Doesn't dcraw (and libraw) use the camera's contrast curves by default? Are contrast curves the same as "tone curves"?
1. To get LibRaw as close as possible to Nikon converter, you need to repeat Nikon converter in most important things: color profile, contrast curves, white balance data, color dithering.
2. output gamma curve (from 16 bit linear to 8-bit gamma) may have holes. Also, in deep shadows some values are missing because of linear sensor nature.
3. You've used no-auto-bright in LibRaw, but no corresponding switch (-W) for dcraw.
Also, if you wish to repeat dcraw as close as possible, set adjust_maximum_thr to 0.f
Once again thanks for the instant response. My unix is as rusty as my c...had to look up mmap. Can't believe I didn't think to check the buffer stream for reference, face::palm.
just checked for ifname:
- these 'names' are under #ifdef DCRAW_VERBOSE, or #ifndef LIBRAW_LIBRARY_BUILD. The only exlusion is parse_external_jpeg(), but also after if(!ifp->ifname())... return;
Yes, you may implement LibRaw datastream without filename. Look into LibRaw_buffer_datastream as a sample.
Your datastream->fname() should return NULL in this case. This should work OK for most RAW data with one exception:
- old (not current) 'CHDK hack' (or DIAG RAW hack, cannot remember now) creates RAW file without any metadata, just sensor dump.
- all metadata is stored in filename.JPG (same file, different extension).
dcraw (and than LibRaw) is able to parse filename.raw + filename.JPG pair to get metadata (exposure parameters, etc) from the second file.
Another story is make_jas_stream() (from Jasper JPEG2000 library). This call is needed only if you want to decode Red cinema file. Again, you may implement it as 'return 0'; and all things will work OK (exception LIBRAW_EXCEPTION_DECODE_JPEG2000 raised within LibRaw, then converted to return code in unpack())
It looks like their are quite a few references to file path throughout the code despite having the file stream.
Long story short, will I be able to create a data stream that does not have access to a true file path? I should be able to get a file name, but unfortunately that's the extent. If it's not possible the rest of this post is irrelevant, just analysis of the impact of file path in the code. If it is possible, if you don't mind elaborating on the impact it would help me greatly. Thanks.
1) I noted a few line directives from dcraw.c. Do I need to be concerned about its use of file path or just within libraw?
LibRaw_abstract_datastream::subfile_open(const char *fn)
2) I only note this used in a parse_external_jpeg() which seems to be a hack to get metadata. It appears to be a warning at worse on a filename issue. Correct?
3) What is the significance of this function? It appears to open any file really, but has a thread lock. Correct?
#define ifname ((char*)libraw_internal_data.internal_data.input->fname())
4) This seems to be more of a concern, there are 17 references in dcraw_common, a lot in the same LibRaw_abstract_datastream::subfile_open(const char *fn) function and some related to minolta. There are many more references to its own similar 'ifname' in dcraw.c itself if I need to be concerned with that?
Thanks for the extremely fast response. I completely missed the difference in the subfile_open of the two streams in my first glance. So it should just be a simple exercise of replicating big with fdopen.
If I really want to strive for the speed in stream I could see if Android supports any method of FILE* to fstream. How much of an improvement do you think there is? It's Android, so linux, if the improvement only lies in Win.
LibRaw_file_datastream is based on С++ iostreams, while bigfile_datastream uses old plain FILE*.
Iostreams implementation is slightly faster (under Win32, at least), but there is no way to limit file buffer size. Also, iostreams implementation is limited to 2GB (or 4Gb??) files on some systems because of 32-bit nature.
Most raw files are much smaller than 2/4GB (and usually fit into 50-100M with very few exceptions), so (faster) iostream implementation is used. Bigfile_datastream is used for larger files to avoid too large buffers and 2/4Gb limit. The only files that go over limit are cinema files (Red's R3D files and other multi-frame files).
In-camera crops are specific to camera (and camera crop mode). LibRaw does not read this data.
Use full image instead.
Thank you for your quick answer.
Ok, so I have to use:
processor.imgdata.params.output_bps = 16; processor.open_file("test1.CR2"); processor.unpack(); processor.dcraw_process(); libraw_processed_image_t* output = processor.dcraw_make_mem_image();But how i can specify to use the JPEG resolution?
I think it is with:
but how specify the JPEG resolution without hard coding?
This is ~2 year old comment.
For now, we know *WHY* Nikon does this: this is camera calibration to make white balance coefficients (presets) the same for all cameras (of given model)
Yes, gamma correction is applied on output. Internal Libraw/dcraw storage is linear, no tone curve applied.
Also found this: http://www.libraw.org/comment/3287#comment-3287
So gamma correction is applied after interpolation? Doesn't dcraw (and libraw) use the camera's contrast curves by default? Are contrast curves the same as "tone curves"?
1. To get LibRaw as close as possible to Nikon converter, you need to repeat Nikon converter in most important things: color profile, contrast curves, white balance data, color dithering.
2. output gamma curve (from 16 bit linear to 8-bit gamma) may have holes. Also, in deep shadows some values are missing because of linear sensor nature.
3. You've used no-auto-bright in LibRaw, but no corresponding switch (-W) for dcraw.
Also, if you wish to repeat dcraw as close as possible, set adjust_maximum_thr to 0.f
Image fixed (you need to check []Display checkbox to display).
LibRaw's and dcraw's images (with same parameters) should be very close (or identical).
For unknown reason you use no_auto_bright for LibRaw and auto-brightening (default) for dcraw. That's why dcraw's histogram is shifted to the right.
I do not see any attached image.
Bayer image contains only one color component per pixel.
image[][4] array has 4 components per pixel (R,G,B,G2 or CMYG or RGBE).
raw2image() copies bayer data into image array
What do you mean for one component out of 4? I actually noticed this, but I don't really understand what's going on.
For testing (it uses extra memory, so not mobile friendly):
should work.
And replace with your own LibRaw_datastream implementation later.
Once again thanks for the instant response. My unix is as rusty as my c...had to look up mmap. Can't believe I didn't think to check the buffer stream for reference, face::palm.
BTW, for debug code try to
- mmap file
- pass mmap to LibRaw_buffer_datastream.
It should work under Linux (Android). It is not working right under Windows because Win32 memory mapping is some strange thing.
Followup:
just checked for ifname:
- these 'names' are under #ifdef DCRAW_VERBOSE, or #ifndef LIBRAW_LIBRARY_BUILD. The only exlusion is parse_external_jpeg(), but also after if(!ifp->ifname())... return;
Yes, you may implement LibRaw datastream without filename. Look into LibRaw_buffer_datastream as a sample.
Your datastream->fname() should return NULL in this case. This should work OK for most RAW data with one exception:
- old (not current) 'CHDK hack' (or DIAG RAW hack, cannot remember now) creates RAW file without any metadata, just sensor dump.
- all metadata is stored in filename.JPG (same file, different extension).
dcraw (and than LibRaw) is able to parse filename.raw + filename.JPG pair to get metadata (exposure parameters, etc) from the second file.
Another story is make_jas_stream() (from Jasper JPEG2000 library). This call is needed only if you want to decode Red cinema file. Again, you may implement it as 'return 0'; and all things will work OK (exception LIBRAW_EXCEPTION_DECODE_JPEG2000 raised within LibRaw, then converted to return code in unpack())
It looks like their are quite a few references to file path throughout the code despite having the file stream.
Long story short, will I be able to create a data stream that does not have access to a true file path? I should be able to get a file name, but unfortunately that's the extent. If it's not possible the rest of this post is irrelevant, just analysis of the impact of file path in the code. If it is possible, if you don't mind elaborating on the impact it would help me greatly. Thanks.
1) I noted a few line directives from dcraw.c. Do I need to be concerned about its use of file path or just within libraw?
LibRaw_abstract_datastream::subfile_open(const char *fn)2) I only note this used in a
parse_external_jpeg()which seems to be a hack to get metadata. It appears to be a warning at worse on a filename issue. Correct?3) What is the significance of this function? It appears to open any file really, but has a thread lock. Correct?
#define ifname ((char*)libraw_internal_data.internal_data.input->fname())
4) This seems to be more of a concern, there are 17 references in dcraw_common, a lot in the same
LibRaw_abstract_datastream::subfile_open(const char *fn)function and some related to minolta. There are many more references to its own similar 'ifname' in dcraw.c itself if I need to be concerned with that?Or use LibRaw::COLOR(row,col) call
Looks like it is actually the duty of imgdata.idata.filters to provide that order.
I'm coming back on that one, I wanted to know how imgdata.idata.cdesc returns the CFA pattern: Is it in clock-wise order or on a per row basis?:
Taking RGBG and respective 1-based indices is it:
or
It seems to me that it is in clock-wise order but I would like confirmation.
Cheers,
Thomas
I do not think you'll see any real difference in speed for compressed RAW files.
Thanks for the extremely fast response. I completely missed the difference in the subfile_open of the two streams in my first glance. So it should just be a simple exercise of replicating big with fdopen.
If I really want to strive for the speed in stream I could see if Android supports any method of FILE* to fstream. How much of an improvement do you think there is? It's Android, so linux, if the improvement only lies in Win.
Thanks again!
LibRaw_file_datastream is based on С++ iostreams, while bigfile_datastream uses old plain FILE*.
Iostreams implementation is slightly faster (under Win32, at least), but there is no way to limit file buffer size. Also, iostreams implementation is limited to 2GB (or 4Gb??) files on some systems because of 32-bit nature.
Most raw files are much smaller than 2/4GB (and usually fit into 50-100M with very few exceptions), so (faster) iostream implementation is used. Bigfile_datastream is used for larger files to avoid too large buffers and 2/4Gb limit. The only files that go over limit are cinema files (Red's R3D files and other multi-frame files).
Yes (use_camera_wb fallbacks to daylight if not camera wb present)
Are these the same white balance multipliers that are used when setting:
RawProcessor.imgdata.params.use_camera_wb=1;
?
Pages