I'm compiling the RC2 code with VS2017, and /W3. I'm getting a lot of warnings like these:
1>c:\users\amonra\documents\github\dss\libraw\src\demosaic\dht_demosaic.cpp(176): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data
1>c:\users\amonra\documents\github\dss\libraw\src\demosaic\dht_demosaic.cpp(747): warning C4305: '/=': truncation from 'double' to 'float'
Yes, of course I can disable these warnings, but I'd like to be sure it's OK do that?
Thanks for the update Alex and all the work you and the LibRaw team do! If you need samples just let me know I currently have two files I could provide. I look forward to an upcoming release with the lossy support added.
Hi Iliah,
Thanks for the pointers. I do have an understanding of color science. Its just that with dcraw having multiple properties that describe very similar things, I was just trying to ensure I was using the right ones.
You can start here: what does cam_xyz represent, what kind of colour transform, what is it applied to, and what's the result? Before you know the answer the life will be not easy. Put it another way, if you are dealing with colour, it's only logical to start with a bit of study of colour engineering, and not to rely on guessing from the code or trial and error.
Alex,
Please forgive me if I gave the impression that I am trying to replicate DCRAW/libraw.
I am creating my own RAW processing pipeline but I am relying on libraw to read the CFA and metadata for the color transformation to implement this pipeline.
Due to the lack of documentation, I am just trying to understand how these coefficients work or are meant to be used to make sure I am implementing my pipeline correctly.
For example, for the Google Pixel 3 camera, I had to derive the cam_xyz from rgb_cam. However, the resulting matrix resulted in a strong color cast probably because I did not undo the normalization after doing INV(rgb_cam).
Also, I assumed that cam_mul would be sufficient because they are the as shot WB multipliers but pre_mul appears to be needed because of the cam_rgb normalization.
Alex,
I noticed that during the computation of rgb_cam from cam_xyz in the cam_xyz_coeff function, there is a normalization that is applied when cam_rgb is computed and the pre_mul is updated. The annotated DCRAW site also does not provide much explanation about this.
The author's comment is "But I don’t fully understand what this tiny little section of code really does. I’d love to hear from someone who can explain it to me."
Given this I have some questions:
1. For camera's (like Google Pixel 3 above), where cam_xyz has to be derived from cmatrix (or rgb_cam) and xyz_rgb, how can I undo the normalization when calculating cam_xyz?
2. During cam_rgb normalization, the pre_mul is updated. So, I assume that the coeffs in pre_mul need to be applied to the RAW data before demosaicing and applying rgb_cam to convert from Camera Space -> sRGB?
Sorry, missed these notices because of *very long* reply:
>> Also properties such as gimbal pitch, yaw, roll among others are removed.
>>Should I give you a copy of this image to take a look at the fields that are removed?
LibRaw::write_tiff_ppm() used in dcraw_emu creates very minimal set of EXIF tags.
There is no plans to extend this.
not too convenient to look at tags as a flat list.
In verbose output I see PrimaryPlatform set to four zeroes in ICC profile section.
I don’t think this is a big problem (unless some real software will complain about it).
Thank you for the quick response; I downloaded compiled and ran the latest github version.
I had another question; the tiff version adds a property, this one in particular looks like the malformed gps info.
[+Primary Platform : Unknown ()]
Is that a normal property value
Also properties such as gimbal pitch, yaw, roll among others are removed.
--- RAW01 2020-07-04 16:29:18.567144467 +0800
+++ TIFF01 2020-07-04 16:29:26.662419543 +0800
@@ -1,138 +1,80 @@
ExifTool Version Number : 11.88
-File Name : IMG_0999.DNG
+File Name : IMG_0999.DNG.tiff
Directory : .
-File Size : 40 MB
-File Modification Date/Time : 2020:07:04 13:48:55+08:00
-File Access Date/Time : 2020:07:04 13:48:55+08:00
-File Inode Change Date/Time : 2020:07:04 13:48:55+08:00
-File Permissions : rwxr-xr-x
-File Type : DNG
-File Type Extension : dng
-MIME Type : image/x-adobe-dng
+File Size : 57 MB
+File Modification Date/Time : 2020:07:04 16:26:51+08:00
+File Access Date/Time : 2020:07:04 16:26:51+08:00
+File Inode Change Date/Time : 2020:07:04 16:26:51+08:00
+File Permissions : rw-rw-r--
+File Type : TIFF
+File Type Extension : tif
+MIME Type : image/tiff
Exif Byte Order : Little-endian (Intel, II)
-Make : Hasselblad
-Camera Model Name : L1D-20c
-Orientation : Horizontal (normal)
-Software : 10.00.11.04
-Modify Date : 2020:06:30 16:02:39
-Image Width : 5568
+Subfile Type : Full-resolution image
+Image Width : 5472
Image Height : 3648
-Bits Per Sample : 16
+Bits Per Sample : 8 8 8
Compression : Uncompressed
-Photometric Interpretation : Color Filter Array
-Strip Offsets : 1041083
-Samples Per Pixel : 1
+Photometric Interpretation : RGB
+Image Description :
+Make : Hasselblad
+Camera Model Name : L1D-20c
+Strip Offsets : 1872
+Samples Per Pixel : 3
Rows Per Strip : 3648
-Strip Byte Counts : 40624128
+Strip Byte Counts : 59885568
+X Resolution : 300
+Y Resolution : 300
Planar Configuration : Chunky
-CFA Repeat Pattern Dim : 2 2
-CFA Pattern 2 : 0 1 1 2
-CFA Plane Color : Red,Green,Blue
-CFA Layout : Rectangular
-Black Level Repeat Dim : 2 2
-Black Level : 4096 4092 4091 4096
-White Level : 65535
-Default Scale : 1 1
-Default Crop Origin : 4 4
-Default Crop Size : 5464 3640
-Bayer Green Split : 0
-Anti Alias Strength : 1
-Best Quality Scale : 1
-Active Area : 0 96 3648 5568
-Opcode List 3 : GainMap, WarpRectilinear
-Default User Crop : 0 0 1 1
-Subfile Type : Reduced-resolution image
-Preview Image Start : 75970
-Preview Image Length : 965113
-Y Cb Cr Coefficients : 0.299 0.587 0.114
-Y Cb Cr Sub Sampling : YCbCr4:2:0 (2 2)
-Y Cb Cr Positioning : Co-sited
-Reference Black White : 0 255 128 255 128 255
-About : DJI Meta Data
-Format : image/dng
-Absolute Altitude : +254.35
-Relative Altitude : +120.20
-Gimbal Roll Degree : +0.00
-Gimbal Yaw Degree : +12.20
-Gimbal Pitch Degree : -90.00
-Flight Roll Degree : -4.90
-Flight Yaw Degree : +8.50
-Flight Pitch Degree : +4.50
-Cam Reverse : 0
-Gimbal Reverse : 0
-Self Data : DJI Self data
-Version : 7.0
-Has Settings : False
-Has Crop : False
-Already Applied : False
+Resolution Unit : inches
+Software : dcraw v9.26
+Modify Date : 2020:06:30 16:02:39
+Artist :
Exposure Time : 1/120
F Number : 4.0
-Exposure Program : Program AE
ISO : 100
-Exif Version : 0230
-Date/Time Original : 2020:06:30 16:02:39
-Create Date : 2020:06:30 16:02:39
-Exposure Compensation : 0
-Max Aperture Value : 2.8
-Metering Mode : Center-weighted average
-Light Source : Fluorescent
-Flash : No Flash
Focal Length : 10.3 mm
-File Source : Digital Camera
-Scene Type : Directly photographed
-Exposure Mode : Auto
-White Balance : Auto
-Digital Zoom Ratio : 1
-Focal Length In 35mm Format : 28 mm
-Scene Capture Type : Standard
-Gain Control : None
-Contrast : Normal
-Saturation : Normal
-Sharpness : Normal
-Serial Number : 0K8TFAA0020268
-Lens Info : 28mm f/2.8-11
-GPS Version ID : 2.3.0.0
+Profile CMM Type :
+Profile Version : 2.1.0
+Profile Class : Display Device Profile
+Color Space Data : RGB
+Profile Connection Space : XYZ
+Profile Date Time : 0000:00:00 00:00:00
+Profile File Signature : acsp
+Primary Platform : Unknown ()
+CMM Flags : Not Embedded, Independent
+Device Manufacturer : none
+Device Model :
+Device Attributes : Reflective, Glossy, Positive, Color
+Rendering Intent : Perceptual
+Connection Space Illuminant : 0.9642 1 0.82491
+Profile Creator :
+Profile ID : 0
+Profile Copyright : auto-generated by dcraw
+Profile Description : sRGB gamma 2.222 toe slope 4.5
+Media White Point : 0.95045 1 1.08905
+Media Black Point : 0 0 0
+Red Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
+Green Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
+Blue Tone Reproduction Curve : (Binary data 14 bytes, use -b option to extract)
+Red Matrix Column : 0.43608 0.2225 0.01393
+Green Matrix Column : 0.38509 0.71689 0.09709
+Blue Matrix Column : 0.14305 0.06061 0.71402
+GPS Version ID : 2.2.0.0
GPS Latitude Ref : North
GPS Longitude Ref : East
GPS Altitude Ref : Above Sea Level
-DNG Version : 1.4.0.0
-DNG Backward Version : 1.3.0.0
-Unique Camera Model : Hasselblad L1D-20c
-Color Matrix 1 : 1.2385 -0.7159 -0.0657 -0.0982 1.0352 0.0721 0.0457 0.0241 0.7437
-Color Matrix 2 : 0.731 -0.2746 -0.0646 -0.2991 1.0847 0.2469 0.0163 0.0585 0.6324
-Analog Balance : 1 1 1
-As Shot Neutral : 0.356545961 1 0.5400843882
-Baseline Exposure : 0
-Baseline Noise : 1
-Baseline Sharpness : 1
-Linear Response Limit : 1
-Camera Serial Number : 0K8TFAA0020268
-Shadow Scale : 1
-DNG Private Data : (Binary data 19829 bytes, use -b option to extract)
-Calibration Illuminant 1 : Standard Light A
-Calibration Illuminant 2 : D65
-Profile Name : Embedded
-Profile Hue Sat Map Dims : 18 6 1
-Profile Hue Sat Map Data 1 : (Binary data 3408 bytes, use -b option to extract)
-Profile Hue Sat Map Data 2 : (Binary data 3439 bytes, use -b option to extract)
-Profile Embed Policy : Allow Copying
-Noise Profile : 8.108e-05 6e-08
-Original Default Final Size : 0 0
-Original Best Quality Size : 0 0
-Original Default Crop Size : 0 0
+GPS Time Stamp : 00:00:00
+GPS Map Datum :
+GPS Date Stamp :
Aperture : 4.0
-CFA Pattern : [Red,Green][Green,Blue]
-Image Size : 5568x3648
-Megapixels : 20.3
-Preview Image : (Binary data 965113 bytes, use -b option to extract)
-Scale Factor To 35 mm Equivalent: 2.7
+Image Size : 5472x3648
+Megapixels : 20.0
Shutter Speed : 1/120
GPS Altitude : 254 m Above Sea Level
+GPS Date/Time : 00:00:00Z
GPS Latitude : xxx
GPS Longitude : xxx
-Circle Of Confusion : 0.011 mm
-Field Of View : 65.5 deg
-Focal Length : 10.3 mm (35 mm equivalent: 28.0 mm)
+Focal Length : 10.3 mm
GPS Position : xxx
-Hyperfocal Distance : 2.39 m
Light Value : 10.9
Should I give you a copy of this image to take a look at the fields that are removed?
I'm compiling the RC2 code with VS2017, and /W3. I'm getting a lot of warnings like these:
1>c:\users\amonra\documents\github\dss\libraw\src\demosaic\dht_demosaic.cpp(176): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data
1>c:\users\amonra\documents\github\dss\libraw\src\demosaic\dht_demosaic.cpp(747): warning C4305: '/=': truncation from 'double' to 'float'
Yes, of course I can disable these warnings, but I'd like to be sure it's OK do that?
Thanks
David
Yes, samples are welcome (although we have several ones).
If you do not want to share it in public, use WeTransfer service/free option and send samples to info@libraw.org.
Thanks for the update Alex and all the work you and the LibRaw team do! If you need samples just let me know I currently have two files I could provide. I look forward to an upcoming release with the lossy support added.
Hi Iliah,
Thanks for the pointers. I do have an understanding of color science. Its just that with dcraw having multiple properties that describe very similar things, I was just trying to ensure I was using the right ones.
Regards,
Dinesh
X-T4
losslesslossy compressed files are not supported for LibRaw yet.UPD: correction, lossless files are supported, lossy are not.
You can start here: what does cam_xyz represent, what kind of colour transform, what is it applied to, and what's the result? Before you know the answer the life will be not easy. Put it another way, if you are dealing with colour, it's only logical to start with a bit of study of colour engineering, and not to rely on guessing from the code or trial and error.
Libraw binary distribution for Mac is built using unedited Makefile.dist
make -f Makefile.dist worked on Intel and ARM. I merged both libs with lipo.
Should I change any settings in the Makefile from the default?
We do not have ARM mac hardware, so unable to test. I do not think we'll have it in very nearest future.
You need to rebuild LibRaw by yourself, use Makefile.dist as a starting point.
Hi,
I work currently on the port of my app for macOS 11 on ARM/Intel.
When will be a Universal Binary available?
Thorsten
pre_mul is not used for normalization. It derived from cam_rgb data (to get daylight wb coeffs)
Alex,
Please forgive me if I gave the impression that I am trying to replicate DCRAW/libraw.
I am creating my own RAW processing pipeline but I am relying on libraw to read the CFA and metadata for the color transformation to implement this pipeline.
Due to the lack of documentation, I am just trying to understand how these coefficients work or are meant to be used to make sure I am implementing my pipeline correctly.
For example, for the Google Pixel 3 camera, I had to derive the cam_xyz from rgb_cam. However, the resulting matrix resulted in a strong color cast probably because I did not undo the normalization after doing INV(rgb_cam).
Also, I assumed that cam_mul would be sufficient because they are the as shot WB multipliers but pre_mul appears to be needed because of the cam_rgb normalization.
I am just looking for more information on these.
Thanks for all your help thus far!
Dinesh
As far as I understood from the previous discussion, you're creating your own postprocessing for RAW.
I do not thing you need to replicate LibRaw/dcraw color processing pipeline.
Alex,
I noticed that during the computation of rgb_cam from cam_xyz in the cam_xyz_coeff function, there is a normalization that is applied when cam_rgb is computed and the pre_mul is updated. The annotated DCRAW site also does not provide much explanation about this.
The author's comment is "But I don’t fully understand what this tiny little section of code really does. I’d love to hear from someone who can explain it to me."
Given this I have some questions:
1. For camera's (like Google Pixel 3 above), where cam_xyz has to be derived from cmatrix (or rgb_cam) and xyz_rgb, how can I undo the normalization when calculating cam_xyz?
2. During cam_rgb normalization, the pre_mul is updated. So, I assume that the coeffs in pre_mul need to be applied to the RAW data before demosaicing and applying rgb_cam to convert from Camera Space -> sRGB?
3. When should I use cam_mul then?
Dinesh
There is no such program in LibRaw 'sample code' programs.
One could implement such program using LibRaw or Adobe DNG sdk for raw (dng) decode and something like libtiff for tiff write.
Apologies for the long response.
Do you have a program that can just convert the dng file to tiff without changing any of the dng's exif metadata?
Sorry, missed these notices because of *very long* reply:
>> Also properties such as gimbal pitch, yaw, roll among others are removed.
>>Should I give you a copy of this image to take a look at the fields that are removed?
LibRaw::write_tiff_ppm() used in dcraw_emu creates very minimal set of EXIF tags.
There is no plans to extend this.
I just downloaded the sample image and the exif file; I ran the test and still get the same errors when I try to load the .tiff file.
I ran a exiftool -v and there were some warnings
[Warning = ValueConv GPSTimeStamp: Argument "undef" isn't numeric in addition (+)]
There's no such errors in the raw file
Here are tiff file produced by dcraw_emu -T (from github/LibRaw/master branch) and result of exiftool -v3 tiff-file: https://www.dropbox.com/sh/6q9d3hd10z8x06m/AAA7QgHY9yf_KFK38JmZ5WF9a?dl=0
There are no tags 0xa40.. in this file.
apologies about the output, I sent a sample DNG file through the contact link.
I am still getting errors when trying to open the .tiff files
I am using colmap; which uses freeimage to import supported files. DNG files are unsupported but TIFF's are.
I think those unknown() values trips up freeimage.
not too convenient to look at tags as a flat list.
In verbose output I see PrimaryPlatform set to four zeroes in ICC profile section.
I don’t think this is a big problem (unless some real software will complain about it).
Thank you for the quick response; I downloaded compiled and ran the latest github version.
I had another question; the tiff version adds a property, this one in particular looks like the malformed gps info.
[+Primary Platform : Unknown ()]
Is that a normal property value
Also properties such as gimbal pitch, yaw, roll among others are removed.
Should I give you a copy of this image to take a look at the fields that are removed?
Followup:
was able to reproduce, fixed by this patch: https://github.com/LibRaw/LibRaw/commit/5a5b227a715c914e8fbd3ee7f97c0d76...
Please provide sample RAW file(s) to play with.
Alex,
Thanks for your help. FC(r, c) worked as expected. I believe I am all set.
Dinesh
Pages