Ahh. Yes, tons more I need to learn. Can you recommend a Libraw process to derive linear RGB and BW images from D7000 NEF? I'm looking for a "no-frills" kind of implementation if possible. The output is for a 3D structured light setup which is dependent on capturing a series of linear images.
Thanks again for your help!
Generally, you need to apply WB before demosaic (if your demosaic method use something like luminosity maps, the components should be balanced before).
Also, bias subtraction should be done before WB (and demosaic).
I do not think, that max(R,G,B) will give you good BW image. Better use something like RGB to Y[CC] or Lab conversion and use Y or L channel as BW image.
To follow up. If I demosaic, subtract black, scale with normalized multipliers(cam_mul), then do a max() of all three color components, I should generally have a linear black and white image, correct? Anything else I should consider or be concerned about?
If you want to use your own processing (demosaic), you need to
- subtract black (bias) level. It is zero for D7000, so if you working with only one camera, skip this step
- multiply to WB coeffs.
To not overflow ushort data, you need multipliers less than 4.0 (16-bit range and 14-bit data), so normalize the multipliers (divide to something) to fit range 0.0..4.0
Here is a dump I extracted using exifprobe (linux). The entry Exif.Canon.OwnerName seems to be empty. The author information appears somewhere else, but see by yourself:
Thanks Alex. Could you please explain to me why it is better to manipulate raw data in the linear domain? I have seen this mentioned before, but the reasoning behind it isn't clear to me. The number of individual brightness levels available is determined by the bit depth of the camera, so any interpolated values in the linear domain (calculated during image manipulation) will have to be remapped to this same set of brightness levels. What am I missing?
Another question: I have noticed that some images (e.g. ARW images from the Sony A7) are reported as having top_margin=0 and left_margin=0, but width does not equal raw_width. Is this just a case of masked pixels being on the *right* of the image (in my experience they are usually on the left)? The DNG SDK reports the width as the full raw_width, and on examination the DNG converted images fill this extra image area with repeated values of the pixel at position(x,width), i.e. it duplicates the last pixel in the row to fill the image area between width and raw_width.
Inverted tone curve after unpack() should work fine too.
BTW, it is better to manipulate raw data in 'linear domain' (so, after real tone curve applied) and apply inverted curve before packing your data back (to DNG).
Thanks Alex, I used your suggested code snippet. I'm also testing for application of the tone curve during unpack(), and applying the inverse tone curve as necessary at that point.
Followup: the only format with linearization data applied after unpack() phase (within raw2image or dcraw_process()) is PhaseOne files (both compressed and uncompressed).
The tone curve for DNG files (Linearization Curve in DNG specs terms) is always applied on unpack() phase. There are no 'documented' options to turn this off (because RAW application always want to see linearized data), but you may use this trick:
LibRaw lr;
lr.open_file(filename);
// Replace tone curve read from metadata by linear one:
for(int i=0;i<0x10000;i++)
lr.imgdata.color.curve[i]=i;
// Do the unpack with linear curve:
lr.unpack();
This trick will work with data format with separate curve in metadata (all DNG files; Sony ARW2 format), but will not work with files where curve is calculated on unpack() phase (e.g. Nikon lossy NEF files).
Also please note, that raw2image() will skip 'masked pixels' (pixels outside of ActiveArea in DNG terms). To avoid this you may do the same trick: set imgdata.sizes.width and iwidth to raw_width and zero left_margin (and same with height, iheight and top_margin).
Followup: ABI/API changes will not break existing programs source compatibility. It is usual additional features we need. But you may need to recompile your code to adopt these changes.
We use current development version in our published apps (RawDigger, FastRawViewer).
It is stable in terms of program stability.
It is not stable in ABI/API terms, because internals may change without notifications
Ahh. Yes, tons more I need to learn. Can you recommend a Libraw process to derive linear RGB and BW images from D7000 NEF? I'm looking for a "no-frills" kind of implementation if possible. The output is for a 3D structured light setup which is dependent on capturing a series of linear images.
Thanks again for your help!
Generally, you need to apply WB before demosaic (if your demosaic method use something like luminosity maps, the components should be balanced before).
Also, bias subtraction should be done before WB (and demosaic).
I do not think, that max(R,G,B) will give you good BW image. Better use something like RGB to Y[CC] or Lab conversion and use Y or L channel as BW image.
Great! Thanks!
To follow up. If I demosaic, subtract black, scale with normalized multipliers(cam_mul), then do a max() of all three color components, I should generally have a linear black and white image, correct? Anything else I should consider or be concerned about?
Thanks!
If you want to use your own processing (demosaic), you need to
- subtract black (bias) level. It is zero for D7000, so if you working with only one camera, skip this step
- multiply to WB coeffs.
To not overflow ushort data, you need multipliers less than 4.0 (16-bit range and 14-bit data), so normalize the multipliers (divide to something) to fit range 0.0..4.0
I am using the exifprobe version 2.0.1 . That is the one from the official Ubuntu 14.04 repository.
Program: 'exifprobe' version 2.0.1
Compiled: Tue Jun 15 23:17:15 UTC 2010
Copyright (C) 2005 Duane H. Hesser
Both fields supported by LibRaw (EXIF tag 315 and Canon tag 9) contains empty author name.
Tag #294 is very uncommon (what version of Exifprobe do you use? Version from git repo knows nothing about this tag)
Here is a dump I extracted using exifprobe (linux). The entry Exif.Canon.OwnerName seems to be empty. The author information appears somewhere else, but see by yourself:
File Name = IMG_2015.CR2
File Type = TIFF:II
File Size = 19860267
@000000000=0 : TIFF(II=0x4949) magic=0x002a='*\0' ifd offset = 0x10/16
@0x0000010=16 : 16 entries starting at file offset 0x12=18
@0x0000012=18 : <0x0100= 256> ImageWidth [3 =SHORT 1] = 5184
@0x000001e=30 : <0x0101= 257> ImageLength [3 =SHORT 1] = 3456
@0x000002a=42 : <0x0102= 258> BitsPerSample [3 =SHORT 3] = @0xd6=214
@0x0000036=54 : <0x0103= 259> Compression [3 =SHORT 1] = 6 = 'Exif/old JPEG'
@0x0000042=66 : <0x010f= 271> Make [2 =ASCII 6] = @0xdc=220
@0x000004e=78 : <0x0110= 272> Model [2 =ASCII 15] = @0xe2=226
@0x000005a=90 : <0x0111= 273> StripOffsets [4 =LONG 1] = @52244
@0x0000066=102 : <0x0112= 274> Orientation [3 =SHORT 1] = 1 = '0,0 is top left'
@0x0000072=114 : <0x0117= 279> StripByteCounts [4 =LONG 1] = 960304
@0x000007e=126 : <0x011a= 282> XResolution [5 =RATIONAL 1] = @0x102=258
@0x000008a=138 : <0x011b= 283> YResolution [5 =RATIONAL 1] = @0x10a=266
@0x0000096=150 : <0x0128= 296> ResolutionUnit [3 =SHORT 1] = 2 = 'pixels per inch'
@0x00000a2=162 : <0x0132= 306> DateTime [2 =ASCII 20] = @0x112=274
@0x00000ae=174 : <0x013b= 315> Artist [2 =ASCII 5] = @0x126=294
@0x00000ba=186 : <0x8298=33432> Copyright [2 =ASCII 8] = @0x166=358
@0x00000c6=198 : <0x8769=34665> ExifIFDPointer [4 =LONG 1] = @0x1a6=422
@0x00000d2=210 : **** next IFD offset 46124(+ 0 = 0xb42c/46124)
@0x00000d6=214 : ============= VALUES, IFD 0 ============
@0x00000d6=214 : BitsPerSample = 8,8,8
@0x00000dc=220 : Make = 'Canon\0'
@0x00000e2=226 : Model = 'Canon EOS 550D\0'
@0x0000102=258 : XResolution = 72
@0x000010a=266 : YResolution = 72
@0x0000112=274 : DateTime = '2014:08:31 15:02:38\0'
@0x0000126=294 : Artist = 'Lops\0'
@0x0000166=358 : Copyright = '(C)LOPS\0'
@0x00001a6=422 : (in IFD 0) 31 entries starting at file offset 0x1a8=424
@0x00001a8=424 : <0x829a=33434> ExposureTime [5 =RATIONAL 1] = @0x320=800
@0x00001b4=436 : <0x829d=33437> FNumber [5 =RATIONAL 1] = @0x328=808
@0x00001c0=448 : <0x8822=34850> ExposureProgram [3 =SHORT 1] = 4 = 'Shutter Priority'
@0x00001cc=460 : <0x8827=34855> ISOSpeedRatings [3 =SHORT 1] = 100
@0x00001d8=472 : <0x9000=36864> Version [7 =UNDEFINED 4] = '0221'
@0x00001e4=484 : <0x9003=36867> DateTimeOriginal [2 =ASCII 20] = @0x330=816
@0x00001f0=496 : <0x9004=36868> DateTimeDigitized [2 =ASCII 20] = @0x344=836
@0x00001fc=508 : <0x9101=37121> ComponentsConfiguration [7 =UNDEFINED 4] = 1,2,3,0 = 'YCbCr'
@0x0000208=520 : <0x9201=37377> ShutterSpeedValue [10=SRATIONAL 1] = @0x358=856
@0x0000214=532 : <0x9202=37378> ApertureValue [5 =RATIONAL 1] = @0x360=864
@0x0000220=544 : <0x9204=37380> ExposureBiasValue [10=SRATIONAL 1] = @0x368=872
@0x000022c=556 : <0x9207=37383> MeteringMode [3 =SHORT 1] = 5 = 'Pattern'
@0x0000238=568 : <0x9209=37385> Flash [3 =SHORT 1] = 16 = 'no flash - suppressed'
@0x0000244=580 : <0x920a=37386> FocalLength [5 =RATIONAL 1] = @0x370=880
@0x0000250=592 : <0x927c=37500> MakerNote [7 =UNDEFINED 44926] = @0x378=888
@0x000025c=604 : <0x9286=37510> UserComment [7 =UNDEFINED 264] = @0xb2f6=45814
@0x0000268=616 : <0x9290=37520> SubSecTime [2 =ASCII 3] = '84\000'
@0x0000274=628 : <0x9291=37521> SubSecTimeOriginal [2 =ASCII 3] = '84\000'
@0x0000280=640 : <0x9292=37522> SubSecTimeDigitized [2 =ASCII 3] = '84\000'
@0x000028c=652 : <0xa000=40960> FlashPixVersion [7 =UNDEFINED 4] = '0100'
@0x0000298=664 : <0xa001=40961> ColorSpace [3 =SHORT 1] = 1 = 'sRGB'
@0x00002a4=676 : <0xa002=40962> PixelXDimension [3 =SHORT 1] = 5184
@0x00002b0=688 : <0xa003=40963> PixelYDimension [3 =SHORT 1] = 3456
@0x00002bc=700 : <0xa005=40965> Interoperability [4 =LONG 1] = @0xb3fe=46078
@0x00002c8=712 : <0xa20e=41486> FocalPlaneXResolution [5 =RATIONAL 1] = @0xb41c=46108
@0x00002d4=724 : <0xa20f=41487> FocalPlaneYResolution [5 =RATIONAL 1] = @0xb424=46116
@0x00002e0=736 : <0xa210=41488> FocalPlaneResolutionUnit [3 =SHORT 1] = 2 = 'pixels per inch'
@0x00002ec=748 : <0xa401=41985> CustomRendered [3 =SHORT 1] = 0 = 'Normal'
@0x00002f8=760 : <0xa402=41986> ExposureMode [3 =SHORT 1] = 0 = 'Auto'
@0x0000304=772 : <0xa403=41987> WhiteBalance [3 =SHORT 1] = 0 = 'Auto'
@0x0000310=784 : <0xa406=41990> SceneCaptureType [3 =SHORT 1] = 0 = 'Standard'
@0x000031c=796 : **** next IFD offset 0
@0x0000320=800 : ============= VALUES, EXIF IFD ============
@0x0000320=800 : ExposureTime = 0.025 sec
@0x0000328=808 : FNumber = 5.6 APEX = 'f7.0'
@0x0000330=816 : DateTimeOriginal = '2014:08:31 15:02:38\0'
@0x0000344=836 : DateTimeDigitized = '2014:08:31 15:02:38\0'
@0x0000358=856 : ShutterSpeedValue = 5.375 APEX = '0.024097 sec'
@0x0000360=864 : ApertureValue = 5 APEX = 'f5.7'
@0x0000368=872 : ExposureBiasValue = 0 APEX
@0x0000370=880 : FocalLength = 55 mm
@0x0000378=888 : length 44926, Plain IFD scheme, 39 entries starting at offset 0x37a/890
@0x0000378=888 : MakerNoteId = '''
@0x000037a=890 : <0X0001= 1> CameraSettings [ 3=SHORT 49] = @0x552=1362
@0x0000386=902 : <0X0002= 2> FocusInfo [ 3=SHORT 4] = @0x5b4=1460
@0x0000392=914 : <0X0003= 3> TAG_0X0003 [ 3=SHORT 4] = @0x5bc=1468
@0x000039e=926 : <0X0004= 4> ShotInfo [ 3=SHORT 34] = @0x5c4=1476
@0x00003aa=938 : <0X0006= 6> ImageType [ 2=ASCII 15] = @0x608=1544
@0x00003b6=950 : <0X0007= 7> FirmwareVersion [ 2=ASCII 24] = @0x628=1576
@0x00003c2=962 : <0X0009= 9> OwnerName [ 2=ASCII 32] = @0x640=1600
@0x00003ce=974 : <0X000C= 12> CameraSerialNo [ 4=LONG 1] = 1433718828 = 'D02C53292'
@0x00003da=986 : <0X000D= 13> TAG_0X000D [ 7=UNDEFINED 1536] = @0x660=1632
@0x00003e6=998 : <0X0010= 16> TAG_0X0010 [ 4=LONG 1] = 2147484272
@0x00003f2=1010 : <0X0013= 19> TAG_0X0013 [ 3=SHORT 4] = @0xc60=3168
@0x00003fe=1022 : <0X0015= 21> TAG_0X0015 [ 4=LONG 1] = 2684354560
@0x000040a=1034 : <0X0019= 25> TAG_0X0019 [ 3=SHORT 1] = 1
@0x0000416=1046 : <0X0026= 38> TAG_0X0026 [ 3=SHORT 48] = @0xc68=3176
@0x0000422=1058 : <0X0083= 131> TAG_0X0083 [ 4=LONG 1] = 0
@0x000042e=1070 : <0X0093= 147> FileInfo** [ 3=SHORT 26] = @0xf28=3880
@0x000043a=1082 : <0X0095= 149> TAG_0X0095 [ 2=ASCII 70] = @0xf5c=3932
@0x0000446=1094 : <0X0096= 150> TAG_0X0096 [ 2=ASCII 16] = @0xfa2=4002
@0x0000452=1106 : <0X0097= 151> TAG_0X0097 [ 7=UNDEFINED 1024] = @0xfb2=4018
@0x000045e=1118 : <0X0098= 152> TAG_0X0098 [ 3=SHORT 4] = @0x13b2=5042
@0x000046a=1130 : <0X0099= 153> TAG_0X0099 [ 4=LONG 50] = @0x13ba=5050
@0x0000476=1142 : <0X00A0= 160> ColorInfo** [ 3=SHORT 14] = @0x1482=5250
@0x0000482=1154 : <0X00AA= 170> TAG_0X00AA [ 3=SHORT 6] = @0x149e=5278
@0x000048e=1166 : <0X00B4= 180> TAG_0X00B4 [ 3=SHORT 1] = 1
@0x000049a=1178 : <0X00D0= 208> TAG_0X00D0 [ 4=LONG 1] = 0
@0x00004a6=1190 : <0X00E0= 224> SensorInfo [ 3=SHORT 17] = @0x14aa=5290
@0x00004b2=1202 : <0X4001=16385> TAG_0X4001 [ 3=SHORT 1338] = @0x14cc=5324
@0x00004be=1214 : <0X4002=16386> TAG_0X4002 [ 3=SHORT 10420] = @0x1f40=8000
@0x00004ca=1226 : <0X4005=16389> TAG_0X4005 [ 7=UNDEFINED 16448] = @0x70a8=28840
@0x00004d6=1238 : <0X4008=16392> TAG_0X4008 [ 3=SHORT 3] = @0xb0e8=45288
@0x00004e2=1250 : <0X4009=16393> TAG_0X4009 [ 3=SHORT 3] = @0xb0ee=45294
@0x00004ee=1262 : <0X4010=16400> TAG_0X4010 [ 2=ASCII 32] = @0xb0f4=45300
@0x00004fa=1274 : <0X4011=16401> TAG_0X4011 [ 7=UNDEFINED 252] = @0xb114=45332
@0x0000506=1286 : <0X4012=16402> TAG_0X4012 [ 2=ASCII 32] = @0xb210=45584
@0x0000512=1298 : <0X4015=16405> TAG_0X4015 [ 7=UNDEFINED 116] = @0xb230=45616
@0x000051e=1310 : <0X4016=16406> TAG_0X4016 [ 4=LONG 6] = @0xb2a4=45732
@0x000052a=1322 : <0X4017=16407> TAG_0X4017 [ 4=LONG 2] = @0xb2bc=45756
@0x0000536=1334 : <0X4018=16408> TAG_0X4018 [ 4=LONG 3] = @0xb2c4=45764
@0x0000542=1346 : <0X4019=16409> TAG_0X4019 [ 7=UNDEFINED 30] = @0xb2d0=45776
@0x000054e=1358 : **** next IFD offset 0
@0x0000552=1362 : ============= VALUES, MakerNote ============
@0x0000552=1362 : 48 entries length 98
@0x0000554=1364 : CameraSettings.01_MacroMode = 2 = 'normal'
@0x0000556=1366 : CameraSettings.02_SelfTimer = 0 = 'off'
@0x0000558=1368 : CameraSettings.03_Quality = 4 = 'Lossless JPEG'
@0x000055a=1370 : CameraSettings.04_FlashMode = 0 = 'flash did not fire'
@0x000055c=1372 : CameraSettings.05_ContinuousMode = 0 = 'single or timer'
@0x000055e=1374 : CameraSettings.06_Unknown = 0
@0x0000560=1376 : CameraSettings.07_FocusMode = 0 = 'One Shot AF'
@0x0000562=1378 : CameraSettings.08_Unknown = 0
@0x0000564=1380 : CameraSettings.09_Unknown = 6
@0x0000566=1382 : CameraSettings.10_ImageSize = 65535 = 'undefined'
@0x0000568=1384 : CameraSettings.11_EasyShootMode = 1 = 'Manual'
@0x000056a=1386 : CameraSettings.12_DigitalZoom = 0 = 'none'
@0x000056c=1388 : CameraSettings.13_Contrast = 0 = 'Normal'
@0x000056e=1390 : CameraSettings.14_Saturation = 0 = 'Normal'
@0x0000570=1392 : CameraSettings.15_Sharpness = 32767 = 'undefined'
@0x0000572=1394 : CameraSettings.16_ISO = 0x7fff/32767 = 'undefined'
@0x0000574=1396 : CameraSettings.17_MeteringMode = 3 = 'Evaluative'
@0x0000576=1398 : CameraSettings.18_FocusType = 2 = 'Auto2'
@0x0000578=1400 : CameraSettings.19_AFPositionSelected = 0 = 'undefined'
@0x000057a=1402 : CameraSettings.20_ExposureMode = 2 = 'Shutter priority AE'
@0x000057c=1404 : CameraSettings.21_Unknown = 65535
@0x000057e=1406 : CameraSettings.22_LensType = 48
@0x0000580=1408 : CameraSettings.23_FocalLength_long = 55 units
@0x0000582=1410 : CameraSettings.24_FocalLength_short = 18 units
@0x0000584=1412 : CameraSettings.25_FocalUnits* = 1 unit per mm
@0x0000586=1414 : CameraSettings.26_Unknown = 160
@0x0000588=1416 : CameraSettings.27_Unknown = 332
@0x000058a=1418 : CameraSettings.28_FlashActivity = 0 = 'did not fire'
@0x000058c=1420 : CameraSettings.29_FlashDetails = 0 = ''
@0x000058e=1422 : CameraSettings.30_Unknown = 0
@0x0000590=1424 : CameraSettings.31_Unknown = 0
@0x0000592=1426 : CameraSettings.32_FocusMode = 65535 = 'undefined'
@0x0000594=1428 : CameraSettings.33_undefined = 0xffff/65535
@0x0000596=1430 : CameraSettings.34_undefined = 0xffff/65535
@0x0000598=1432 : CameraSettings.35_undefined = 0/0
@0x000059a=1434 : CameraSettings.36_ZoomedResValue** = 0
@0x000059c=1436 : CameraSettings.37_ZoomedResBase** = 0
@0x000059e=1438 : CameraSettings.38_undefined = 0/0
@0x00005a0=1440 : CameraSettings.39_undefined = 0xffff/65535
@0x00005a2=1442 : CameraSettings.40_undefined = 0xffff/65535
@0x00005a4=1444 : CameraSettings.41_undefined = 0/0
@0x00005a6=1446 : CameraSettings.42_ColorTone** = 0/0
@0x00005a8=1448 : CameraSettings.43_undefined = 0x7fff/32767
@0x00005aa=1450 : CameraSettings.44_undefined = 0xffff/65535
@0x00005ac=1452 : CameraSettings.45_undefined = 0xffff/65535
@0x00005ae=1454 : CameraSettings.46_undefined = 0/0
@0x00005b0=1456 : CameraSettings.47_undefined = 0/0
@0x00005b2=1458 : CameraSettings.48_undefined = 0xffff/65535
@0x00005b3=1459 :
@0x00005b4=1460 : 4 entries length 8
@0x00005b4=1460 : FocusInfo.00_unknown = 0
@0x00005b6=1462 : FocusInfo.01_FocalLength = 55
@0x00005b8=1464 : FocusInfo.02_FocalPlaneXSize = 8902
@0x00005ba=1466 : FocusInfo.03_FocalPlaneYSize = 19690
@0x00005bb=1467 :
@0x00005bc=1468 : TAG_0X0003 = 0,0,0,0
@0x00005c4=1476 : 33 entries length 68
@0x00005c6=1478 : ShotInfo.01_Unknown = 00
@0x00005c8=1480 : ShotInfo.02_ISO = 160
@0x00005ca=1482 : ShotInfo.03_Unknown = 0x3048
@0x00005cc=1484 : ShotInfo.04_Unknown = 0xa0160
@0x00005ce=1486 : ShotInfo.05_Unknown = 0xac172
@0x00005d0=1488 : ShotInfo.06_ExposureCompensation** = 0
@0x00005d2=1490 : ShotInfo.07_WhiteBalance = 0 = 'Auto'
@0x00005d4=1492 : ShotInfo.08_Unknown = 0x33
@0x00005d6=1494 : ShotInfo.09_SequenceNumber = 0
@0x00005d8=1496 : ShotInfo.10_Unknown = 0x88
@0x00005da=1498 : ShotInfo.11_Unknown = 0x88
@0x00005dc=1500 : ShotInfo.12_Unknown = 0x97151
@0x00005de=1502 : ShotInfo.13_Unknown = 00
@0x00005e0=1504 : ShotInfo.14_AFPositionUsed = 0 = 'MF'
@0x00005e2=1506 : ShotInfo.15_FlashBias = 0 = '0 EV'
@0x00005e4=1508 : ShotInfo.16_AutoExposureBracketing** = 0
@0x00005e6=1510 : ShotInfo.17_AEBracketValue** = 0
@0x00005e8=1512 : ShotInfo.18_Unknown = 0x11
@0x00005ea=1514 : ShotInfo.19_FocusDistanceMax** = 0 mm
@0x00005ec=1516 : ShotInfo.20_FocusDistanceMin** = 0 mm
@0x00005ee=1518 : ShotInfo.21_ApertureValue** = 160
@0x00005f0=1520 : ShotInfo.22_ExposureTime** = 172
@0x00005f2=1522 : ShotInfo.23_Undefined = 0x64100
@0x00005f4=1524 : ShotInfo.24_BulbDuration** = 0
@0x00005f6=1526 : ShotInfo.25_Undefined = 00
@0x00005f8=1528 : ShotInfo.26_Undefined = 0xf8248
@0x00005fa=1530 : ShotInfo.27_AutoRotate** = 65535
@0x00005fc=1532 : ShotInfo.28_Undefined = 0xffff65535
@0x00005fe=1534 : ShotInfo.29_SelfTimer2** = 65535
@0x0000600=1536 : ShotInfo.30_Undefined = 0xffff65535
@0x0000602=1538 : ShotInfo.31_Undefined = 00
@0x0000604=1540 : ShotInfo.32_Undefined = 00
@0x0000606=1542 : ShotInfo.33_Undefined = 00
@0x0000607=1543 :
@0x0000608=1544 : ImageType = 'Canon EOS 550D\0'
@0x0000628=1576 : FirmwareVersion = 'Firmware Version 1.0.9\0\0'
@0x0000640=1600 : OwnerName = '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0'
@0x0000660=1632 : TAG_0X000D : length 1536 # UNDEFINED (not dumped, use -U)
@0x0000c60=3168 : TAG_0X0013 = 0,159,7,112
@0x0000c68=3176 : TAG_0X0026 = 96,2,9,9,5184 ... ,65535 (48) -> @3270
@0x0000f28=3880 : FileInfo** = 52,0,0,0,0 ... ,0 (26) -> @3930
@0x0000f5c=3932 : TAG_0X0095 = 'EF-S18-55mm f/3.5-5.6 IS\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0'
@0x0000fa2=4002 : TAG_0X0096 = 'VC0645382\0\0\0\0\0\0\0'
@0x0000fb2=4018 : TAG_0X0097 : length 1024 # UNDEFINED (not dumped, use -U)
@0x00013b2=5042 : TAG_0X0098 = 0,0,0,0
@0x00013ba=5050 : TAG_0X0099 = 200,4,1 ... ,0 (50) -> @5246
@0x0001482=5250 : ColorInfo** = 28,0,1,0,0 ... ,0 (14) -> @5276
@0x000149e=5278 : TAG_0X00AA = 12,466,1024,1024,722,1
@0x00014aa=5290 : 16 entries length 34
@0x00014ac=5292 : SensorInfo.01_SensorWidth = 5344
@0x00014ae=5294 : SensorInfo.02_SensorHeight = 3516
@0x00014b0=5296 : SensorInfo.03_Unknown = 1
@0x00014b2=5298 : SensorInfo.04_Unknown = 1
@0x00014b4=5300 : SensorInfo.05_SensorLeftBorder = 152
@0x00014b6=5302 : SensorInfo.06_SensorTopBorder = 56
@0x00014b8=5304 : SensorInfo.07_SensorRightBorder = 5335
@0x00014ba=5306 : SensorInfo.08_SensorBottomBorder = 3511
@0x00014bc=5308 : SensorInfo.09_Undefined = 00
@0x00014be=5310 : SensorInfo.10_Undefined = 00
@0x00014c0=5312 : SensorInfo.11_Undefined = 00
@0x00014c2=5314 : SensorInfo.12_Undefined = 00
@0x00014c4=5316 : SensorInfo.13_Undefined = 00
@0x00014c6=5318 : SensorInfo.14_Undefined = 00
@0x00014c8=5320 : SensorInfo.15_Undefined = 00
@0x00014ca=5322 : SensorInfo.16_Undefined = 00
@0x00014cb=5323 :
@0x00014cc=5324 : TAG_0X4001 = 7,766,1024,1024,376 ... ,0 (1338) -> @7998
@0x0001f40=8000 : TAG_0X4002 = 20840,8197,0,0,0 ... ,0 (10420) -> @28838
@0x00070a8=28840 : TAG_0X4005 : length 16448 # UNDEFINED (not dumped, use -U)
@0x000b0e8=45288 : TAG_0X4008 = 129,129,129
@0x000b0ee=45294 : TAG_0X4009 = 0,0,0
@0x000b0f4=45300 : TAG_0X4010 = '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0'
@0x000b114=45332 : TAG_0X4011 : length 252 # UNDEFINED (not dumped, use -U)
@0x000b210=45584 : TAG_0X4012 = '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0'
@0x000b230=45616 : TAG_0X4015 : length 116 # UNDEFINED (not dumped, use -U)
@0x000b2a4=45732 : TAG_0X4016 = 24,0,1 ... ,0 (6) -> @45752
@0x000b2bc=45756 : TAG_0X4017 = 262918,1409286145
@0x000b2c4=45764 : TAG_0X4018 = 12,1,3
@0x000b2d0=45776 : TAG_0X4019 : length 30 # UNDEFINED (not dumped, use -U)
*0x000b2ed=45805 : ---- End of values before end of MakerNote
0x000b2ee=45806 : 49 49 2a 00 78 03 00 00 |II*.x... |
@0x000b2f5=45813 :
@0x000b2f6=45814 : UserComment = length 256+8: (CC=undefined) = (256 nulls)
@0x000b3fe=46078 : 2 entries starting at file offset 0xb400=46080
@0x000b400=46080 : <0x0001= 1> InteroperabilityIndex [2 =ASCII 4] = 'R98'
@0x000b40c=46092 : <0x0002= 2> InteroperabilityVersion [7 =UNDEFINED 4] = '0100'
@0x000b418=46104 : **** next IFD offset 0
@0x000b41b=46107 :
@0x000b41c=46108 : FocalPlaneXResolution = 5728.18
@0x000b424=46116 : FocalPlaneYResolution = 5808.4
-0x000b42b=46123 :
@0x000b42b=46123 :
>0x000cc14=52244 : <=====> Start of JPEG data for IFD 0, data length 960304
>0x000cc14=52244 :
>0x000cc16=52246 : length 132
>0x000cc9c=52380 : length 17, 8 bits/sample, components=3, width=5184, height=3456
>0x000ccaf=52399 : length 418 table class = 0 table id = 0
>0x000ce53=52819 : length 12 start of JPEG data, 3 components 17915904 pixels
>0x00f7342=1012546 : JPEG length 960304
>0x00f7343=1012547 : End of JPEG data ====
@0x000b42c=46124 : 2 entries starting at file offset 0xb42e=46126
@0x000b42e=46126 : <0x0201= 513> JPEGInterchangeFormat [4 =LONG 1] = @0xb564=46436
@0x000b43a=46138 : <0x0202= 514> JPEGInterchangeFormatLength [4 =LONG 1] = 5807
@0x000b446=46150 : **** next IFD offset 46154(+ 0 = 0xb44a/46154)
>0x000b564=46436 : #### Start of JPEG thumbnail data for IFD 1, length 5807 ####
>0x000b564=46436 :
>0x000b566=46438 : length 132
>0x000b5ec=46572 : length 17, 8 bits/sample, components=3, width=160, height=120
>0x000b5ff=46591 : length 418 table class = 0 table id = 0
>0x000b7a3=47011 : length 12 start of JPEG data, 3 components 19200 pixels
>0x000cc11=52241 : JPEG length 5807
>0x000cc12=52242 : #### End of JPEG thumbnail data for IFD 1, length 5807 ####
@0x000b449=46153 :
@0x000b44a=46154 : 13 entries starting at file offset 0xb44c=46156
@0x000b44c=46156 : <0x0100= 256> ImageWidth [3 =SHORT 1] = 668
@0x000b458=46168 : <0x0101= 257> ImageLength [3 =SHORT 1] = 432
@0x000b464=46180 : <0x0102= 258> BitsPerSample [3 =SHORT 3] = @0xb4ec=46316
@0x000b470=46192 : <0x0103= 259> Compression [3 =SHORT 1] = 1 = 'uncompressed'
@0x000b47c=46204 : <0x0106= 262> PhotometricInterpretation [3 =SHORT 1] = 2 = 'RGB'
@0x000b488=46216 : <0x0111= 273> StripOffsets [4 =LONG 1] = @1012548
@0x000b494=46228 : <0x0115= 277> SamplesPerPixel [3 =SHORT 1] = 3
@0x000b4a0=46240 : <0x0116= 278> RowsPerStrip [3 =SHORT 1] = 432
@0x000b4ac=46252 : <0x0117= 279> StripByteCounts [4 =LONG 1] = 1731456
@0x000b4b8=46264 : <0x011c= 284> PlanarConfiguration [3 =SHORT 1] = 1 = 'chunky/contig'
@0x000b4c4=46276 : <0xc5d9=50649> TAG_CR2c5d9 [4 =LONG 1] = 2
@0x000b4d0=46288 : <0xc6c5=50885> TAG_0xc6c5 [4 =LONG 1] = 3
@0x000b4dc=46300 : <0xc6dc=50908> TAG_0xc6dc [4 =LONG 4] = @0xb4f2=46322
@0x000b4e8=46312 : **** next IFD offset 46338(+ 0 = 0xb502/46338)
@0x000b4ec=46316 : ============= VALUES, IFD 2 ============
@0x000b4ec=46316 : BitsPerSample = 16,16,16
@0x000b4f2=46322 : TAG_0xc6dc = 650,432,19,0
@0x000b501=46337 :
>0x00f7344=1012548 : <=-=-=> Start of TIFF RGB uncompressed image data for IFD 2, data length 1731456
0x00f7344=1012548 : 02 08 01 08 01 08 02 08 ff 07 ff 07 ff 07 00 08 |................|
0x00f7354=1012564 : 00 08 04 08 00 08 02 08 fe 07 01 08 00 08 ff 07 |................|
0x00f7364=1012580 : 01 08 01 08 ff 07 ff 07 ff 07 03 08 02 08 fe 07 |................|
0x00f7374=1012596 : 02 08 00 08 00 08 02 08 ff 07 00 08 08 08 01 08 |................| etc...
>0x029dec3=2744003 : End of image data
@0x000b502=46338 : 7 entries starting at file offset 0xb504=46340
@0x000b504=46340 : <0x0103= 259> Compression [3 =SHORT 1] = 6 = 'Exif/old JPEG'
@0x000b510=46352 : <0x0111= 273> StripOffsets [4 =LONG 1] = @2749252
@0x000b51c=46364 : <0x0117= 279> StripByteCounts [4 =LONG 1] = 17111015
@0x000b528=46376 : <0xc5d8=50648> TAG_CR2c5d8 [4 =LONG 1] = 1
@0x000b534=46388 : <0xc5e0=50656> TAG_CR2c5e0 [4 =LONG 1] = 3
@0x000b540=46400 : <0xc640=50752> TAG_CR2c640 [3 =SHORT 3] = @0xb55c=46428
@0x000b54c=46412 : <0xc6c5=50885> TAG_0xc6c5 [4 =LONG 1] = 1
@0x000b558=46424 : **** next IFD offset 0
@0x000b55c=46428 : ============= VALUES, IFD 3 ============
@0x000b55c=46428 : TAG_CR2c640 = 2,1728,1888
@0x000b561=46433 :
@0x029f344=2749252 : <=====> Start of JPEG data for IFD 3, data length 17111015
@0x029f344=2749252 :
@0x029f346=2749254 : length 66 table class = 0 table id = 0
@0x029f38a=2749322 : length 20, 14 bits/sample, components=4, width=1336, height=3516
@0x029f3a0=2749344 : length 14 start of JPEG data, 4 components 4697376 pixels
@0x12f0b29=19860265: JPEG length 17111015
@0x12f0b2a=19860266: End of JPEG data ====
-0x12f0b2a=19860266: END OF FILE
@0x000cc14=52244 : Start of JPEG baseline DCT compressed image [5184x3456] length 960304 (IFD 0)
-0x00f7343=1012547 : End of JPEG image data
@0x000b564=46436 : Start of JPEG baseline DCT compressed reduced-resolution image [160x120] length 5807 (IFD 1)
-0x000cc12=52242 : End of JPEG reduced-resolution image data
@0x00f7344=1012548 : Start of TIFF RGB uncompressed reduced-resolution image [668x432] length 1731456 (IFD 2)
-0x029dec3=2744003 : End of TIFF reduced-resolution image data
@0x029f344=2749252 : Start of JPEG lossless seq Huffman reduced-resolution image [1336x3516] length 17111015 (IFD 3)
-0x12f0b2a=19860266: End of JPEG reduced-resolution image data
Number of images = 4
File Format = TIFF/EXIF [CR2] # with MakerNote (Canon [1])
Linear domain is very clear in noise characteristics and so. If you work in compressed domain than noise stats becomes non-symmetric.
Sony A7 definitely has masked pixels on right side. You may use, RawDigger (for example) to examine the file and see black area clearly.
Thanks Alex. Could you please explain to me why it is better to manipulate raw data in the linear domain? I have seen this mentioned before, but the reasoning behind it isn't clear to me. The number of individual brightness levels available is determined by the bit depth of the camera, so any interpolated values in the linear domain (calculated during image manipulation) will have to be remapped to this same set of brightness levels. What am I missing?
Another question: I have noticed that some images (e.g. ARW images from the Sony A7) are reported as having top_margin=0 and left_margin=0, but width does not equal raw_width. Is this just a case of masked pixels being on the *right* of the image (in my experience they are usually on the left)? The DNG SDK reports the width as the full raw_width, and on examination the DNG converted images fill this extra image area with repeated values of the pixel at position(x,width), i.e. it duplicates the last pixel in the row to fill the image area between width and raw_width.
Any comments would be greatly appreciated!
Inverted tone curve after unpack() should work fine too.
BTW, it is better to manipulate raw data in 'linear domain' (so, after real tone curve applied) and apply inverted curve before packing your data back (to DNG).
Thanks Alex, I used your suggested code snippet. I'm also testing for application of the tone curve during unpack(), and applying the inverse tone curve as necessary at that point.
LibRaw extracts artist from Canon's makernotes tag #9 and IFD tag #315. Could you please provide EXIF data dump of your file?
Case B: initialize the image variable:
Without initialization, you pass random garbage to dcraw_clear_mem()
(A) = CORRECT WAY =
libraw_processed_image_t* image;
image = iProcessor.dcraw_make_mem_image(&rc);
LibRaw::dcraw_clear_mem(image);
(B) = ERROR =
libraw_processed_image_t* image;
LibRaw::dcraw_clear_mem(image);
How can i know if i have used the (A) or (B) approach to initialize image?
Followup: the only format with linearization data applied after unpack() phase (within raw2image or dcraw_process()) is PhaseOne files (both compressed and uncompressed).
The tone curve for DNG files (Linearization Curve in DNG specs terms) is always applied on unpack() phase. There are no 'documented' options to turn this off (because RAW application always want to see linearized data), but you may use this trick:
This trick will work with data format with separate curve in metadata (all DNG files; Sony ARW2 format), but will not work with files where curve is calculated on unpack() phase (e.g. Nikon lossy NEF files).
Also please note, that raw2image() will skip 'masked pixels' (pixels outside of ActiveArea in DNG terms). To avoid this you may do the same trick: set imgdata.sizes.width and iwidth to raw_width and zero left_margin (and same with height, iheight and top_margin).
Thanks! Sounds great.
Followup: ABI/API changes will not break existing programs source compatibility. It is usual additional features we need. But you may need to recompile your code to adopt these changes.
We use current development version in our published apps (RawDigger, FastRawViewer).
It is stable in terms of program stability.
It is not stable in ABI/API terms, because internals may change without notifications
Do you consider the current rev to be stable?
I do not understand your question.
If dcraw_make_mem_image() fails, it will return NULL.
Also, if you pass NULL to dcraw_clear_mem, it will do nothing.
So, what is the *real* problem?
No exact date for 'release'. You may use development trunk from GitHub: https://github.com/LibRaw/LibRaw
Thank you for the report. Flash data is parsed only for old Canon (CRW) files. To be added to EXIF parser section on the next update.
Thanks for the answer.
I`ve found that RawProcessor.imgdata.color.flash_used is always 0 even on photos when I use flash. Check this file https://dl.dropboxusercontent.com/u/7392238/Raw%20camera%20samples/P7312...
Canon CR2 files are lossless JPEG (Huffman) compressed. If raw data is damaged, you'll unable to correctly decompress data up to nex (L)JPEG sync tag.
Pages