Looks like we can safely ignore 'not enough data problem' for last line(s) of last block (it contains masked pixels, not needed for data display).
It may be Firmware issue(??)
Here is the patch (for last line of last block only):
diff --git a/src/libraw_xtrans_compressed.cpp b/src/libraw_xtrans_compressed.cpp
index b6555d3..ec6dbb0 100644
--- a/src/libraw_xtrans_compressed.cpp
+++ b/src/libraw_xtrans_compressed.cpp
@@ -46,6 +46,7 @@ struct xtrans_block {
unsigned max_read_size; // Amount of data to be read
int cur_buf_size; // buffer size
uchar *cur_buf; // currently read block
+ bool lastlast;
LibRaw_abstract_datastream *input;
struct int_pair grad_even[3][41]; // tables of gradients
struct int_pair grad_odd[3][41];
@@ -143,7 +144,7 @@ static inline void fuji_fill_buffer(struct xtrans_block *info)
#ifndef LIBRAW_USE_OPENMP
info->input->unlock();
#endif
- if(info->cur_buf_size<1) // nothing read
+ if(info->cur_buf_size<1 && info->max_read_size && !info->lastlast) // nothing read
throw LIBRAW_EXCEPTION_IO_EOF;
info->max_read_size -= info->cur_buf_size;
@@ -159,6 +160,7 @@ void LibRaw::init_xtrans_block(struct xtrans_block* info, const struct xtrans_pa
INT64 fsize = libraw_internal_data.internal_data.input->size();
info->max_read_size = _min(unsigned(fsize-raw_offset),dsize+16); // Data size may be incorrect?
+ info->lastlast = false;
info->input = libraw_internal_data.internal_data.input;
info->linebuf[_R0] = info->linealloc;
for(int i = _R1; i<=_B4;i++)
@@ -668,6 +670,7 @@ void LibRaw::xtrans_decode_strip(const struct xtrans_params* info_common, int cu
ztable[3]= {{_R2,3},{_G2,6},{_B2,3}};
for (cur_line = 0; cur_line < libraw_internal_data.unpacker_data.fuji_total_lines; cur_line++)
{
+ info.lastlast = (cur_block == libraw_internal_data.unpacker_data.fuji_total_blocks-1) && (cur_line ==libraw_internal_data.unpacker_data.fuji_total_lines-1);
xtrans_decode_block(&info, info_common, cur_line);
// copy data from line buffers and advance
I have only one sample. But the photographer that sent me that sample showed me a case where out of 36 images he had, 4 of them appeared corrupted. I'll try to get more samples.
Thank you for sharing the sample.
In this sample
data offset (of raw data) 805904
data size 25251744
sum is 26057648
but file size is 26057632, so 16 bytes missing somewhere
Do you have only one sample, or some camera produces many such files?
If only one, this looks like some data error (camera, card, transfer). If this is systematic error it should be handled in software (e.g. ignore last bytes without error raise)
1) We want to drop this license, so please select from CDDL/LGPL2 (btw, LibRaw license do not cover updates, so in general you need to re-sign when you go from LibRaw 0.17 to 0.18).
2-3) CDDL is very permissive, so yes for both questions
Thanks for claryfying, Alex :). I have a few follow-up questions though.
1.) Is it possible to use the (own) libraw-license for 0.17?
2.) Which licenses allow static linking without disclosing the complete source?
3.) Which licenses allow including of the source-code without disclosing the complete source?
LibRaw is licensed in triple-licensing form:
- LGPL2
- or CDDL
- or LibRaw own license (to be dropped in 0.18-release, because all signed agreements has ended).
You (app developer) is free to choose license (from LGPL/CDDL if you wish to use LibRaw 0.18 when we'll release it) that suits you best.
In short:
LGPL2 allows use in Commercial App if you use Library (LibRaw in this case) as shared-library and has published in open-source (under LGPL2) any changes you made to Library source. If you use LibRaw unmodified, you only need to use it in shared-library form (.DLL/.so/.dylib)
CDDL is more permissive, you may just use the library and do not forced to publish your changes to it.
Attribution is needed in both cases. Written/signed form is not needed.
We publish 'public snapshot' twice a year and 'release' once a year or so.
5D4 (and dozens of other cameras) to be supported in next snapshot (in October or so). Current snapshot can read 5D4 files, but you'll need to set black level manually.
You say that 45deg rotation allows to use only integer math for the scale? What is the function in dcraw (or libraw) doing that ?
fuji_rotate() is only use in the case of debayered data to go back to normal picture.
What I mean, is when I run this:
dcraw -v -T -4 -o 0 -D -t 0 -k 0 -H 1 image.CR2
I do not have any black frames, and this is the behavior I'm trying to reproduce. I've had this with the use of width and height (but RAF files was buggy).
Correct me if I'm not right, but it was not working because in RAF file you have left and right frame. But in others, like CR2 you have left and top (If I well remember).
So how we remove frames if they aren't in the same place ?
Thanks for the patch, Alex. I'm waiting for more images so we can properly test the patch. I'll send another update when I've done that.
diff --git a/src/libraw_xtrans_compressed.cpp b/src/libraw_xtrans_compressed.cpp index b6555d3..ec6dbb0 100644 --- a/src/libraw_xtrans_compressed.cpp +++ b/src/libraw_xtrans_compressed.cpp @@ -46,6 +46,7 @@ struct xtrans_block { unsigned max_read_size; // Amount of data to be read int cur_buf_size; // buffer size uchar *cur_buf; // currently read block + bool lastlast; LibRaw_abstract_datastream *input; struct int_pair grad_even[3][41]; // tables of gradients struct int_pair grad_odd[3][41]; @@ -143,7 +144,7 @@ static inline void fuji_fill_buffer(struct xtrans_block *info) #ifndef LIBRAW_USE_OPENMP info->input->unlock(); #endif - if(info->cur_buf_size<1) // nothing read + if(info->cur_buf_size<1 && info->max_read_size && !info->lastlast) // nothing read throw LIBRAW_EXCEPTION_IO_EOF; info->max_read_size -= info->cur_buf_size; @@ -159,6 +160,7 @@ void LibRaw::init_xtrans_block(struct xtrans_block* info, const struct xtrans_pa INT64 fsize = libraw_internal_data.internal_data.input->size(); info->max_read_size = _min(unsigned(fsize-raw_offset),dsize+16); // Data size may be incorrect? + info->lastlast = false; info->input = libraw_internal_data.internal_data.input; info->linebuf[_R0] = info->linealloc; for(int i = _R1; i<=_B4;i++) @@ -668,6 +670,7 @@ void LibRaw::xtrans_decode_strip(const struct xtrans_params* info_common, int cu ztable[3]= {{_R2,3},{_G2,6},{_B2,3}}; for (cur_line = 0; cur_line < libraw_internal_data.unpacker_data.fuji_total_lines; cur_line++) { + info.lastlast = (cur_block == libraw_internal_data.unpacker_data.fuji_total_blocks-1) && (cur_line ==libraw_internal_data.unpacker_data.fuji_total_lines-1); xtrans_decode_block(&info, info_common, cur_line); // copy data from line buffers and advanceBTW, ignoring this error for the last bytes of last line looks OK.
I've digged this data in depth: decoder really wants to see these extra bytes for the last line, EOF error looks correct for this data.
I have only one sample. But the photographer that sent me that sample showed me a case where out of 36 images he had, 4 of them appeared corrupted. I'll try to get more samples.
Thank you for sharing the sample.
In this sample
data offset (of raw data) 805904
data size 25251744
sum is 26057648
but file size is 26057632, so 16 bytes missing somewhere
Do you have only one sample, or some camera produces many such files?
If only one, this looks like some data error (camera, card, transfer). If this is systematic error it should be handled in software (e.g. ignore last bytes without error raise)
CDDL will be the weapon of my choice then. Thanks for your instant support. :)
1) We want to drop this license, so please select from CDDL/LGPL2 (btw, LibRaw license do not cover updates, so in general you need to re-sign when you go from LibRaw 0.17 to 0.18).
2-3) CDDL is very permissive, so yes for both questions
Thanks for claryfying, Alex :). I have a few follow-up questions though.
1.) Is it possible to use the (own) libraw-license for 0.17?
2.) Which licenses allow static linking without disclosing the complete source?
3.) Which licenses allow including of the source-code without disclosing the complete source?
Cheers, Brian
LibRaw is licensed in triple-licensing form:
- LGPL2
- or CDDL
- or LibRaw own license (to be dropped in 0.18-release, because all signed agreements has ended).
You (app developer) is free to choose license (from LGPL/CDDL if you wish to use LibRaw 0.18 when we'll release it) that suits you best.
In short:
LGPL2 allows use in Commercial App if you use Library (LibRaw in this case) as shared-library and has published in open-source (under LGPL2) any changes you made to Library source. If you use LibRaw unmodified, you only need to use it in shared-library form (.DLL/.so/.dylib)
CDDL is more permissive, you may just use the library and do not forced to publish your changes to it.
Attribution is needed in both cases. Written/signed form is not needed.
For EOS 6D different color profile is used because dcraw's is not good enough.
Also, LibRaw has changed auto-brighness method, so disable auto-brightness for comparison
Thanks, I look forward to the next snapshot.
We publish 'public snapshot' twice a year and 'release' once a year or so.
5D4 (and dozens of other cameras) to be supported in next snapshot (in October or so). Current snapshot can read 5D4 files, but you'll need to set black level manually.
Thank you.
Just for knowledge.
How does it work without Bayer filter to get color ?
EDIT: nevermind, I think I found on wikipedia.
Thanks again for your kindness
Yes.
There are 3 pointers in rawdata (raw_data, color3_data, color4_data and only one is not NULL after unpack)
Damn, no C API :).
I have to take a look in dcraw.c to see how it is done .... But this code is no easily readable :).
Best regards
Thanks !! I will take a look.
Regards
Sorry for misinformation.
There is LibRaw::copy_fuji_uncropped() that do the trick.
If you do not use image[][4] array, you may copy-paste it and adopt to your needs.
There is no separate LibRaw call to rotate fuji data.
Rotation is performed within LibRaw::raw2image_ex call (which copies data from raw_image array to image[][4] array)
Hello Alex.
I've correctly opened RAF file without frames thanks to you.
Now I have to apply the scale to transform my picture from this http://hpics.li/ffe7b4f to this http://hpics.li/f653896.
You say that 45deg rotation allows to use only integer math for the scale? What is the function in dcraw (or libraw) doing that ?
fuji_rotate() is only use in the case of debayered data to go back to normal picture.
Cheers
I would like to know that as well, the only java solution so far was found in the jAlbum binaries (they seem to have written their own port).
I think I should go ahead because I did understand.
Thanks for your help with Fuji
What I mean, is when I run this:
dcraw -v -T -4 -o 0 -D -t 0 -k 0 -H 1 image.CR2
I do not have any black frames, and this is the behavior I'm trying to reproduce. I've had this with the use of width and height (but RAF files was buggy).
By the way, many thanks for helping me!!
Sorry, cannot understand your question.
Initially your question was about 'dcraw -D' so document mode, so full access to raw with black frame.
Correct me if I'm not right, but it was not working because in RAF file you have left and right frame. But in others, like CR2 you have left and top (If I well remember).
So how we remove frames if they aren't in the same place ?
Pages