RAW vs. DNG file formats

[California L33]

I've been taking CRW (RAW) photos in my Canon A650 (using CHDK) and converting them to DNG for Photoshop CS2. When I read the general information for DNG it says that it preserves the RAW data but stores it in a universal RAW format. One thing I noticed is that the DNG files are smaller than the CRW files- not much, just a couple of megs, and it varies. The CRW files are all the same 15 point something megs. The DNG files vary from about 11 to 14 megs. Does DNG use some kind of compression (it certainly seems to)? If so, is it a lossless compression? Or is DNG not saving some of the CRW data? It's no big deal to archive them both to DVD, but it does take more than twice as much space.

[Canuck935]

.DNG files are definitely smaller than their RAW counterparts, making a strong case for making the switch. It is definitely compressing the data, but AFAIK it is lossless.

[Ron Kruger]

Why are you saving images in DNG, instead of TIFF or even JPEG?
This is an accronym I've only seen a few times. What is DNG best suited for?

[SmartWombat]

For me, the issue is that it converts from RAW to DNG.
So in the process it must change the information.
I prefer not to do that, but load the RAW file direct into Lightroom and process is it there.

[California L33]

Why are you saving images in DNG, instead of TIFF or even JPEG?
This is an accronym I've only seen a few times. What is DNG best suited for?

The reason not to save in Jpeg is obvious. It's only an 8 bit file format, and it has lossy compression. Tiff can save in 16 bit, but it throws away all the data the camera records, such as white balance and exposure. (Just did a little research, and apparently, TIFF is capable of storing metadata, but isn't often set up to do so).

DNG was created by Adobe as a universal RAW format. (It is apparently TIFF format with standardized places for storing camera metadata). The problem with RAW is that it's not a universal format. Every manufacturer saves RAW data in it's on unique file format that can vary even model to model. In theory when you convert to DNG it saves all the camera's RAW data without being make/model specific. This allows software makers to concentrate on tweaking their software rather than writing new RAW compatibility modules every time a camera is introduced. So you only need one converter for each new RAW format. You convert RAW to DNG then any software that supports DNG can use it.

I posed the original question when I noticed that the DNG files were smaller (by just a bit- the original files were always 15 megs, the new ones between 11-14 megs) than the original CRW (RAW) files. It appears that Adobe has used a lossless compression. (It would have been idiotic to use a lossy compression like Jpeg). I was just trying to confirm this. I just did a little more poking around the web and think I've confirmed this. Here's a good source-

http://www.barrypearson.co.uk/articles/dng/benefits.htm

which led me here-

http://www.digitalpreservation.gov/f...ferences.shtml

[Stevvie]

you can now set the camera to save in DNG instead of CRW mode with the latest CHDK from http://mighty-hoernsche.de/

[OldClicker]

I realize that this is a couple of weeks old, but DNG is nothing like tif. Tif is an image file. DNG is a RAW file (not an image) containing only the color (1 of 3) and the intensity (probably 12-bit) of each pixel. It requires a conversion program to render it into an image. - TF

[SmartWombat]

Tif is an image file. DNG is a RAW file

Well of course I disagree
Technically, both are just data files, nothing more important than that.

TIFF files are more widely adopted than DNG - they've been around longer.
Adobe want to push DNG as a standard (OK, everyone use the Adobe standard) image format.

But both are image files, in that both store an image, both are lossless compressed.

I reckon it's all in the marketing

Canon's RAW format is a kind of TIF file, internally, with proprietary add-ons that make them non standard.
Before Thumbs+ supported the latest .CR2 format, I was getting TIF conversion errors when loading the Canon raw files.

It requires a conversion program to render it into an image.

Every file requires conversion to display it, or to print it.
Conversion from a packed, compressed representation (maybe with layers to be combined) into a flattened rendering of the pixels on the screen, or on the printer.

I defy you to display a .TIFF file without a program to turn it from a file full of pixel colours and intensities and render it as an image.


Oh and even a JPEG file can contain four colours, a friend sent me a CMYK format file that was going to the printer. My program couldn't display it properly because it just assumed all JPG files must be RGB - doh!
There are versions of JPEG that are seldom used that store more than 8 bits/colour for medical imaging - but it's been largely replaced by the DICOM format.

I assume that DNG just like CR2 and JPG contains more than the image data; it probably also stores the camera metadata on exposure, camera model, lens information, metering, flash - all the stuff that usually goes in the EXIF information in a JPEG.
Plus for professional uses I wouldn't be surprised if it can stores the IPTC information too, for caption, copyright, subject, location, publisher, etc.

I know that the Canon raw format includes a low resolution JPEG version of the image, that's how you can quickly review the picture, it means you don't need to read the whole file to show the picture, at least an initial preview while at the same time software can unpack and draws the full resolution image.

[OldClicker]

Well of course I disagree
Technically, both are just data files, nothing more important than that.

TIFF files are more widely adopted than DNG - they've been around longer.
Adobe want to push DNG as a standard (OK, everyone use the Adobe standard) image format.

But both are image files, in that both store an image, both are lossless compressed.

I reckon it's all in the marketing

Canon's RAW format is a kind of TIF file, internally, with proprietary add-ons that make them non standard.
Before Thumbs+ supported the latest .CR2 format, I was getting TIF conversion errors when loading the Canon raw files.



Every file requires conversion to display it, or to print it.
Conversion from a packed, compressed representation (maybe with layers to be combined) into a flattened rendering of the pixels on the screen, or on the printer.

I defy you to display a .TIFF file without a program to turn it from a file full of pixel colours and intensities and render it as an image.


Oh and even a JPEG file can contain four colours, a friend sent me a CMYK format file that was going to the printer. My program couldn't display it properly because it just assumed all JPG files must be RGB - doh!
There are versions of JPEG that are seldom used that store more than 8 bits/colour for medical imaging - but it's been largely replaced by the DICOM format.

I assume that DNG just like CR2 and JPG contains more than the image data; it probably also stores the camera metadata on exposure, camera model, lens information, metering, flash - all the stuff that usually goes in the EXIF information in a JPEG.
Plus for professional uses I wouldn't be surprised if it can stores the IPTC information too, for caption, copyright, subject, location, publisher, etc.

I know that the Canon raw format includes a low resolution JPEG version of the image, that's how you can quickly review the picture, it means you don't need to read the whole file to show the picture, at least an initial preview while at the same time software can unpack and draws the full resolution image.

Bottom line, they are all data files. But to me, there is there is a fundamental difference. A RAW file has (at the most) 2-bit color resolution. It only has (usually) red, green and blue - not 256+ shades of red, green and blue. A transform (usually Bayer sp??) has to be applied to get an image. Once the program decodes the pixels of an image file (jpg, tif, bmp) the pixels look like the image. The pixels decoded in a RAW file may look like this,



except that the pixels have different brightness levels. What I'm calling an 'image' file mathematically decodes the file data to get the pixels that form the image. RAW types have to THEN mathematically transform the pixel data into an image.

The file can have EXIF, jpg or anything else in along with the RAW data.

TF

[California L33]

you can now set the camera to save in DNG instead of CRW mode with the latest CHDK from http://mighty-hoernsche.de/

I just tested this 'new' version of CHDK and like it better than the previous 'All Best' version I was using. Its display is less cluttered, and it looks like it correctly monitors the memory. (The 'All Best' didn't take into account the CRW files stored on the card and falsely displayed much more memory than actually available). There is one feature on this version that just isn't very good. That's the on board RAW>DNG processing. I did just a few tests, but the DNG files it produced had strange grain in the mid-range tones when magnified. They actually looked worse than the JPG files. The CRW files were fine, and when converted with software on my computer (dng4ps) were better than the JPGs.

EDIT: And this answers my own question. The 'processing' to create the DNG obviously messes with the data (regardless of whether it's compressing it or not) since the DNGs created by CHDK has noise not in any of the other formats, so even though it's probably wise to save DNGs (converted from some other program, for compatibility) also archiving the RAW files is probably also a good idea.

[drg]

I have posted an Article about Adobe's Digital Negative (DNG) file format in the Feature Articles area here at Photography Review.

The article " DNG - Format for the Future ", is meant as an introduction to this topic. It includes many resources for further research.

I have been using DNG files as an additional archive format for over a year. There's some added functionality starting to appear from ADOBE built around the DNG specs, i.e. the Camera Profiling tools.

If you have a question not readily addressed or you find some of the documents in the Article 'dense' let me know and I'll do my best to interpret them.