Skip to content

Converting RAW to JPEG

Many photographers keep their originals in some sort of lossless RAW format instead of compressed JPEG, especially when shooting with a Digital SLR. Some mobile phones also support RAW or use HEIC/HEIF for a similar purpose. PhotoPrism aims at providing excellent support for all RAW formats, independent of camera brand and model. Please let us know when there is an issue with your specific device.

Web browsers in general can not display RAW image files. They need to be converted, which is what our import and convert commands do. You'll also find a checkbox for this step in our Web UI.

In addition, PhotoPrism now also supports TIFF, PNG, BMP and GIF files. Be aware that files in those formats often don't contain useful metadata and are typically used for screenshots, charts, graphs and icons only.

Read-only mode

Conversion is currently disabled in read-only mode because creating JPEGs from RAWs is a pretty expensive operation and it makes sense to store JPEGs right next to their RAW files so that they can be found and used again in the future. Otherwise it might not be possible to match the files again and you don't want to put the expensively created JPEG file in a temporary directory or do all this on the fly in memory, unless you have a really fast computer. Most users want to make a backup of their files, so files that belong together should not be randomly distributed across the file system. 20 years from now, it's probably easier to open a JPEG than render a proprietary RAW file again in good quality.

However, we see the point of users who want to use PhotoPrism as a read-only photo viewer that should not create any files in their originals directory. The easiest solution is to create JPEGs manually or use another tool you trust to do this automatically.

With some effort, it might be possible to extract embedded JPEGs in acceptable quality from many RAW formats and use those in read-only mode. Alternatively, JPEGs could be created in the cache directory with all the disadvantages mentioned above, the storage will be gone one way or the other. Note that JPEGs in full-resolution are relatively large and you won't be able to find or reuse them as files there have a different naming scheme based on the hash of the original file.

Idea: Put converted images in cache/converted by default with an option to automatically move / copy them to the originals directory. Feedback welcome.

Sips

On a Mac, PhotoPrism can convert multiple files at once using Sips (pre-installed on OS X). It is not available for other operating systems, as far as we know.

Darktable

We've recently upgraded Darktable from 2.6.2 to 3.0.0. Since the old PPA is not maintained anymore, we switched to this repository that is hosted by SuSE:

https://software.opensuse.org/download.html?project=graphics:darktable:master&package=darktable

Note that PhotoPrism can only run one instance of darktable-cli because it uses a lock file. If you run it more than once, it will fail. Be prepared to wait a bit.

Using Darktable as library

We had the idea to use cgo and link directly against libdarktable.so to convert RAW images to JPEG, see darktable-dev mailing list. However this requires calling/wrapping a large number of C functions and also has other disadvantages. The idea is postponed until a clear benefit becomes visible. Running darktable-cli or any other binary that does the job (see above) seems to be the way to go.

Comparison of RAW to JPEG converters

  • darktable - popular open-source photography app and raw developer; available for Mac, Linux and Windows; supports XMP (compatible with photoshop/lightroom?)
  • Mac OS X ships with sips: sips -s format jpeg IMAGE.RAW --out IMAGE.JPG
  • Photivo - open-source photo processor; available for Mac, Linux and Windows; no XMP support?
  • RawTherapee - open-source RAW image processing app; available for Mac, Linux and Windows; no XMP support?
  • digiKam - open-source digital photo management application based on Qt (KDE); available for Mac, Linux and Windows; supports XMP (compatible with photoshop/lightroom?)
  • UFRaw - Unidentified Flying Raw is a utility to read and manipulate raw images from digital cameras
Tool Command line options Compatible OS JPG Diff* XMP support Possible settings EXIF Diff* Compatible with Raspberry (ARM64)
Darktable 1) darktable-cli IMG_0310.CR2 IMG_0310_darktable1.jpg 2) darktable-cli IMG_0310_EDITED.CR2 IMG_0310_EDITED.xmp IMG_0310_EDITED_darktable2.jpg MacOSx, Linux, Windows **** yes (but seems to be not compatible with adobe xmps) ? **** yes
Sips sips -s format jpeg IMG_0310.CR2 --out IMG_0310_sips.jpg MacOSx ***** no ? **** not available for ubuntu
Rawtherapee rawtherapee-cli -o IMG_0310_rawtherapee.jpg -c IMG_0310.CR2 MacOSx, Linux, Windows *** no ? **** yes
UFraw ufraw-batch --out-type=jpg --output=IMG_0310_ufraw.jpg IMG_0310.CR2 MacOSx, Linux, Windows * no ? ** yes
ImageMagick magick IMG_0310.CR2 IMG_0310_magick.jpg MacOSx, Linux, Windows * no ? ** yes
Digikam ? MacOSx, Linux, Windows - - - - -
Photiva ? MacOSx, Linux, Windows - - - - -
* Compared to JPG/EXIF converted from photoshop

Image Diff

The following table shows the difference between the JPEG files converted by Darktable, Sips, Rawtherapee, UFraw and ImageMagick compared to Adobe Photoshop. Red are pixel that differ from the photoshop version, white are equal pixels. In total 5 Images have been compared (full results). The diff was created using https://github.com/ewanmellor/git-diff-image.

Tool Diff (left is Photoshop)
Darktable
Sips
Rawtherapee
UFraw
ImageMagick

EXIF Diff

The following table shows the difference between the JPG files converted by the tools compared to JPG files converted by photoshop. 5 Images have been compared.

Info Photoshop Sips Darktable Raw-therapee Image Magick UFraw
Kind identical identical identical identical identical identical
Size 2.881.969 bytes (EXIF)/ 2.7M (ls -alh) 2.673.777 bytes / 2.5M 5.446.710 bytes / 5.2M always bigger 3.467.998 bytes / 3.3M 3.467.998 bytes / 3.3M 1.558.587 bytes / 1.5M always smaller
Where reference different different different different different
Created reference different different different different different
Modified reference different different different different different
Dimension identical - 5472 × 3648 5472 × 3648 different - 5494 × 3666 different - 5488 × 3662 different - 5496 × 3670 different - 5496 × 3670
Device Make refrence identical identical identical not set not set
Colour space reference identical identical identical identical identical
Colour profile Adobe RGB (1998) different - Display P3 different - sRGB different - RTv2_sRGB not set not set
Focal length reference identical identical identical not set not set
Alpha Channel reference identical identical identical not set not set
Red eye reference identical identical identical not set not set
Metering Mode reference identical identical identical not set not set
F number reference identical identical identical not set not set
Exposure program reference identical identical identical not set not set
Exposure time reference identical identical identical not set not set
Latitude reference sometimes very small differences identical identical not set not set
Longitude reference sometimes very small differences identical identical not set not set

Adobe Photoshop / Lightroom

Ideally we can convert images directly with Photoshop or Lightroom, if installed. It seems like Photoshop comes with some sort of command-line automation tool based on NodeJS: https://github.com/adobe-photoshop/generator-core

See https://photo.stackexchange.com/questions/39532/command-line-approach-to-develop-raw-images-with-adobe-xmp-sidecars

This needs further investigation. Contributions welcome!

Todo / Reading List