DNG 1.4 Parser

This tutorial describes how to build the Adobe DNG SDK on Linux.
It generates the dng_validate C++ program that can parse any DNG images, a bit like a "Hello world" for DNG image processing.

1. Adobe DNG SDK 1.4
   • XMP SDK
   • DNG SDK

Adobe DNG SDK 1.4

The goal of this blog entry is to execute the dng_validate command built from the source.
We show how to build the XMP libraries then suggest a Makefile configuration to build a DNG SDK sample app.
All the patches were generated by the diff command.

XMP SDK

In the current directory, download the XMP SDK:

wget http://download.macromedia.com/pub/developer/xmp/sdk/XMP-Toolkit-SDK-CC-201306.zip
unzip XMP-Toolkit-SDK-CC-201306.zip
mv XMP-Toolkit-SDK-CC201306 xmp_sdk

Make sure you have at least the following packages installed:

sudo apt-get install cmake libjpeg8-dev uuid-dev

The 2 next snippets download zlib and expat sources. Instructions are explained in the ReadMe.txt files in the third-party directories.

zlib

cd xmp_sdk/third-party/zlib
wget http://zlib.net/zlib-1.2.8.tar.gz
tar xzf zlib-1.2.8.tar.gz
cp zlib-1.2.8/*.h zlib-1.2.8/*.c .

expat

cd xmp_sdk/third-party/expat

Download here the source at http://sourceforge.net/projects/expat/files/expat/2.1.0/expat-2.1.0.tar.gz/download.
tar xzf expat-2.1.0.tar.gz
cp -R expat-2.1.0/lib .

For cross-platform compatibility, to be able to support both Mac & Windows, cmake is the tool selected by the SDK authors to build the source. We go through all the build errors.

cd xmp_sdk/build

Build error 1

make
...
CMake Error at shared/SharedConfig_Common.cmake:38 (if):
  if given arguments:

    "LESS" "413"

  Unknown arguments specified
...

I use gcc 4.8.2 for the build. There seems to be a XMP_VERSIONING_GCC_VERSION variable not properly set. We just delete the version check by removing line 37 till 42 in xmp_sdk/build/shared/SharedConfig_Common.cmake:

xmp_sdk/build/shared/SharedConfig_Common.cmake
36a37,41
>               # workaround for visibility problem and gcc 4.1.x
>               if(${${COMPONENT}_VERSIONING_GCC_VERSION} LESS 413)
>                       # only remove inline hidden...
>                       string(REGEX REPLACE "-fvisibility-inlines-hidden" "" CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS})
>               endif()

Build error 2

make
...
Linking CXX shared library /home/alexis/linux/dng2/xmp_sdk/public/libraries/i80386linux_x64/release/libXMPCore.so
g++: error: /usr/lib64/gcc/x86_64-redhat-linux/4.4.4//libssp.a: No such file or directory
...

The README.txt mentions a XMP_ENABLE_SECURE_SETTINGS:

xmp_sdk/build/README.txt
... He may also want to change the parameter XMP_ENABLE_SECURE_SETTINGS as per the configured gcc.
 a) If the gcc is configured with --enable-libssp (can be checked by executing gcc -v), he has to set the variable XMP_GCC_LIBPATH inside of file /build/XMP_Linux.cmake to the path containing the static lib( libssp.a).In this case he can set the variable the XMP_ENABLE_SECURE_SETTINGS on.
b) If the gcc is configured with --disable-libssp, he has to set the variable XMP_ENABLE_SECURE_SETTINGS off.

I don't have any --enable-libssp in my gcc version:

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.8.2-21' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --with-arch-32=i586 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.2 (Debian 4.8.2-21)

so I disable it in xmp_sdk/build/shared/ToolchainGCC.cmake line 37:

xmp_sdk/build/shared/ToolchainGCC.cmake
37c37
< set(XMP_ENABLE_SECURE_SETTINGS "OFF")
---
> set(XMP_ENABLE_SECURE_SETTINGS "ON")

Build error 3

/home/alexis/linux/dng2/xmp_sdk/XMPFiles/build/../../XMPFiles/source/NativeMetadataSupport/ValueObject.h:111:60: error: 'memcmp' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
    doSet = ( memcmp( mArray, buffer, numElements*sizeof(T) ) != 0 );
                                                            ^
In file included from /home/alexis/linux/dng2/xmp_sdk/XMPFiles/build/../../XMPFiles/source/FormatSupport/TIFF_Support.hpp:17:0,
                 from /home/alexis/linux/dng2/xmp_sdk/XMPFiles/build/../../XMPFiles/source/FormatSupport/ReconcileLegacy.hpp:15,
                 from /home/alexis/linux/dng2/xmp_sdk/XMPFiles/build/../../XMPFiles/source/FormatSupport/Reconcile_Impl.hpp:15,
                 from /home/alexis/linux/dng2/xmp_sdk/XMPFiles/source/FormatSupport/WAVE/WAVEReconcile.cpp:24:
/usr/include/string.h:65:12: note: 'int memcmp(const void*, const void*, size_t)' declared here, later in the translation unit
 extern int memcmp (const void *__s1, const void *__s2, size_t __n)

This is an error relative to my new version of gcc, described in the Name lookup changes section.
We simply add the #include "string.h" in xmp_sdk/XMPFiles/source/NativeMetadataSupport/ValueObject.h, line 16:

xmp_sdk/XMPFiles/source/NativeMetadataSupport/ValueObject.h
16,17d15
< #include "string.h"

Validate the build

make
[100%] Built target XMPFilesStatic

Check that the shared and static libraries got properly generated:

$ ls xmp_sdk/public/libraries/i80386linux_x64/release
libXMPCore.so*  libXMPFiles.so*  staticXMPCore.ar  staticXMPFiles.ar

DNG SDK

In the current directory, where the XMP SDK got downloaded, download the DNG SDK:

wget http://download.adobe.com/pub/adobe/dng/dng_sdk_1_4.zip
unzip dng_sdk_1_4.zip

There are multiple ways to come up with the binary. In this case,

• We seperate the compilation and linking steps in 2 separate targets.
• We link with the shared libraries of the XMP Toolkit.
• We did not install system wide in /usr/local/lib and /usr/local/include the required XMP files that were generated during the XMP build.

cd dng_sdk/source

Makefile

Create the "Makefile" build configuration file:

# Binary name
EXECUTABLE=dng_validate

# A DNG image
DNG_IMAGE=~/job/image_samples/9436b5a2336f0a575a5b0ef3adf0b25171125081.dng

# The XMP SDK build directory if we don't want to install it system-wide.
XMP_PUB_DIR=/home/alexis/linux/dng/xmp_sdk/public

INCL=-I $(XMP_PUB_DIR)/include
XMP_RELEASE=$(XMP_PUB_DIR)/libraries/i80386linux_x64/release
LIB=-ljpeg -lz -lpthread -ldl -L $(XMP_RELEASE) -lXMPCore -lXMPFiles
SOURCES:=$(shell ls *.cpp)
OBJECTS=$(SOURCES:.cpp=.o)

# Execute the binary
all: $(EXECUTABLE) $(SOURCES)
        LD_LIBRARY_PATH=$(XMP_RELEASE) ./$< -v $(DNG_IMAGE)

# Linking
$(EXECUTABLE): $(OBJECTS)
        g++ $^ $(LIB) -o $@

# Compilation
.cpp.o:
        g++ -c -Wall -g $(INCL) $^

clean:
        rm $(EXECUTABLE) *.o

Build error 1

To avoid errors like

dng_flags.h:36:28: fatal error: RawEnvironment.h: No such file or directory
 #include "RawEnvironment.h"

Create the RawEnvironment.h file containing build settings:

#define qLinux 1
#define qDNGThreadSafe 1
#define UNIX_ENV 1

Build error 2

To avoid errors like

dng_flags.h:40:2: error: #error Unable to figure out platform

A qLinux setting got created in RawEnvironment.h. The "!defined(qMacOS) || !defined(qWinOS)" kind of test line 39 in dng_flags.h does not seem effective on Linux platform even when qMacOS or qWinOS are defined. We replace it by something more standard, for example "#ifndef qLinux":

39 #ifndef qLinux
40 #error Unable to figure out platform
41 #endif

chmod +w dng_flags.h
39c39
< #ifndef qLinux
---
> #if !defined(qMacOS) || !defined(qWinOS)

Build error 3

dng_string.cpp:1792:24: error: 'isdigit' was not declared in this scope
    if (isdigit ((int) c) || c == '.' || c == '-' || c == '+' || c == 'e' || c == 'E')

We simply need to include "ctype.h" in dng_string.cpp for linux platform at line 33:

32 #if qiPhone || qAndroid || qLinux
33 #include <ctype.h> // for isdigit
34 #endif

At line 32 we add qLinux is the list of platforms on when to include ctype.h.

chmod +x dng_string.cpp
32c32
< #if qiPhone || qAndroid || qLinux
---
> #if qiPhone || qAndroid

Build error 4

/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'

The main function declared in dng_validate.cpp was not found. Enable the command build setting in line 200 in dng_flags.h:

200c200
< #define qDNGValidateTarget 1
---
> #define qDNGValidateTarget 0

Build error 5

make clean
make

We now have a linking error

dng_xmp_sdk.cpp:(.text._ZN9TXMPFilesISsE8OpenFileEP6XMP_IOjj[_ZN9TXMPFilesISsE8OpenFileEP6XMP_IOjj]+0x40): undefined reference to `WXMPFiles_OpenFile_2'
collect2: error: ld returned 1 exit status

We want to link with the shared libraries. Delete the static build setting in dng_xmp_sdk.cpp line 48:

chmod +w dng_xmp_sdk.cpp
48a49,50
> #define XMP_StaticBuild 1
> 

Validate the build

The execution of the command looks like:

make

...
LD_LIBRARY_PATH=/home/alexis/linux/dng/xmp_sdk/public/libraries/i80386linux_x64/release ./dng_validate ~/job/image_samples/9436b5a2336f0a575a5b0ef3adf0b25171125081.dng
Validating "/home/alexis/job/image_samples/9436b5a2336f0a575a5b0ef3adf0b25171125081.dng"...
Raw image read time: 0.074 sec
Linearization time: 0.051 sec
Interpolate time: 0.000 sec
Validation complete