Sunday, May 4, 2014

DNG 1.4 Parser

This tutorial describes how to build the Adobe DNG SDK on Linux.
It generates the dng_validate C++ program that can parse any DNG images, a bit like a "Hello world" for DNG image processing.

  1. Adobe DNG SDK 1.4

Adobe DNG SDK 1.4

The goal of this blog entry is to execute the dng_validate command built from the source.
We show how to build the XMP libraries then suggest a Makefile configuration to build a DNG SDK sample app.
All the patches were generated by the diff command.


In the current directory, download the XMP SDK:
mv XMP-Toolkit-SDK-CC201306 xmp_sdk

Make sure you have at least the following packages installed:

sudo apt-get install cmake libjpeg8-dev uuid-dev

The 2 next snippets download zlib and expat sources. Instructions are explained in the ReadMe.txt files in the third-party directories.


cd xmp_sdk/third-party/zlib
tar xzf zlib-1.2.8.tar.gz
cp zlib-1.2.8/*.h zlib-1.2.8/*.c .


cd xmp_sdk/third-party/expat
Download here the source at
tar xzf expat-2.1.0.tar.gz
cp -R expat-2.1.0/lib .

For cross-platform compatibility, to be able to support both Mac & Windows, cmake is the tool selected by the SDK authors to build the source. We go through all the build errors.

cd xmp_sdk/build

Build error 1

CMake Error at shared/SharedConfig_Common.cmake:38 (if):
  if given arguments:

    "LESS" "413"

  Unknown arguments specified

I use gcc 4.8.2 for the build. There seems to be a XMP_VERSIONING_GCC_VERSION variable not properly set. We just delete the version check by removing line 37 till 42 in xmp_sdk/build/shared/SharedConfig_Common.cmake:

>               # workaround for visibility problem and gcc 4.1.x
>               if(${${COMPONENT}_VERSIONING_GCC_VERSION} LESS 413)
>                       # only remove inline hidden...
>                       string(REGEX REPLACE "-fvisibility-inlines-hidden" "" CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS})
>               endif()

Build error 2

Linking CXX shared library /home/alexis/linux/dng2/xmp_sdk/public/libraries/i80386linux_x64/release/
g++: error: /usr/lib64/gcc/x86_64-redhat-linux/4.4.4//libssp.a: No such file or directory


... He may also want to change the parameter XMP_ENABLE_SECURE_SETTINGS as per the configured gcc.
 a) If the gcc is configured with --enable-libssp (can be checked by executing gcc -v), he has to set the variable XMP_GCC_LIBPATH inside of file /build/XMP_Linux.cmake to the path containing the static lib( libssp.a).In this case he can set the variable the XMP_ENABLE_SECURE_SETTINGS on.
b) If the gcc is configured with --disable-libssp, he has to set the variable XMP_ENABLE_SECURE_SETTINGS off.

I don't have any --enable-libssp in my gcc version:

$ gcc -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.8.2-21' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --with-arch-32=i586 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.2 (Debian 4.8.2-21)

so I disable it in xmp_sdk/build/shared/ToolchainGCC.cmake line 37:


Build error 3

/home/alexis/linux/dng2/xmp_sdk/XMPFiles/build/../../XMPFiles/source/NativeMetadataSupport/ValueObject.h:111:60: error: 'memcmp' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
    doSet = ( memcmp( mArray, buffer, numElements*sizeof(T) ) != 0 );
In file included from /home/alexis/linux/dng2/xmp_sdk/XMPFiles/build/../../XMPFiles/source/FormatSupport/TIFF_Support.hpp:17:0,
                 from /home/alexis/linux/dng2/xmp_sdk/XMPFiles/build/../../XMPFiles/source/FormatSupport/ReconcileLegacy.hpp:15,
                 from /home/alexis/linux/dng2/xmp_sdk/XMPFiles/build/../../XMPFiles/source/FormatSupport/Reconcile_Impl.hpp:15,
                 from /home/alexis/linux/dng2/xmp_sdk/XMPFiles/source/FormatSupport/WAVE/WAVEReconcile.cpp:24:
/usr/include/string.h:65:12: note: 'int memcmp(const void*, const void*, size_t)' declared here, later in the translation unit
 extern int memcmp (const void *__s1, const void *__s2, size_t __n)

This is an error relative to my new version of gcc, described in the Name lookup changes section.
We simply add the #include "string.h" in xmp_sdk/XMPFiles/source/NativeMetadataSupport/ValueObject.h, line 16:

< #include "string.h"

Validate the build

[100%] Built target XMPFilesStatic

Check that the shared and static libraries got properly generated:

$ ls xmp_sdk/public/libraries/i80386linux_x64/release**


In the current directory, where the XMP SDK got downloaded, download the DNG SDK:


There are multiple ways to come up with the binary. In this case,
  • We seperate the compilation and linking steps in 2 separate targets.
  • We link with the shared libraries of the XMP Toolkit.
  • We did not install system wide in /usr/local/lib and /usr/local/include the required XMP files that were generated during the XMP build.

cd dng_sdk/source


Create the "Makefile" build configuration file:

# Binary name

# A DNG image

# The XMP SDK build directory if we don't want to install it system-wide.

INCL=-I $(XMP_PUB_DIR)/include
LIB=-ljpeg -lz -lpthread -ldl -L $(XMP_RELEASE) -lXMPCore -lXMPFiles
SOURCES:=$(shell ls *.cpp)

# Execute the binary

# Linking
        g++ $^ $(LIB) -o $@

# Compilation
        g++ -c -Wall -g $(INCL) $^

        rm $(EXECUTABLE) *.o

Build error 1

To avoid errors like

dng_flags.h:36:28: fatal error: RawEnvironment.h: No such file or directory
 #include "RawEnvironment.h"

Create the RawEnvironment.h file containing build settings:

#define qLinux 1
#define qDNGThreadSafe 1
#define UNIX_ENV 1

Build error 2

To avoid errors like

dng_flags.h:40:2: error: #error Unable to figure out platform

A qLinux setting got created in RawEnvironment.h. The "!defined(qMacOS) || !defined(qWinOS)" kind of test line 39 in dng_flags.h does not seem effective on Linux platform even when qMacOS or qWinOS are defined. We replace it by something more standard, for example "#ifndef qLinux":

39 #ifndef qLinux
40 #error Unable to figure out platform
41 #endif

chmod +w dng_flags.h
< #ifndef qLinux
> #if !defined(qMacOS) || !defined(qWinOS)

Build error 3

dng_string.cpp:1792:24: error: 'isdigit' was not declared in this scope
    if (isdigit ((int) c) || c == '.' || c == '-' || c == '+' || c == 'e' || c == 'E')

We simply need to include "ctype.h" in dng_string.cpp for linux platform at line 33:

32 #if qiPhone || qAndroid || qLinux
33 #include <ctype.h> // for isdigit
34 #endif

At line 32 we add qLinux is the list of platforms on when to include ctype.h.

chmod +x dng_string.cpp
< #if qiPhone || qAndroid || qLinux
> #if qiPhone || qAndroid

Build error 4

/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'

The main function declared in dng_validate.cpp was not found. Enable the command build setting in line 200 in dng_flags.h:

< #define qDNGValidateTarget 1
> #define qDNGValidateTarget 0

Build error 5

make clean

We now have a linking error

dng_xmp_sdk.cpp:(.text._ZN9TXMPFilesISsE8OpenFileEP6XMP_IOjj[_ZN9TXMPFilesISsE8OpenFileEP6XMP_IOjj]+0x40): undefined reference to `WXMPFiles_OpenFile_2'
collect2: error: ld returned 1 exit status

We want to link with the shared libraries. Delete the static build setting in dng_xmp_sdk.cpp line 48:

chmod +w dng_xmp_sdk.cpp
> #define XMP_StaticBuild 1

Validate the build

The execution of the command looks like:

LD_LIBRARY_PATH=/home/alexis/linux/dng/xmp_sdk/public/libraries/i80386linux_x64/release ./dng_validate ~/job/image_samples/9436b5a2336f0a575a5b0ef3adf0b25171125081.dng
Validating "/home/alexis/job/image_samples/9436b5a2336f0a575a5b0ef3adf0b25171125081.dng"...
Raw image read time: 0.074 sec
Linearization time: 0.051 sec
Interpolate time: 0.000 sec
Validation complete


  1. unfortunately, I fail compiling it so far...xmp_sdk does not eat -ldl for some reason


    make CFLAGS="${CFLAGS} -ldl"
    LIBS="$LIBS -lm -ldl"
    LDLIBS=-ldl make
    SET (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -ldl") in CMakeLists.txt

    no avail..

    any hints?

  2. found it, you have to add ${CMAKE_DL_LIBS} as last item (after ${LIB_ADOBEXMP} ) in line 319 to the function target_link_libraries() in XMP-SDK/XMPFiles/build/CMakeLists.txt

  3. Thank you, thank you, thank you, for your time and for sharing a step-by-step. Now I need no more boring reboots to develop (in Linux) and validate (in Windows). As a side note, shalamander's tip saved the day, a sunny sunday. Thank all of you guys!

  4. The tutorial was very helpful, but I ran into a few extra snags. I was building for Ubuntu 14.04 64-bit, sourcing dng_sdk_1_3 and XMP-Toolkit-SDK-CC2014-12. I've included the diff's of the directories that I finally got to build.

    Unified diff of the XMP_SDK directory (command: diff -r XMP-SDK XMP-Toolkit-SDK-CC201412):

    Unified diff of the dng_sdk directory (command: diff -r dng_sdk dng_sdk_original):


  5. That is very interesting; you are a very skilled blogger. I have shared your website in my social networks..!

    Digital Marketing Services in Chennai

    Digital Marketing Company in Chennai

  6. keep sharing your information regularly for my future reference. This content creates a new hope and inspiration with in me

    Content marketing services in chennai

  7. This idea is mind blowing. I think everyone should know such information like you have described on this post. Thank you for sharing this explanation.Your final conclusion was good. We are sowing seeds and need to be patiently wait till it blossoms.
    Software Testing Training

  8. wow amazing post.The key points you mentioned here related to maintenance of car is really awesome.Checking all fluid levels,changing oil and of course the regular service of the car which is necessary to maintain our vehicle.Thank you for the information.

    bike spa services in mumbai
    house cleaning services in mumbai
    car wash services in mumbai


  9. yes good description about xmp it will be remember to me,but i want more information from this topic.
    Back to original

  10. Superb. I really enjoyed very much with this article here. Really it is an amazing article I had ever read. I hope it will help a lot for all. Thank you so much for this amazing posts and please keep update like this excellent article.thank you for sharing such a great blog with us. expecting for your.
    Digital Marketing Company in India

  11. This was so useful and informative. The article helped me to learn something new. Android App Development Company in India

  12. Very nice post here thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.

    Salesforce Training in Chennai

  13. I just see the post i am so happy the post of information's.So I have really enjoyed and reading your blogs for these posts.Any way I’ll be subscribing to your feed and I hope you post again soon.
    Skilled Manpower Services in Chennai

  14. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command.

    Digital Marketing Company in Chennai