Image Transform Manual

General aspects of the library

Usage with the C API or C++ API

The Image Transform library receives images from the C API or the C++ API. You can transform these images into several common image formats without knowing GenICam’s PFNC (Pixel Format Naming Convention) naming scheme.

Image transformation with the .NET or Python API

The Python API offers convenience functions for image transformation. Please always use these convenience functions.
The .NET API transforms images via method Frame.Fill() without accessing the Image Transform library.
For details, see the API manuals.

Variants

The Image Transform Library is available in two variants:

  • The standard variant executes a function call in a single thread.

  • The OpenMP variant distributes the function calls over multiple processor cores (parallel computing).

The OpenMP variant is recommended for achieving a high performance with multi-core PCs: Although additional time is required to schedule the subtasks, its performance roughly scales by the number of free cores. Both library variants have the same interface and the same name, but are delivered in different directories in the SDK installation, allowing you to use the variant you want. The OpenMP variant is located in the subdirectory OpenMP of the single-threaded library variant.

Windows supports OpenMP V2.0, whereas Unix-based systems support OpenMP V4.0.

Optimizing OpenMP performance

The motivation behind OpenMP is to speed up computations by distributing them to the available computing cores (hyper-threading or real cores). To minimize the additional cost of thread management (starting or stopping threads), the OpenMP runtime environment tries to reuse threads as much as possible. Since the default values are not adjusted to your particular use case, we recommend optimizing them to achieve high performance with low CPU load.

Active waiting tasks and CPU load

Ideally, threads are created when OpenMP is first used in an application, and they are always reused without being interrupted by the system. In cases where the application has no work items ready when tasks finish, the waiting tasks consume CPU power for a certain time and then become passive. During active waiting, OpenMP uses high CPU load while no real work is done. This may lead to situations where OpenMP consumes too many resources although the use case isn’t demanding (low image resolution, slow frame rate).

Example

If frames arrive at 10 frames per second, there is one work item every 100 ms. Even if image transformation only takes 10 ms, the threads are active for about 100 ms. OpenMP is underutilized in this case because 90% of what it’s doing is waiting for new work. If frames arrive at 100 frames per second (frames arrive at 10 ms intervals), OpenMP is working to capacity, and all threads can take on new work directly after they have finished the previous one.

Tip

It is recommended to use the same OpenMP runtime environment that the Image Transform library uses (Windows: Visual Studio 10 or higher; Linux: gomp). In this case, Image Transform and the application share OpenMP threads, workload is optimally distributed between worker tasks, and waiting threads get new work fast.

Setting up environment variables

To optimize OpenMP, set up environment variables that determine the number of threads used and OpenMP’s waiting time behavior.

Linux and Windows:

Linux:

  • If OpenMP is set to OMP_WAIT_POLICY=ACTIVE, this variable determines how long a thread waits actively with consuming CPU power before waiting passively without consuming CPU power. If work items arrive at a high rate, it is best to let the thread’s spin wait actively for the next work load. If work items arrive sporadically, a high spin count leads to unnecessary high CPU load. Setting GOMP_SPINCOUNT to 0 is the same as setting OMP_WAIT_POLICY to PASSIVE.

See also

GCC Environment Variables , Visual Studio OpenMP Environment Variables

Supported image data formats

Image file formats vs image data formats

In contrast to image file formats such as PNG, image data formats contain no embedded metadata, but only pure images as rectangular arrays of pixels. The pixels have characteristics such as:

  • Color

  • Bit depth

  • Endianness for a bit depth greater than eight

  • Order

The pixel format name describes these characteristics.

Vimba pixel formats and PFNC pixel formats

The Image Transform library uses:

  • Pixel formats from the GigE Vision and USB Vision interface specification

  • Pixel formats from GenICam’s Pixel Format Naming Convention (PFNC)

  • Common display formats on Windows and Linux platforms

Note

All implemented image transformations only accept multi-byte pixel formats in little-endian byte order.

The names of the pixel formats mainly match the list of values of the PixelFormat feature in the SFNC (Standard Features Naming Convention) of GenICam, but the actual name depends on the interface standard according to the following table:

Vimba pixel formats and their counterparts in interface standards

Vimba Pixel format
(VmbPixelFormat…)

GigE Vision <2.0,
FireWire

PFNC (USB3 Vision,
GigE Vision >=2.0)

Mono8

Mono8

Mono8

Mono10

Mono10

Mono10

Mono10p

Mono10p

Mono12

Mono12

Mono12

Mono12p

Mono12Packed

Mono12p

Mono12Packed

Mono12Packed

not available

Mono14

Mono14

Mono14

Mono16

Mono16

Mono16

BayerXY*8

BayerXY*8

BayerXY*8

BayerXY*10

BayerXY*10

BayerXY*10

BayerXY*10p

BayerXY*10p

BayerXY*12

BayerXY*12

BayerXY*12

BayerXY*12p

BayerXY*12p

BayerXY*12Packed

BayerXY*12Packed

not available

BayerXY*16

BayerXY*16

BayerXY*16

Rgb8

RGB8Packed

RGB8

Bgr8

BGR8Packed

BGR8

Rgba8

RGBa8Packed

RGBa8

Bgra8

BGRa8Packed

BGRa8

Rgb10

RGB10Packed

RGB10

Bgr10

BGR10Packed

BGR10

Rgba10

RGBa10

Bgra10

BGRa10

Rgb12

RGB12Packed

RGB12

Bgr12

BGR12Packed

BGR12

Rgba12

RGBa12

Bgra12

BGRa12

Rgb16

RGB16Packed

RGB16

Bgr16

BGR16Packed

BGR16

Rgba16

RGBa16

Bgra16

BGRa16

Yuv411

Yuv411Packed

YUV411_8_UYYVYY

Yuv422

Yuv422Packed

YUV422_8_UYVY

Yuv444

Yuv444Packed

YUV8_UYV

YCbCr411_8_CbYYCrYY

YCbCr411_8_CbYYCrYY**

YCbCr422_8_CbYCrY

YCbCr422_8_CbYCrY**

YCbCr422_8_YCbYCr

YCbCr422_8_YCbYCr**

YCbCr8_CbYCr

YCbCr8_CbYCr (4:4:4)

* BayerXY is any of BayerGR, BayerRG, BayerGB, or BayerBG

** Layout/values same as Yuv422

Because the following pixel formats span a few bytes, or in some cases several lines, their width and height or size (in pixels) must be divisible by the following values:

Minimum sizes for certain pixel formats

Pixel format

Width

Height

Size

Mono12Packed,Mono12p

1

1

2

BayerXY…

2

2

1

Yuv411

4

1

1

Yuv422

2

1

1

Supported transformations

Tip

For applying transformations, see the section Transformation examples in this manual and the AsynchrounousGrab example.

Note

Converting an image from a lower to a higher bit depth aligns the image data to the least significant bit. The remaining bits are set to zero.

All pixel formats can be transformed into:

  • Mono8, Rgb/Bgr8

  • Rgba/Bgra8

Note

Transformations from higher into lower bit depths or from color into mono are lossy.

Note

Transforming a pixel format into itself (for troublechecking purposes) is not always supported, but you can always apply the transformations listed above.

Additionally, the following transformations are supported:

All 10-bit pixel formats including “p” and “Packed” can be transformed into:

  • Mono10

  • Rgb/Bgr10, Rgba/Bgra10

All 12-bit pixel formats including “p” and “Packed” can be transformed into:

  • Mono12

  • Rgb/Bgr12, Rgba/Bgra12

Mono14 can be transformed into Rgb/Bgr14, Rgba/Bgra14.

All 16-bit pixel formats including “p” and “Packed” can be transformed into:

  • Mono16

  • Rgb/Bgr16, Rgba/Bgra16

The following pixel formats can be transfomed into Yuv422:

  • Mono8

  • BayerXY8

Additionally to the lossless transformations described above, lossy transformations are supported: Mono10p, Mono12p, and Mono12Packed have a lossy conversion to 8-bit formats and a lossless conversion to Mono10 and Mono12 (both LSB and Little Endian). The same is true for Bayer and RGB formats. Additionally, they have a lossy conversion to Mono formats.

The following sections of this document describe how to apply a transformation.

API usage

General aspects

Every concrete image transformation must specify its three parameters:

  • The fully-qualified input image (including complete format specification)

  • The complete output format (plus a pointer to enough memory to hold the transformed image)

  • The transformation itself (including transformation options if there is more than one possible way from the input format to the output format)

The main function VmbImageTransform() covers all required transformation parameters. The function uses two pointers of the VmbImage struct to specify the source and destination image and a list of VmbTransformInfo structs to specify the transformation.

VmbImageTransform ( &sourceImage , &destinationImage , &info , 1 );

To ease filling the structs that are needed for a successful call of VmbImageTransform(), several helper functions for initializing them are available:

  • Methods for filling the necessary format information of either the input or the output image (described below in chapter Filling Image Information).

  • Methods for filling optional transformation parameters to invoke special and additional functionality.

Filling Image Information

The Image Transform library offers several methods to find an set a compatible destination format.

To easily set a compatible destination image format, you can initialize the VmbImage struct with:

  • VmbSetImageInfoFromInputImage() by providing an input image from the C or C++ API, the output layout, and bit depth

  • VmbSetImageInfoFromInputParameters() by providing the input format, the input image size, the output layout, and bit depth

  • VmbSetImageInfoFromPixelFormat() by providing a VmbPixelFormat_t value and the size

Experts can use VmbSetImageInfoFromString() by providing a string that describes the pixel format and the size.

Not recommended: Although it is not recommended, it is possible to fill the fields explicitly one by one.

Specifying transformation options

Depending on your application and the desired image characteristics, you can choose between several transformation options.

Adding color transformations

A common way to transform color images is using a 3x3 matrix with method VmbSetColorCorrectionMatrix3x3(). The matrix you have to specify is a 3x3 row order float matrix.

\[\begin{split}\begin{pmatrix} rr & rg & rb\\ gr & gg & gb\\ br & bg & bb \end{pmatrix}\end{split}\]

Changing the debayering algorithm

Every transformation of a Bayer image requires defining how to interpolate or combine the color pixels. For best system performance, a simple 2x2 demosaicing algorithm is the default. You can change this algorithm with an additional transformation parameter to VmbImageTransform(), which can be filled with function VmbSetDebayerMode(), using a VmbDebayerMode() as input.

Possible values of VmbDebayerMode() are listed in the following table:

Value

Description

VmbDebayerMode2x2

2x2 with green averaging (default, OpenMP only)

VmbDebayerMode3x3

3x3 with equal green weighting per line

VmbDebayerModeLCAA*

Debayering with horizontal local color anti-aliasing

VmbDebayerModeLCAAV*

Debayering with horizontal and vertical local color anti-aliasing

VmbDebayerModeYUV422*

Debayering with YUV422-alike sub-sampling

* For 8-bit images only

Note

If VmbDebayerModeLCAA, VmbDebayerModeLCAAV, or VmbDebayerModeYUV422 is used, the input buffers serve as intermediate buffers for the transformation for performance reasons. To use the input buffer for another purpose, debayer afterwards or copy the input buffer beforehand.

Transforming images

When all the parameters of a transformation are prepared, use the pre-filled structs and call VmbImageTransform().

Note

Inapplicable transformation options are ignored (for example, debayering options to a monochrome input image).

Transformation examples

Debayering into an Rgb8 image

Transformation of a BayerGR8 image with 640x480 pixels into an Rgb8 image of the same size:

VmbImage                sourceImage;
VmbImage                destinationImage;
VmbTransformInfo        info;

// Set size member for verification inside API
sourceImage.Size        = sizeof ( sourceImage );
destinationImage.Size   = sizeof ( destinationImage );

// Attach the data buffers
sourceImage.Data        = pInBuffer;
destinationImage.Data   = pOutBuffer;

// Fill image info from pixel format
VmbSetImageInfoFromPixelFormat ( VmbPixelFormatBayerGR8 ,
                                 640,
                                 480,
                                 &sourceImage );

// Fill destination image info from input image
VmbSetImageInfoFromInputImage  ( &sourceImage ,
                                 VmbPixelLayoutRGB ,
                                 8,
                                 &destinationImage );

// Set the debayering algorithm to 2x2
VmbSetDebayerMode ( VmbDebayerMode2x2 , &info );

// Perform the transformation
VmbImageTransform ( &sourceImage , &destinationImage , &info , 1 );

Debayering into a Mono8 image

Preparing a transformation of a BayerRG8 image with 640x480 pixels into a Mono8 image of the same size:

VmbImage                sourceImage;
VmbImage                destinationImage;
VmbTransformInfo        info;

// Set size member for verification inside API
sourceImage.Size        = sizeof ( sourceImage );
destinationImage.Size   = sizeof ( destinationImage );

// Attach the data buffers
sourceImage.Data        = pInBuffer;
destinationImage.Data   = pOutBuffer;

// Fill image info from pixel format string
std:: string name ( "PixelFormatBayerRG8" );
VmbSetImageInfoFromString ( name.c_str(),
                            static_cast <VmbUint32_t >( name.size() ),
                            640,
                            480,
                            &sourceImage );

// Fill image info from pixel known input parameters
VmbSetImageInfoFromInputParameters ( VmbPixelFormatBayerRG8 ,
                                     640,
                                     480,
                                     VmbPixelLayoutMono ,
                                     8
                                     &destinationImage );

// Set the debayering algorithm to 3x3
VmbSetDebayerMode ( VmbDebayerMode3x3 , &info );

// Perform the transformation
VmbImageTransform ( &sourceImage , &destinationImage , &info , 1 );

Color correction of an Rgb8 image

Applying a color correction to an Rgb8 image with 640x480 pixels:

VmbImage          sourceImage;
VmbImage          destinationImage;
VmbTransformInfo  info;
const VmbFloat_t  mat[] = { 1.0f, 0.0f, 0.0f,
                            0.0f, 1.0f, 0.0f,
                            0.0f, 0.0f, 1.0f };

// Set size member for verification inside API
sourceImage.Size        = sizeof ( sourceImage );
destinationImage.Size   = sizeof ( destinationImage );

// Attach the data buffers
sourceImage.Data        = pInBuffer;
destinationImage.Data   = pOutBuffer;

// Fill image info from pixel format
VmbSetImageInfoFromPixelFormat ( VmbPixelFormatRgb8 ,
                                 640,
                                 480,
                                 &sourceImage );

// Fill destination image info from input image
VmbSetImageInfoFromInputImage ( &sourceImage ,
                                VmbPixelLayoutRGB ,
                                8,
                                &destinationImage );

// Set the transformation matrix
VmbSetColorCorrectionMatrix3x3 ( mat , &info );

// Perform the transformation
VmbImageTransform ( &sourceImage , &destinationImage , &info , 1 );

Advanced transformation with debayering

Transformation of a BayerGR12 image with 640x480 pixels into a Mono16 image of the same size:

VmbImage          sourceImage;
VmbImage          destinationImage;
VmbTransformInfo  info [2];
const VmbFloat_t  mat[] = { 1.0f, 0.0f, 0.0f,
                            0.0f, 1.0f, 0.0f,
                            0.0f, 0.0f, 1.0f };

// Set size member for verification inside API
sourceImage.Size        = sizeof ( sourceImage );
destinationImage.Size   = sizeof ( destinationImage );

// Attach the data buffers
sourceImage.Data        = pInBuffer;
destinationImage.Data   = pOutBuffer;

// Fill image info from pixel format
VmbSetImageInfoFromPixelFormat ( VmbPixelFormatRgb8 ,
                                 640,
                                 480,
                                 &sourceImage );

// Fill destination image info from input image
VmbSetImageInfoFromInputImage ( &sourceImage ,
                             VmbPixelLayoutMono ,
                             16,
                             &destinationImage );

// Set the debayering algorithm to 2x2
VmbSetDebayerMode ( VmbDebayerMode2x2 , &info [0] );

// Set the transformation matrix
VmbSetColorCorrectionMatrix3x3 ( mat , &info [1] );

// Perform the transformation
VmbImageTransform ( &sourceImage , &destinationImage , &info , 2 );

Function reference

Information

VmbGetVersion()

This function inquires the library version. It can be called at anytime, even before the library is initialized.

Type

Name

Description

out

VmbUint32_t*

pValue

Contains the library version

VmbErrorBadParameter: If the given pointer is NULL

VmbGetErrorInfo()

Type

Name

Description

in

VmbError_t

errorCode

The error code to get a readable string for

out

VmbANSIChar_t*

pInfo

Pointer to a zero terminated string that
contains error information on return

in

VmbUint32_t

maxInfoLength

The length of the pInfo buffer

VmbGetApiInfoString()

Type

Name

Description

in

VmbAPIInfo_t

infoType

Type of information to return

out

VmbANSIChar_t*

pInfo

Pointer to a zero terminated string that

contains error information on return

in

VmbUint32_t

maxInfoLength

The length of the pInfo buffer
VmbErrorBadParameter: If the given pointer is NULL
VmbErrorMoreData: If maxInfoLength is too small to hold the complete information

The parameter infoType may have one of the following values:

Value

Description

VmbAPIInfoAll

Return all information about the API

VmbAPIInfoPlatform

Return information about the platform (x86 or x64)

VmbAPIInfoBuild

Return info about the API build (debug or release)

VmbApiInfoTechnology

Return info about supported technologies, like OpenMP

Transformation

VmbImageTransform()

This function transforms images from one pixel format to another with possible transformation options. The transformation is defined by the provided images and the desired transformation.

Type

Name

Description

in

const VmbImage*

pSource

Image to transform

in/out

VmbImage*

pDestination

Destination image

in

const VmbTransformInfo*

pParameter

Optional transform parameters

in

VmbUint32_t

parameterCount

Number of transform parameters

VmbErrorBadParameter:

  • If any image pointer or their “Data” members is NULL

  • If Width or Height doesn’t match between source and destination

  • If one of the parameters for the conversion does not fit

VmbErrorStructSize: If the image structs size don’t match their “Size” member
VmbErrorNotImplemented: If there is no supported transformation between source and destination format

Tip

Fill the source and destination image info struct with VmbSetImageInfoFromPixelFormat() or VmbSetImageInfoFromString() and keep those structs as template. For calls to transform, attach the image to the “Data” member. When set, the optional parameters are constraints on the transform.

Helper functions

VmbSetImageInfoFromInputImage()

This function sets image info member values in VmbImage from input VmbImage. Use simple target layout and bit per pixel values to set the desired target format.

Type

Name

Description

in

const VmbImage*

pInputmage

Image source to transform

in

VmbPixelLayout_t

OutputPixelLayout

Pixel layout of the target image

in

VmbUint32_t

BitPerPixel

Bits per pixel of the target image

in/out

VmbImage*

pOutputImage

Pointer to Vmb image to set the info to

VmbErrorBadParameter:

  • If the given pointer is NULL

  • If one of the parameters for the conversion does not fit

VmbErrorNotImplemented: Transformation is not supported

VmbSetImageInfoFromInputParameters()

Set image info member values in VmbImage from pixel format. Use simple target layout and bit per pixel values to set the desired target format.

Type

Name

Description

in

VmbPixelFormat_t

pInputmage

Image source to transform

in

VmbUint32_t

Width

Image width in pixels

in

VmbUint32_t

Height

Image height in pixels

in

VmbPixelLayout_t

OutputPixelLayout

Pixel layout of the target image

in

VmbUint32_t

BitPerPixel

Bits per pixel of the target image

in/out

VmbImage*

pOutputImage

Pointer to Vimba image to set the info to
VmbErrorBadParameter: If the given pointer is NULL or one of the image struct members is invalid
VmbErrorNotImplemented: Transformation is not supported

VmbSetImageInfoFromPixelFormat()

This is an expert function. Set image info member values in VmbImage from pixel format. As a reference, refer to section Supported transformations.

Type

Name

Description

in

const VmbANSIChar_t*

imageFormat

Image format as a (const) case insensitive
string that is either a PixelFormat
(Vmb is optional) or a pixel struct name

in

VmbUint32_t

StringLength

The length of the pixel format string

in

VmbUint32_t

Width

Image width in pixels

in

VmbUint32_t

Height

Image height in pixels

in/out

VmbImage*

pImage

Pointer to Vimba image to set the info to
VmbErrorBadParameter: If the given pointer is NULL or one of the image struct members is invalid
VmbErrorStructSize: If the image struct size doesn’t match its “Size” member

Tip

VmbPixelFormat_t can be obtained from the C/C++ API’s frame or from the PixelFormat feature. For displaying images, you can use VmbSetImageInfoFromString() or to look up a matching VmbPixelFormat.

VmbSetImageInfoFromString()

This is an expert function. Set image info member values in VmbImage from string.

Type

Name

Description

in

const VmbANSIChar_t*

imageFormat

Image format as a (const) case insensitive
string that is either a PixelFormat
(Vmb is optional) or a pixel struct name

in

VmbUint32_t

StringLength

The length of the pixel format string

in

VmbUint32_t

Width

Image width in pixels

in

VmbUint32_t

Height

Image height in pixels

in/out

VmbImage*

pImage

Pointer to Vimba image to set the info to

VmbErrorBadParameter:

  • If one of the given pointers or the “Data” member in the image is NULL

  • If Width or Height don’t match between source and destination

  • If one of the parameters for the conversion does not fit

VmbErrorStructSize: If the image struct size doesn’t match its “Size” member
VmbErrorResources: If the there was a memory fault during processing

VmbSetDebayerMode()

Set transformation options to a predefined debayering mode.

Note

Debayering is only applicable to image formats with both an even width and an even height.

Type

Name

Description

in

VmbDebayerMode_t

DebayerMode

Used for debayering the source raw
image. Default: 2x2.

in/out

VmbTransformInfo_t*

pTransformInfo

Parameter that contains information about
special transform functionality

VmbErrorBadParameter: If the given pointer is NULL

VmbSetColorCorrectionMatrix3x3()

Set transformation options to a 3x3 color matrix transformation.

Type

Name

Description

in

const VmbFloat_t*

DebayerMode

Color correction matrix

in/out

VmbTransformInfo_t*

pTransformInfo

Parameter that contains information about
special transform functionality

VmbErrorBadParameter: If one of the given pointers is NULL

Structs

VmbImage

VmbImage encapsulates image data for the transformation function.

Struct entry

Purpose

VmbUint32_t

Size

Size of the structure

void*

Data

Pointer to the payload received from the C/C++ API
or to the display image data

VmbImageInfo

ImageInfo

Internal information data for mapping the images
to the correct transformation. Set with
VmbSetImageInfo() helper functions from VmbPixelFormat_t
or format string

VmbImageInfo

VmbImageInfo contains image information needed for the transformation function.

Struct entry

Purpose

VmbUint32_t

Width

Image width in pixels or sub-pixels for macro pixel
formats like YUV

VmbUint32_t

Height

The height of the image in pixels

VmbInt32_t

Stride

The offset from the current line to the next line,
a value not equal to Width is currently not supported

VmbPixelInfo

PixelInfo

Information about the pixel format

VmbPixelInfo

The pixel format of an image is described within VmbPixelInfo.

Struct entry

Purpose

VmbUint32_t

VmbUint32_t

Number of bits for one image pixel or sub-pixel in macro

pixel formats

VmbUint32_t

BitsUsed

Number of bits used per pixel, e.g. RGB12 has 48 bits per

pixel and 36 bits used

VmbAlignment_t

Alignment

For image formats where BitsPerPixel is not equal to

bitsUsed, the alignment specifies the bit layout of the pixel

VmbEndianness_t

Endianness

Specifies the endianness of pixels that are larger than

one byte

VmbPixelLayout_t

PixelLayout

Describes the layout of the pixel component, e.g., RGB or

BGR layout

VmbBayerPattern_t

BayerPattern

For raw image data, this field specifies the color filter
array layout of the sensor

VmbTransformInfo

Transformation parameters given as a pointer to a VmbTransformInfo struct are used to invoke special and additional functionality, which is applied while interpreting the source image (debayering) or transforming the images (color correction).

Struct entry

Purpose

VmbTransformType_t

TransformType

Transform info type, may be
VmbTransformTypeDebayerMode
or VmbTransformTypeColorCorrection Matrix

VmbTransformParameter

Parameter

Transform info variant, may be
VmbTransformParameterDebayer
or VmbTransformParameterMatrix3x3