|
Identifying Frame Grabber
Core Competencies
by Tom Lecklider, Senior Technical Editor
Do you know where your frame grabber functionality is located?
Automated parts inspection is a good example of a machine vision application. In a typical case,
the parts may be moving on a conveyor belt and have unknown orientation. The
inspection system must capture a picture of each part and determine the
presence, absence, or quality of a feature such as laser-etched text. How does
such a system work in detail?
Typically, the image-sensing device would be an area scan CMOS or CCD camera.
CCD sensors long dominated the camera market, but the CCD process never was a
good match to mainstream semiconductor processing. From that viewpoint, CMOS
sensors are much preferred, but their performance has been inferior until
recently. Today, high-resolution CMOS cameras are available with very good
performance.
For this product inspection example, a camera with square pixels arranged in a
conventional rectangular sensor area would be appropriate. Cameras with square
pixels make computations straightforward. If the product being inspected had
been a web of material instead of a solid object, then a linear CCD-based
line-scan camera would be a better choice. Pixel resolution across the web can
be more than 8,000 points, and the motion of the web provides scanning in the
other direction.
Obviously, there are many more camera specifications to consider, lighting can
be critical, and there’s a computer to specify as well. But the largest
undefined hardware requirement is for the frame grabber, the functional block
that synchronizes image acquisition and transfers data from the camera to the
computer.
Background
When only analog cameras were available, frame grabbers performed digitization,
synchronization, data formatting, local storage, and transfer to the computer.
The advent of the peripheral component interconnect (PCI) bus provided
high-speed data transfer and reduced or eliminated the need for local storage
for some applications. More recently, so-called smart cameras have integrated
the frame grabber functions and computing necessary to perform relatively simple
tasks.
Nevertheless, frame grabbers are very much alive and well, just different than
they used to be. For one thing, higher-end applications generally use digital
cameras. Digitization in the frame grabber is no longer required, but these
cameras often have multiple taps to allow simultaneous parallel data output from
several sections of the CMOS target. Consequently, the overall data rate can be
hundreds of megahertz and exceed the capacity of common PCI-32 buses.
The Camera Link standard governs the camera-to-frame grabber interface. Based on
National Semiconductor’s channel link technology, the specification provides
from four to 12 low-voltage differential-signaling (LVDS) serial data channels,
four control signals, one to three clock lines, and a duplex serial link.
Base-level cameras use four channels, medium-level eight, and full-level 12,
corresponding to one, two, or three Camera Link chips, respectively. Composite
data rates as high as 680 MB/s can be handled today, and higher rates are on the
drawing board.
One of the major physical problems solved by the Camera Link standard is
cabling. Prior to the standard’s introduction, LVDS interfaces were being used
for their high speed, but different camera and frame grabber manufacturers
adopted a variety of connector types. With Camera Link, there now is a stand-ard
cable that simply works.
Nevertheless, many non-Camera Link LVDS systems were built, and frame grabbers
are available to cope with them. For example, MuTech’s Series MV2500 products
use a MDR 68 Connector and cater to 16 b of LVDS data. In addition, they provide
synchronization, event input, strobe output, trigger, and RS-232 I/O
functionality. The company also produces the MV2600 Camera Link-compatible
series with one Hirose and two MDR26 Connectors (Figure 1).
|
Figure 1. Block Diagram of a Camera Link Frame Grabber
Source: MuTech |
An interesting perspective on the evolution of frame grabbers was given by
Pierantonio Boriero, a product line manager at Matrox Imaging. “Any machine
vision system, whether it is a high-performance custom system or a
cost-efficient smart camera, requires common components: lighting; lenses; image
sensors, capture, processing, and analysis; and communications. Where they
differ,” he continued, “is in the level of integration and performance, not
functionality.”
On this basis, you can consider frame grabber functionality to include
digitization; camera control; image acquisition, storage, time stamping,
processing, and analysis; synchronization; data transfer to the host computer;
and communications related to performance monitoring. When cost is very
important, cheaper analog cameras may be used, so most of these functions reside
on the frame grabber board although image storage may be via the PCI bus to part
of the computer’s memory.
The relative cost advantages of analog cameras and smart cameras were discussed
by Dr. Reinhard Borst, vice president of engineering at ELTEC Elektronik AG.
“Digital interfaces and computing power are integrated into smart cameras. These
are additional cost factors, and while it’s true that a smart camera may be
lower cost than a camera and a frame grabber, this is only the case when
considering a single camera.
“If four or eight cameras are attached to one frame grabber, the additional cost
of a smart camera solution is very noticeable,” he explained. “In that case,
using analog cameras and a frame grabber is simply lower cost.”
In high-end digital systems, the frame grabber may use an onboard look-up table
(LUT) to reformat the image data ahead of local storage. If the frame grabber
includes a DSP, it could run a Bayer filter algorithm to interpolate colors
before transferring the data to the computer. And, the byte order could be
swapped to suit Big Endian computers.
Triggering and Control
Especially in applications involving motion, it’s important to trigger image
acquisition when the UUT is in the correct position. In the case of products on
a conveyor belt, for example, a shaft encoder might trigger the frame grabber
shortly before the UUT reached the correct position. Some frame grabbers include
a programmable delay function that allows the image to be positioned accurately
relative to the encoder output. A camera’s asynchronous reset feature also can
be driven by the frame grabber to ensure synchronization.
“Modern frame grabbers now include many of the special features that required
extra circuitry on older models,” said Colin Pearce, the managing director of
Active Silicon. “A considerable amount of circuit integration is used either in
the form of custom silicon or field programmable gate arrays (FPGAs). Thus, such
features as triggering and I/O tend to be integrated onboard as standard.
“Data formatting also is an important feature and contributes to optimum system
performance. For example,” he continued, “a frame grabber can format the data so
the red, green, blue (RGB) data appears in memory in the best order for
processing or display.”
A frame grabber that accepts standard analog video can synchronize itself to the
video stream. This may be done via an analog phase-locked loop (PLL) or much
more quickly via digital synchronization circuitry. Alternatively, some frame
grabbers operate in the sync mastering mode in which they provide horizontal and
vertical synchronization to the camera.
In this case, the frame grabber is the center of the image acquisition system,
accepting triggers from sensors specific to the application, determining how the
camera should be operated in relation to those triggers, and providing the
required control signals.
The Data Translation Model DT3162 Frame Grabber is a good example of a 10-b
resolution analog video frame grabber. Its programmable 40-MHz input timing
supports a variety of standard or nonstandard video formats on up to three input
channels. Synchronization information is stripped from the video, provided
separately by the camera, or supplied by the DT3162. Eight digital I/O lines are
available as well as trigger, strobe, external pixel clock, and exposure control
signals (Figure 2).
Figure 2. Block Diagram of an Analog Video Frame Grabber
Source: Data
Translation |
The software you need to program operation comes as part of the Imaging Omni CD
that ships with the DT3162. Included are an ActiveX control used to develop your
own application software, a device operation verification utility, a device
driver, and evaluation copies of DT Vision Foundry and GLOBAL LAB Image/2. The
first application provides click-and-drag program development, and the second is
a complete image analysis package. Although hardware supplies the processing
horsepower, you need comprehensive software to control a frame grabber.
If a frame grabber is required in a system, it’s logical to ask what else it can
do. Triggering and synchronization are necessities, but what about providing a
number of auxiliary digital I/O lines as in the DT3162 example? Many
applications require some type of actuation before or after an image is
acquired, so if the frame grabber can fill the need, the cost of a separate I/O
board can be saved.
Industrial applications may involve RS-485 differential serial communications or
some other RS serial link. Interfaces to these can be provided by the frame
grabber. Similarly, some frame grabbers incorporate watchdog timers and inputs
for the quadrature signals from a rotary encoder.
Offloading the Host
Capabilities more closely associated with the captured image include cropping
and definition of the area of interest. The two functions may provide the same
end result, but cropping reduces the amount of video data to be transferred
after the entire image has been captured while selecting the area of interest
may cause the frame grabber to capture only pixel data from a certain area.
Alternatively, some cameras can be programmed to output only pixel data from a
selected area.
The differences among these modes of operation may be important depending on the
application. If a camera outputs a smaller amount of data per frame, it may be
possible to achieve a faster frame rate. If the frame grabber isn’t required to
capture complete images, a larger number of smaller image segments can be held
in memory. And, transferring cropped images requires less bus bandwidth and host
memory.
Further operations such as image flipping and data formatting are useful if your
application requires them and if the host processor can’t perform the operation
for some reason. For instance, the additional processing time required might
preclude running an important real-time analysis routine.
LUTs provide a very fast hardware-based data-formatting method. For example,
intensity weighting could be accomplished by using 8-b pixel intensity data to
address a LUT. The LUT output values could be designed to shift the whole
intensity curve upwards or change its sensitivity to compensate for a known
camera characteristic. LUTs typically are specified as 8-8 or 16-16, for
example, meaning that an 8-b (16-b) word addresses them, and the output also is
8-b (16-b) data.
The Cognex MVS-8600 Series is a good example of a flexible, two-channel Camera
Link product. The MVS-8602 accommodates two independent base-level cameras or
one medium-level camera with up to 12-b image data. Onboard 12-b LUTs, pipelined
processing, and image buffers support overlapped DMA transfer to the host during
simultaneous new-image acquisition.
Although frame grabbers initially may have dealt only with image capture and
data transfer, their role today has become much broader. Many times, a frame
grabber is considered a single-board solution to machine vision applications.
You know that you can’t directly connect your camera to the host computer, so if
a frame grabber is needed, it had better include all the additional
functionality that’s required.
This theme was elaborated on by Kirk Petersen, the director of marketing and
communications at DALSA Coreco. “Not only do today’s frame grabbers incorporate
acquisition control features such as trigger inputs and strobe outputs, they
also provide I/O control signals and tag incoming images with unique time stamps
and can format data from multitap cameras into seamless image data.
“Beyond this, some frame grabbers are taking on pre-processing heavy lifting in
an effort to ease the burden on busy host PCs,” he continued. “Image correction
and processing algorithms such as Bayer inversion filters are being done on the
frame grabber rather than by the PC.”
What about those really difficult image-processing applications that can
overwhelm a PC, even with a frame grabber’s help? According to VMETRO, you may
want to consider using a large FPGA perhaps working in conjunction with one or
more DSPs. The company’s PMC-FPGA03 Board is in the PCI mezzanine card (PMC)
format defined by IEEE 1386.1.
When interfaced to a PC via a suitable PCI carrier board and used with the
CAML-MOD3 Camera Link Adapter Module, the PMC-FPGA03 supports a camera’s full
680-MB/s data rate into a large user-programmable Xilinx Virtex II Pro XC2VP50
FPGA supplemented by fast quad data rate (QDR) SRAM and double data rate (DDR)
SDRAM. Communication from the carrier to the PC is via a 64-b PCI interface.
In very advanced frame grabbers, performance is limited as much by your
imagination as by the hardware. For example, JPEG2000 compression/decompression
acceleration, image authentication based on the 256-b secure hash algorithm
(SHA-256), and image filtering are a few of the functions that can be performed.
Decentralization
Rather than anticipating that a separate frame grabber will be needed, some
camera manufacturers are supporting direct high-speed image transfer to host
memory. Similar to the frame grabber changes that occurred when fast PCI-based
data transfer became available, gigabit Ethernet (GbE) and Express PCI (PCIe)
are challenging traditional frame grabber architectures.
Together with ever-faster microprocessors and DDR memory, these two interfaces
make the direct transfer of images from a fast digital camera to host memory
very attractive. What about all the other things that a frame grabber adds to an
application? If you opt for GbE or PCIe data transfer, all other functionality
is split between the PC and camera.
For example, Pleora Technologies has introduced products that support direct GbE
connection between a camera and a standard GbE network interface card (NIC) in a
PC. The iPORT™ PT1000-VB is a small PCB intended to be mounted inside a camera.
The board replaces a camera’s existing back-end circuitry and is said to provide
lower overall cost, longer distance reach, greater networking flexibility, and
more scalable PC processing compared to Camera Link solutions. In addition, the
board includes triggering and auxiliary I/O capabilities and complies with the
automated imaging association (AIA) emerging GbE vision standard.
Conclusion
Clearly, the frame grabber market has changed. You still can address low-cost
applications by using analog cameras, and in fact, your choices have become
broader. You may decide to use a conventional frame grabber, and through modern
ASIC technology, many products offer multiple channel image-handling capability
together with synchronization and triggering functions.
Alternatively, cameras are available with IEEE 1394 (FireWire) interfaces that
simplify connection to a PC. As National Instruments’ Kyle Voosen, product
manager for NI Vision, explained, “FireWire cameras provide digital image
quality, simple cabling, standard software, and a low cost. They are replacing
analog cameras, and today they are supported by a range of machine vision I/O
boards that provides frame grabber functionality as well as digital I/O.”
One reason that frame grabbers are changing so quickly is the rate of PC change.
The PCI bus, for example, has been developed into at least three parallel
versions and most recently the serial PCIe bus. Bill Tanner, CEO of Sensoray,
discussed some of the features required in the video-surveillance,
traffic-monitoring, and medical-imaging markets. He pointed out that changes to
the PC’s internal bus create a risky environment for new frame grabber designs.
“In an attempt to combat this problem, Sensoray’s latest frame grabbers are
external modules that use the USB or FireWire bus,” he commented. “Large systems
of at least 24 channels can be assembled by daisy-chaining multiple frame
grabbers. Also, some products achieve MPEG image compression of over 100,
facilitating image transmission over low-bandwidth communications lines.”
Because of the auxiliary functionality that a frame grabber may include, you
can’t choose the best one for your application by considering only image-related
parameters. You must look more widely at your entire machine vision problem,
especially any actuation, triggering, and synchronization aspects, to determine
if you even need a frame grabber. Similarly, if a large amount of signal
processing is necessary, then that aspect of any proposed solution must be well
understood.
Although frame grabbers have been in use for many years, this is anything but a
one-size-fits-all product. What a frame grabber comprises depends on the problem
it is addressing as well as your point of view. Both software-centric and
hardware-centric solutions can address many of the same applications. Make
certain that you fully understand all the details of your particular application
before deciding which approach is more appropriate. |