Adding Color to Machine Vision
by Robert Howison, DALSA
Machine vision has evolved to become a fast
and reliable tool for quality inspection. In many cases, a
machine vision system can perform inspections more quickly and
accurately than humans and at a lower cost. However, can a
machine see in color? And, does introducing color into
the equation help improve the quality of the inspection?
A machine vision system acquires images of an
object with a camera and then uses computers to process,
analyze, and measure various characteristics of that object so
decisions can be made. One of the characteristics analyzed can
be an object’s color.
In the past, color was not widely used in
optical inspection because of the cost and processing power
required. However, as costs decrease and processing power ceases
to be an issue, solutions providers are beginning to incorporate
color into machine vision optical inspection systems to provide
higher quality.
Color is Better, Right?
The basic assumption is that color is more
advanced than black and white or monochrome so it must be
better. However, this is not always true in machine vision.
DALSA offers the Genie, Trillium, and Piranha Cameras for machine vision, each based on a variety of distinct color imaging technologies including Bayer color filter array, beam splitter prism, and trilinear sensor.
There are some things that monochrome cameras
do better than color cameras. Take resolution and speed, for
example. There are more choices in high resolution and high
speed when shopping for monochrome cameras.
In many cases, color images do not offer any
advantage over monochrome images in resolving a machine vision
problem, such as optical character recognition, optical
character verification, bar-code reading, gauging, and
applications dependent on high-resolution spatial information.
For example, in typical inspection applications where defects
like cracks or scratches are detected, the use of color is not
necessary because the goal is to discern a difference in
lightness on the object’s surface.
How Machines See in Color
A machine cannot actually see in color.
Machines use mathematical models to approximate human color
detection and can be calibrated against the average human
response to see color because it gives consistent
responses to colors observed in a controlled setting. This
calibrated color vision is useful for measuring and matching
colorants such as in paint, plastics, and fabrics.
We don’t think of this as seeing color like a
human. It is important to make the distinction between the
relative measurements that can be made with a machine vision
system vs. absolute measurements that are only possible with
devices such as photospectrometers.
Human color vision evolved to reliably
extract information about the material properties of objects
seen under huge variations of illumination and view. For
example, fruit color has to be reliably determined despite
varying illumination to pick ripe from unripe or bad fruit.
Human color vision has mechanisms to factor
out variations in illumination and view that we don’t know
how or don’t bother to put into machine color vision. It also is
relative: Nearby colors influence the perception of a color,
human color vision has low resolution, and there are wide
differences between individuals.
As a result, human color vision is not a good
measuring tool. Machine color vision is not influenced by nearby
colors, can have high resolution, and does not vary much from
machine to machine, making it a good measuring tool.
Types of Color Machine Vision Systems
Most color machine vision systems use a mix
of hardware and software to detect colors. For point or spot
color measures, a mostly hardware solution is fine.
Sophisticated detection systems rely more heavily on software to
give flexibility to the designer and user.
The main types of color cameras used in
machine vision applications are 3CCD, trilinear, and Bayer
pattern. These can be either area-scan or line-scan cameras
depending upon the type of color incorporated.
3CCD has excellent color registration and can
be applied to the majority of applications; however, the cost is
higher. In a 3CCD color camera, color is selected using a
prism-based interference filter that splits the incoming light
into red (R), green (G), and blue (B) components.
Each of the three primary colors then is
detected by a CCD, respectively, and the final color image is
reconstructed by combining the outputs from the three CCDs. All
three color images are captured at the same object spot and at
the same time.
Trilinear provides high performance and has
advantages in terms of its low cost. It can be used in many
applications such as 100% print inspection. However, spatial
correction cannot be achieved properly in certain applications
that involve rotating or randomly moving objects.
In a trilinear color camera, three linear
arrays are fabricated on one single die and coated with RGB
color filters, respectively. These are absorbing filters using
dye or pigment. In the trilinear camera, the three linear arrays
detect a slightly different field of view (FOV) of the object,
and spatial correction is needed in the reconstruction of a
color image.
Bayer pattern cameras offer the lowest cost
solution. These cameras tend to be used in lower-end
applications and have reduced color precision compared to 3CCD
and trilinear cameras. However, there is broad understanding of
the Bayer pattern, and many algorithms exist to optimize its
color performance.
When to Use Color
Aside from the obvious applications where the
color of an object needs to be evaluated in some way, sometimes
color can help make an inspection situation easier by
facilitating the identification of objects, such as in the
verification of fuse values in car fuse boxes. But should color
eventually be used in all machine vision applications? No,
because there are some things that monochrome cameras will
always do better than color cameras.
When faced with an application that instills
some doubt, ask yourself the following questions:
• Are the object’s color quality and
consistency key factors in the overall quality of the product?
• Can the object’s color help you ascertain the
relative quality of the product?
• Will color facilitate detection of the object?
• If the answer to any of these questions is
yes, then take a serious look at the color side of machine
vision.
Applications
Let’s look at a few real-world applications
that will help facilitate the decision of whether or not color
should be part of your application.
Food probably is the one application that
everyone understands the best.
As daily consumers, we are constantly judging the quality and
consistency of the food we buy. For fruit, color
allows us to ascertain ripeness and grade product quality. In
the case of grains and legumes, color helps to distinguish
foreign matter in a steady stream of product. In meat
processing, color can detect spoilage and discriminate areas of
fat, bone, and gristle for automatic trimming.
Color machine vision even is used to inspect
the build quality of frozen pizza. With a monochrome image, you
might be able to tell if the density of ingredients is correct.
But you will have a great deal of trouble identifying some of
the chopped ingredients, such as orange, red, and green peppers.
In color, they are easy to tell apart; in monochrome, not so
much.
Color machine vision also is used in
automotive inspection. Although you may think that exterior
paint would be where machine vision is used, the bulk of the
effort goes into inspecting the fine visual details that make up
the user interface, such as ensuring the consistency and
evenness of the instrument panel. This is important because the
look and overall quality of the dashboard go a long way toward
contributing to a driver’s impression of a car.
Obviously, there are many other applications
that could require color, like print quality and registration,
pharmaceutical label verification, part presence and detection,
and PCB assembly. In addition, there’s a slew of quality and
grading applications that involve color and texture
classification for things like wood, textiles, and ceramic tile.
Software and Color Detection
Most color machine vision systems use a mix
of hardware and software to detect colors. Major differences in
the software approaches focus on the classifier that detects
colors and assigns color pixels to a class such as good or bad.
A good classifier has some tolerance to illumination changes, is
quick to train and run, and reliably assigns pixels to their
correct classes. Classifiers are an area of continuing
development and competition among vendors.
The Future
Market studies by the Automated Imaging
Association show that only about 25% of the cameras sold for
imaging applications in 2005 and 2006 were color cameras; the
rest were monochrome. But the trend is upward for sales of color
cameras.
Most experts in the field believe the use of
color will expand in machine vision. Color provides much more
visual detail than monochrome grayscale and adds a new dimension
in analyzing data in the real world.
For example, PCB inspection applications use
color cameras to identify oxidized copper wires that would be
difficult to see in a monochrome system. Color machine vision
also is growing in popularity in bank-note inspection
applications for scanning, processing, and confirming
authenticity. In some Asian countries, color inspection is
required by the government because people use seals rather than
signatures when issuing personal checks. The seals are used
generally with red ink, which has poor contrast in a monochrome
system.
Better color fidelity, lower cost, and ease
of use are the primary drivers in the market. And new
technologies are continually being developed to address these
needs.
About the Author
Robert Howison is project leader for OEM
custom projects at DALSA. He holds a bachelor’s and a master’s
of engineering from Ecole de Technologie Superieure and has more
than 12 years experience in machine vision applications. DALSA
Montreal, 7075 Place Robert-Joncas, Suite #142, St. Laurent,
Quebec Canada, H4M 2Z2, 514-333-1301, e-mail:
robert.howison@dalsa.com