Low-cost, high-performance image sensor for industrial barcode reader applications

Summary:

With today’s trend of lower cost and higher performance industrial cameras, higher requirements are also placed on CMOS image sensors, which need to be achieved by designing a system-on-chip (SoC). To achieve this goal, it is necessary to integrate multiple image processing tasks into a single device through 3D chip stacking and back side illuminated (BSI) technology. In the future, there will be solutions with sophisticated machine learning and proprietary intelligent computing chips combined with image capture capabilities to create compact high-speed computing vision systems.

But before a new large-scale technology integration can be achieved, two major development hurdles must be removed—the chip’s thermal management and power consumption.

Today, advanced front side illuminated (FSI) CMOS sensors integrate analog and digital functions for a cost-effective solution. The key to achieving these goals lies in the clever separation of various factors that are beneficial to system performance and embedded in the image sensor SOC. Here, the role of separating image system applications and existing image processing devices such as CPUs, FPGAs and DSPs is a central factor, as duplication of functions leads to increased costs. To develop a standard image SOC product that can provide a feasible solution for the target application market to achieve mass production (to achieve the lowest production cost) requires an in-depth dialogue between the two modules in advance.

The major market players of end camera products, including hardware, software, system builders and optical engineers, as well as a multi-disciplinary image sensor development team, are contributing their respective knowledge in semiconductor technology and end camera product technologies and applications, Find innovative product solutions.

This article will introduce a family of CMOS image sensors for barcode readers and other embedded vision applications, their application examples, and some future trends.

Barcodes – an overview of today’s most popular coding systems and associated reading technologies

In industries that rely on holiday and seasonal consumption for retail and online shopping, industries such as logistics, manufacturing and wholesale scan more than 5 billion barcodes every day. With the world’s first barcode appearing on a pack of chewing gum in the 1970s, it is now clearly the most popular readable coding system, with new applications constantly developing. 1D barcodes were the first barcodes to appear (and keep getting smaller due to advances in printing technology), and they are still the mainstream technology found in UPC (Unique Product Code) applications in retail, transportation and logistics, and other industries. 2D barcodes come in a variety of different specifications, providing more programmable data than 1D barcodes: 1D barcodes can be loaded with a maximum of 20-25 characters, depending on the barcode type; 2D barcodes can be loaded with a maximum of more than 2000 characters character. In addition to the general function of writing product information and details, 2D barcodes also write checksums and other correction techniques to ensure greater tolerance for misprinted or damaged barcodes. Two-dimensional codes have been widely used in some specific industries, such as automated manufacturing industry, direct part marking (DPM) of parts, etc.

The 2D barcode reading technology upgrade started about 15 years ago as it was able to read both 2D and 1D codes and became mainstream in today’s market.

Not all QR code readers have the same functionality

Barcode verification and decoding systems are rapidly evolving and improving to provide faster, more compact, lower prices and more powerful reading capabilities.

While laser-based 1D readers are still in production and in use, the most significant technological advancement in reading systems has come from 2D readers. The 2D reader through the image sensor enables it to provide significant computing power, bringing additional functions that were not possible before. These capabilities include taking photos and recording videos, as well as adding more advanced features, such as document scanning, optical character recognition (OCR), object recognition, size measurement, and many more.

Image sensor of Teledyne-e2vis a unique product for this market, offering more than a variety of 2D sensor options. One of the main reasons is that it was designed specifically for barcode reading, rather than a product generally geared towards the multi-purpose, consumer or automotive market. This means a precise and powerful solution that meets all the requirements of market leading barcode reader products.

Teledyne-e2v has recently developed a series of low-noise global shutter CMOS image sensors with small pixels, which have unique functions and can bring applications to the Automatic Data Collection System (ADCS) and Automatic Identification (AI) markets. Solutions with significant cost savings and/or performance improvements. In this market segment, although sensor unit cost is the most important factor, cost reduction options such as lighting/optical lenses also need to be considered.

QR code reading systems require very fast frame capture to avoid smearing. This requires the shortest possible exposure time. On the other hand, to obtain maximum depth of field (DOF) or scanning range, lenses with very small optical apertures (typically F/8 or smaller) are often used. The very small number of photons that can enter an image sensor pixel, combined with the short integration time, means barcode reading is possible in low-brightness applications (see Figure 5). The global shutter also facilitates reading moving barcodes.

The main sensor parameters that affect terminal reader performance are therefore particularly suitable for barcode reading applications. Figure 4 lists some of the key sensor/barcode reading performance requirements and demonstrates the benefits of the Snappy sensor family as an example of application-specific CMOS image sensors.

Figure 4: Snappy’s main specifications for barcode scanning functionality

signal to noise ratio

Irradiance (photons/pixel)

Figure 5: The low luminance signal-to-noise ratio of the Snappy sensor offers advantages in reducing system lighting optical energy consumption and cost

The effect of temperature rise

If you take a closer look at the differences between the various components that make up the noise parameters at 25°C, and how those components behave at >65°C temperature rises, there are limitations in the performance of some of the components‘ parameters, which are limited in the sensor The effect of temperature rise should be considered in the selection process. Spatial row-column fixed read noise is one of the particularly important parameters for barcode reading. Considering that fixed pattern noises are similar in shape to straight and horizontal lines, they are easily confused with barcodes, or can add erroneous information to barcode readings in images.

The Snappy series of image sensors use an advanced semiconductor process, with only a few dark signal photons at 25°C, and only 77 photons per second even at 65°C. This helps the row and column embedded fixed pattern noise cancellation algorithms to achieve only a few percent fixed pattern noise even at high operating temperatures.

Very low readout noise (combining temporal and spatial elements) is typically 3 photons. It will not deteriorate even in a high temperature environment. If the sensor performance degrades at high temperature, that means more lighting is required, and the introduction increases the system cost.

Unique black and white + color pixel filter mode—combines the advantages of low-light and high-sensitivity data of black and white and color pixels

The sensor can use colored pixels to facilitate adding additional object/label recognition capabilities, providing additional security features to avoid spoofing or situations where the barcode itself cannot be read. However, because of the lower transfer characteristics of the organic color filters of color sensors and the need to combine red, green, and blue pixels to create “color” pixels, this means that color chess has relatively low transfer characteristics compared to monochrome image sensors. Low spatial resolution and lower sensitivity. Teledyne-e2v’s Jade image sensor is an interesting innovation that uses monochrome pixels but adds a color pixel to every four monochrome pixels. This preserves the spatial resolution and sensitivity critical for reading barcodes, while allowing lower resolution color images to be captured simultaneously.

Figure 6: Innovative color-sensing applications do not need to compromise reading performance

Innovative Embedded Application-Specific Features

Achieving fast (Snappy) barcode reading is not just a result of frame readout rate. While stationary noise is a limiting factor, Snappy sensors make no compromises for this. The sensor offers excellent performance with an 8 bit depth approaching 120 frames per second. A unique power-on mode ensures that the device is powered on or on standby when the first image (or fast self-exposure sub-image) is captured within the signal-to-noise ratio specification. This is not a standard feature of a typical rover or global shutter CMOS sensor for other applications, as it means that the system must discard multiple full-frame images before fully settling and reaching the SNR performance stated in the datasheet. This unique ability to read the first frame after power-on can provide a differentiating factor for the camera, achieve the highest speed barcode reading, provide end users with “snapshot” scanning, and achieve higher productivity for enterprises. The following describes two of the most innovative and patented capabilities of the Teledyne-e2v imaging team in the Snappy sensor family: They are designed for ultra-high-speed barcode reading, identification and decoding applications for end product scanning.

1. Fast Self Exposure (FSE) mode(for Snappy 2MP and 5MP CMOS sensors):

Quick self-exposure modeAllows exposure times to be optimized under varying light (see Figure 7). Compared to the traditional auto exposure mode, FSE brings the advantages of more fusion time and powerful functions, including full user programming and providing end users with the advantages of stable and fast reading, adaptive to any light source or dynamic light source environment, and the frame rate. Almost no effect.

Figure 7: Novel on-chip auto-exposure method for barcode reading and all machine vision applications

The patented FSE mode uses multiple on-chip components to achieve the following functions:

(a) A unique vertical analog-to-digital converter (ADC) allows 4 different exposure periods to be set in successive row segments, which are then repeated across the array, producing 4 low-resolution images with different exposure values. This function can also be used as a powerful high dynamic range image capture function.

(b) Lateral incremental subsampling, max 1/64 line

(c) On-chip statistics include saturated pixel values, and provide a 16-bin histogram output, which can directly read the data of the frame or area in the image footer

(d) Viewfinder window (ROI) mode supports FSE subframe, multi-region, and region-in-region

(e) Fine control using histogram values, average values ​​and combinations of the two

(f) Programmable buffers provide intuitive user controls and settings

These features bring scanning speed advantages to the end application, as FSE mode typically only uses less than 10% of the frame period. Traditional embedded automatic exposure control (AEC) for other CMOS sensors uses an asymptotic technique to avoid flashing and provide target images, making image fusion slower. The whole process consumes a lot of frames, so that the speed cannot meet the requirements of barcode reading applications.

2. Smart ROI (Smart ROI) mode (for Snappy 5MP sensor):

Smart Viewfinder (Smart ROI) Use on-chip algorithms to detect one or more barcodes on the image. The barcode decoding image processing system needs to separate the range containing the barcode from the rest, so as to process the useful part. This work is typically performed on an FPGA or CPU, as this task requires a lot of gate/real-time clock (RTC) and processing power, resulting in additional high costs and complex technical constraints in choosing a processing engine.

Figure 8: Detection of multiple moving barcodes on the image sensor using the Smart Viewfinder (Smart_ROI) feature

Embedding this barcode detection function into the sensor can realize overall cost savings, not only because the processing overhead is significantly reduced, but also because the task is completed in the sensor, and no other digital signal processing is required to realize the system. Performance and stability benefits. 1D or 2D codes detected in valid frames will be in block form with X/Y coordinates added as part of the readable information in the image footer area (invisible). The sensor can detect multiple fields (or barcodes) simultaneously, and can also detect other codes, such as reading printed characters in optical character recognition applications (OCR). This feature works even in applications where barcodes/objects/cameras are moving.

Significant cost savings and system simplification are the main advantages. The 5MP sensor is primarily used in high-end barcode applications because it requires a larger lens and more processing power to perform 5MP real-time image processing/decoding, which negates the major advantage provided by a larger scanning range or area. However, the small pixels of the Snappy family of sensors and the processing overhead savings brought about by the on-chip smart viewfinder window allow for smaller system-level costs. Due to its low power consumption and the ability to recognize multiple barcode symbols in a single frame, this product is becoming a market driver for high-resolution sensors, and it is an obvious advantage in the e-commerce logistics industry.

Fast self-exposure and smart viewfinder functions work simultaneously on the Snappy 5MP sensor to ensure fast and powerful operation in environments with changing ambient light.

The Snappy sensor family is optimized for low-cost, low-power system needs. The sensor, also mentioned above, processes data on-chip, and the work performance can be significantly improved. While satisfying not only barcode reading applications and markets, but also other types of machine vision (MV) applications, including inspection, metrology, optical character recognition, and more, can also benefit from the performance and embedded capabilities of Snappy sensors. Other applications include embedded vision systems, IoT edge devices, drones, augmented reality, biometric systems, and more.

Figure 9: Snappy sensor family for other machine vision, smart IoT and other industrial computing vision applications

Snappy sensor integrated lens and advanced micromotor-based autofocus provide added value

Consumer-grade camera optical modules are not suitable for the harsh long-term work environment of B2B industrial applications. In addition, for example, lenses used for barcode reading generally have specialized performance and optical characteristics that require maximum working distance, while optical aberrations such as optical modulation transfer function (MTF) need to be minimized in order to effectively detect features Barcodes with a size smaller than the Nyquist frequency are decoded. The MIPI Optical Module (MOM) integrating a powerful Snappy image sensor and high-performance lens can increase the value of the system and save development time and cost.

Figure 10: MIPI Optical Module (MOM) using an integrated fixed focal length lens and Snappy sensor

MIPI Optical Module An ideal solution for embedded vision applications, it allows the use of custom lenses to a certain extent, providing excellent performance and flexibility in a small 20mm x 20mm module. Teledyne-e2v is now sampling the first 2MP MIPI optical module to end users, and plans to launch a 5MP version, as well as a powerful and lightweight 2MP autofocus version based on micro-motor technology. The biggest advantage of autofocus is that it allows the use of larger optical apertures to achieve comparable or better scanning ranges or working distances compared to fixed focus optics, but requires significantly less illumination power consumption. Future 2MP MIPI optical modules will provide more performance/or cost improvements for industrial imaging applications. Details of the new products will be announced in mid-2020. However, there is now an evaluation platform using the new open-loop ‘multi-focus’ function, which provides the largest working range and maximum frame rate, and has a MEMS autofocus component that can be used in the existing Snappy 2MP demonstration kit.

New trends and changes brought about by the explosive growth of e-commerce

The huge double-digit compound annual growth rate (CAGR) target (more than 25% per year) brought about by the explosive growth of e-commerce has not only brought about changes in logistics center operations, but also provided protection for traditional brick-and-mortar retail points. The retail market is facing a dramatic shift to provide a better customer experience and reduce checkout times through the use of unmanned automated ‘self-scanning’ systems. The key to the success of these systems is not only reliable barcode recognition and decoding capabilities, but also more sophisticated object recognition tasks that require more use of color imaging devices.

Figure 11: The explosive growth of e-commerce has led to new trends and changes in CMOS sensor requirements and capabilities

To fully realize the high growth potential, the first condition is the realization of higher speed or higher throughput scanners and cameras. Higher resolution sensors with wide frames will allow for faster reading speeds and larger surface area readings (including multiple packages and barcodes in the same image), so that in the future a single sensor can cover the entire warehouse area.

Regarding image sensors, we see the advantages of flexible sensor technology applications, which can reduce the complexity of external optical components and alleviate some of the existing limitations of small pixels such as the diffraction limit of mirrors. This development can provide two advantages, one is to simplify and save the optical cost, and the other is to allow small pixels below 2.5µ without reducing the MTF and still reach the optical diffraction limit of the lens.

In warehouse applications, the need for a small package and scanning the barcode on the package often stems from the ever-increasing efficiency goals of e-commerce distribution centers. Every square centimeter of storage and shipping space needs to be maximized. To achieve 3D dimensional monitoring of the entire shipment and supply chain, so that the relevant code/text label can be read through the QR code, it is now necessary to use two separate cameras, one for 3D (mostly using structured light sources or 3D-based technology for stereoscopic vision), while the other uses non-connected 2D cameras.

Current research focuses on developing a 2D and 3D CMOS sensor that can simultaneously provide traditional 2D images and 3D point clouds. Teledyne-e2v is committed to providing cutting-edge leadership technology for these new market segments, and has developed future product roadmaps and intellectual property that can be matched with next-generation cameras and imaging systems, enabling related technologies to focus on application-specific needs. Anticipate the advent of new products.

The Links:   DMF-50262NF-FW-ME   MG200Q1US41   SHARP-LCD