Apqt Vision Algorithms
- 1. Print Quality Striping/Streaking due to: Missing/Weak Nozzles PAD Theta-z rotation Swath Height Error (SHE) Edge raggedness due to: Tails Image Quality Banding due to: Shape SAD Dot size/shape variation Theta-z die rotation bi-directional tails SHE Granularity due to: Mean SAD Dot placement "Greyscale" variation due to: Variation in dot size/drop volume Bi-directional tails Dot Greyscale APQT: KUKLA / KOLOR/ HIVAS / CANDIDE ALGORITMS: Don Welch Page 1 1/23/2010
- 2. FIRST PATTERN SECOND PATTERN OTHER FEATURES STARTING COLUMN ENDING COLUMN A StairStep pattern has been illustrated to shown how pattern FEATURES look in the profiles. The second pattern is obscured from view by the profile plot. Disregard the RED vertical lines, they are left over from the light level adjustment and are of no consequence here. The Profile illustration in this document is two fold, one, to illustrate the primary vision functions used by the KAPQT, and two, to show which row and column the patterns are registered at. These row and column positions are used by the programmer to adjust sizes and locations of evaluation windows. The Row profile is an organized representation of each rows gray level pixel sum from rows 0 - 240. Likewise, the column profile is the data representing , either plotted or stored in memory, of each columns gray level pixel sum, from columns 0 - 256. A “PIXEL” is a 6 bit word (values from 0-256) representing gray level or light intensity for each particular pixel. By summing all of these pixels (256 columns) for each row, a total gray level sum is developed for each row (240 rows total), and stored in memory for latter analysis or plotting. By summing the pixels together for each column (240 rows) a total gray level sum is developed for each column (256 columns total), and stored in memory for latter analysis or plotting. Don Welch Page 2 1/23/2010
- 3. VALLEY VALUE VALLEY WIDTH VALLEY PRESENT ROW PROFILES The “Valley Value” is a parameter that defines the depth for each valley and the “Valley Width” parameter defines the valley width. Because the application depends on contrast to evaluate image features, any lighting variation can change the contrast of the images and make the valleys appear shallow if the lighting contrast is too high or appear very deep if the lighting contrast is too low. BLACKOUT / BANDING Don Welch Page 3 1/23/2010
- 4. The first step that can be viewed is the BlackOut registration test, which defines the position for the evaluation windows. Second, is the Deprime test, where smaller rectangular boxes are positioned over the BlackOut patterns. These smaller boxes are located at a fixed positions in the cameras field of view, and do not move. A sum of all the gray level pixel values of this area determine the Deprime Test PASS / FAIL criteria. Thirdly, the Edge Detection generates the white outline of the pattern for the white banding algorithms. Finally, the row profiles are performed on the resultant edge detection’s to determine qualifying peaks, representing white banding. A row profile is performed on each of the white pattern outlines of the Edge detection, of which, yields a “PEAK” for the top and bottom of the BlackOut pattern. The Peaks must meet criteria for width and value to qualify as a “WHITE BAND”. Because the top and bottom peaks are White Bands, there must be at lease two peaks for the White Band Test to pass. If an actual white band occurs because of trajectory or starvation errors, more peaks will be present within the BlackOut pattern. If these peaks also qualify based on PEAK WIDTH and PEAK VALUE, they will be detected as well. Any number of peaks greater than two will cause the BlackOut test to Fail. A long horizontal line is drawn across from each peak indicating that it qualified for Peak Width and Peak Value. If a particular peak did not meet the qualification criteria, the line will be missing. If another peak appears somewhere between these two top and bottom peaks, a white band has occurred, representative of trajectory or starvation errors, and the primitive will fail. Don Welch Page 4 1/23/2010
- 5. MAXIMUM PEAK This option provides additional information for the MAXIMUM PEAK. PEAK WIDTH is how wide the MAXIMUM peak is. MAX VALUE is how high or long the MAXIMUM PEAK is. The MAX POSITION is where the MAX PEAK was found inside of the black evaluation window. For example, 4 means that the UPPER or TOP PEAK was the MAXIMUM PEAK. This function is used to ensure the pattern is registered or that the MAGENTA evaluation window for the primitive is positioned and spaced over the pattern correctly. Likewise, with the “CHAR” button selected, the row profiles for the evaluation window can be monitored to see if the “PEAKS” representing the TOP and BOTTOM of the BlackOut pattern outlines are detected. By monitoring the Row Profiles for performance, different PEAK VALUES and PEAK WIDTHS can be tried for test characterization. Don Welch Page 5 1/23/2010
- 6. DEPRIME WINDOWS TOP ROW RIGHT COLUMN GREEN CROSSES REPRESENTING REGISTRATION Like the StairStep test, the green crosses represent the top row and right columns of the where the algorithm thinks the test patterns are located. When the Magenta pattern is registered, two different edge detection windows are developed from the top row and right column data. Due to the fixed geometry of the pen, location, spacing, and size of BOTH windows can easily be determined. Thus, if the magenta registration is wrong, the cyan registration will likewise be wrong and fail the evaluation as well. If the “DEPRIME” button is selected smaller rectangular evaluation windows positioned at predetermined locations are placed over the BlackOut patterns. The smaller rectangular windows are used for the DEPRIME TEST and the ILLUMINATION CHECK. If the gray level sum for each particular window is LESS than the value specified by the “DEPRIME” parameter in the limits window, the Deprime test for that particular primitive will PASS. Similarly, if the gray level sum of each window is GREATER than the value specified by the “ILLUM” parameter for the primitive, the Illumination check will PASS. The smaller evaluation windows are positioned at a fixed location in the cameras field of view, and do not move. Normally, the Deprime failure mode is not seen because, the APQT Prefire station screens all Deprime failure before the actual vision print test. The only time this failure mode occurs, is when the paper does not track on the paper rollers properly. Likewise, when the patterns come under the cameras view too high or low, or to the left or right too far, the failure mode will result, indicating the camera alignment is off in the X or Y axis. Don Welch Page 6 1/23/2010
- 7. NOZZLE #11 FAILED FAILED “PEAK” DETECTION MISS-REGISTERED For the BlackOut test above, the lower peak was not detected for each of the patterns, as indicated by the missing long horizontal line. The StairStep pattern was miss-registered, but most of the line segments were still found, except nozzle #11. It is likely that the VALLEY WIDTH or VALLEY VALUE parameters are set wrong. MONET: Description of Blackout Test, lighting dependent. Threshold Passed Normal Banding? Peaks Representing WhiteBars Failed Don Welch Page 7 1/23/2010
- 8. Row and Column profiling are the primary (only) analysis algorithms used by the HRAPQT, illustrated above. Pass and Fail criteria is based upon “Thresholds”. Unfortunately, row and column profiles can be manipulated very easily by light changes. Kukla and Robinhood utilized an edge detection algorithm that normalized the lighting differences to test the Blackout pattern which is very similar to the New “BlackLine” test developed by David Collins. These Profiling algorithms are easily written but hard to maintain, they are characteristic of apqt testing previously done in San Diego on earlier “INTRO” Lines and have influenced the new HRAPQT’s. Our Vertical Lines and Blackout test suffer the most from these algorithms. This is why there was a push years ago to ‘fix’ the apqt test. Fortunately for us, there is a FIX and it has demonstrated itself testing the ‘Yellowstone’ Product. It has also been used to test Monet Pens ( 2nd and 3rd Pass) on Canvas I. Similarly, it has already been used to test many thousand NITRO Pens, (pens built on Nitro but, taken over to Canvas for Testing). Inherent problems with calibration procedures... Hum, where do I start? Procedural problems that will most likely be inherent with ANY calibration used are: Discovered more by accident, we found that when the apqt is calibrated with the doors opened, which was normally the case, (it allowed the technicians to move around more freely) the calibration had a different grading response than when the calibration was performed with the doors closed. Ambient light was biasing the calibration.... Also discovered, was the fact that if the calibration was performed on a calibration pattern that was scewed ever so slightly, in the clockwise direct, the grading response was again very different. The failure mode seems to be more or less concentrated on the rows it spans, depending on the orientation. Likewise, the paper from the Calibration pattern does not match our actual viper+ brightness, and the resulting “peaks” from the row profiling can and will appear to be larger or smaller depending on how the calibration paper reflects light. These findings combined with the fact that the “printed calibration patterns” now used by Canvas and Nitro are NOT representive of our actual printed patterns. Likewise, the calibration standards are not traceable to ... well, any - thing... The cal patterns can only get us in the ball park, the fine tweeks will be a nessaccery evil until we get our new calibration standards The Vertical Lines test has been the most difficult to maintain because of the light dependent algorithms that the test uses. This has been overcome with a new test developed last year by David Collins. Extensive characterization had to be done for Zaphod, so Dave Collins developed this new test and new features for it. A new test pattern was also created to take advantage of the new testing capabilities. The blackout test hasn’t been effective for discovering “Jack Frost” fail modes. Jack Frost is a group of nozzles that are slow to startup because of air pockets, etc, an effective test for ‘bag’ products than for Foam Products. The primary test improvement is: the Blackout pattern and Vertical lines pattern have been combined to form one test pattern. By combining the test patterns, magnification could be increased 2x by evaluating two “blackline” patterns instead of 1- Vertical Line and 1- Blackout pattern by different cameras. NEW “BlackLine” Test. The new vertical test algorithms is based on Edge Detection instead of Row Profiling. Difference being, that Row profiles can change dramatically based on light changes or light distribution across the field of view. Edge Detection looks only for ‘changes’ between light and dark to detect “features”. So the lighting can change across the filed of view in alot of different ways but only the ‘normalized’ changes are seen by the vision system. The StairStep Test and the BlackOut Test has remained the same. Which means the risk for change is reduced , because the stairstep and blackout test results utilize the same algorithms as before Don Welch Page 8 1/23/2010
- 9. and are used for the blackout section of the new “blackline” test. In other words, the only change is to the Vertical Lines Test! Good setup: light is even Bad lighting setup: top is darker An “Edge” is defined to mean, when profile is created, the profile is then scaled to 255(or normalized from 0 to 255). The modified profile is placed in a window. The Intelledex function, vmnedges, is run on the window, and the edges are found with sub pixel resolution. An edge is defined as the zero crossing of the second derivative. This method is much less sensitive to lighting . Another advantage of this edge based method is that it is significantly faster. Intelledex has optimized the code at the assembly language level. Because of speed issues the edgebase algorithms will most certainly become the fundamentally algorithm used by all of our newer tests, that require better resolution for varying die loads, smaller drop size & weight, and an overall need for an better characterization tool. This will also give our Core Technology a baseline for newer Operating Systems, (contact: Tim Hubley or Ken Tubbs) that contain alternative DSP programming on a PCB instead of chassis vision engine now used in manufacturing. By utilizing this test now on the Nitro line, we will be able to qualify for FLASH, and have any of the algorithms’ weaknesses identified and enhanced before a complete rollover to a “Common” tester proposed by Craig Olbright. ADVANTAGES. • Parameteric data (more than go / no go) • Increased camera magnification • Decreased cycle time • Lighting control for the image acquisition • Improved resolution for lighting • Improved resolution for trajectory errors and line width • Improved display management (necessary at 45 parts per minute) • R&R study based on real pens and parameteric data • Ability to measure SAD • Auto ColorMix calibration for registration and process. • Real World calibration and Test measures that report in microns. • Improved Data Logging and Data reporting. • Automatic Signal / Noise evaluation, (Manual procedure has been automated). • Colormix characterization tools. • Edge based processing instead of gray level profiles. • 2x magnification for the vertical lines test, (better resolution). • Test algorithms utilize vision system hardware, Vision A/D gain, etc.. The chief benefit observed from this change has been the repeatability of calibrations from one Hrapqt to the other. Meaning, the process limits generated for one Hrapqt has can be applied to the other Hrapqts. This can only be accomplished by the pixel to world calibration preformed by the “blackline” tests prior to the grading calibration. Since the pixels in the (FOV) Field of View are transformed to microns, this makes all measurement independent of magnification. So if we determine that a failure mode Don Welch Page 9 1/23/2010
- 10. should be 50 um then each Hrapqt would measure it’s own failmodes based upon it’s own pixel-world transformation and fail at 50 um. A new pixel-world calibration must be done prior to testing, it is accomplished with a reticule grid, by placing the grid under each ‘BlackLine’ camera and selecting real world calibration instead of automatic. A calibration standard for the default pattern and algorithm based upon an etching process somewhat different that what was used for the Yellowstone calibration standard. Where as, The Nitro/default standard opted for default chrome patterns etched on glass, recommended by the mask designers. The default chrome actually reflected light much differently compared to the process used by Yellowstone. This was observed first when the targets were first imaged by our Nitro Hrapqts. Coupled with the high reflectively of the chrome and the mono-chromatic wavelength of the light (filtered light) the optical resolution was severely reduced, in other words, the small failmodes built into the calibration standard could not be seen by machine vision, rendering the calibration slides useless… Where, Yellowstone had an extra etching process preformed on the standard by Raynard Corp. of Santa Clara, Calif. to match the reflectively (reflectance of light) of our inks. A TRUE calibration for our color products could not have been accomplished without understanding this particular optic phenomena and climbing the learning curve. Zaphod Two significant paradigm shifts have occurred. At first, the changes may seem rather simple, but the power of these changes will become more obvious. The first major change is that the patterns I have Don Welch Page 10 1/23/2010
- 11. created do not fit completely under the camera. Since the blackout pattern no longer is confined to the field of view of the camera, it can be printed for a longer period of time. Thus, I have increased the likelihood that I will be able to induce area fill problems that take time to develop. The intent is to highlight defects similar to Triad starvation and jack frost. Also, since the entire blackout region was not in the camera’s field of view this left room for the vertical lines to be incorporated into the pattern. The Tiger HRAPQT has three blackout patterns, three vertical line patterns, and six stairstep patterns. By combining the blackout and vertical line patterns into one pattern, I freed up three cameras. The three additional cameras were used to increase the magnification. Instead of three blackout and three vertical line patterns per pen, four black line patterns per pen were used, and instead of six stairstep patterns per pen, eight double stair patterns per pen were used. Thus, each camera was viewing a smaller portion of the pen (i.e. higher magnification). The following picture represents the first black line pattern, the first double stair pattern, and the fifth double stair pattern. Three cameras would be used to analyze the picture below. The black line pattern is on the left. The first double stair pattern is in the center, and the fifth double stair pattern is on the left of the image. The second significant change was with the stairstep. One of the challenges with the stairstep pattern in the past has been registration. The camera magnification used on the stairstep pattern is roughly twice that of the black line pattern. Thus, any registration problems are magnified on the stair step pattern (i.e. a one mm registration problem is twice as many pixels on the stairstep pattern camera). During my development Don Welch Page 11 1/23/2010
- 12. work, I noticed that the HRAPQT was more prone to misregistering images in the paper pull direction. Thus, the stair step pattern that I developed is in fact two stairsteps patterns printed end to end. This provided for significant registration tolerance in the paper pull direction. This also allowed for better utilization of the image space. Very little space is reserved on the left and right margin of the double stair pattern for registration. (Note: The print time has not changed from the Hobbes HRAPQT. The length of paper that the images are printed on is the same on the ZAPQT and the Hobbes HRAPQT.) The Double Stair Pattern The double stair pattern has several new features to it. First, as described above, the image can be registered with much greater flexibility. The pattern is in effect two stairstep patterns printed side by side with a thick vertical line separating the two images. The pattern also incorporates several vertical lines throughout the pattern to improve the registration of the columns of nozzles. This improved registration is necessary because each stair has two pieces. For the first half of each stair the nozzle is only printed for every other dot. In other words, for the first half of the stair the nozzle is only printed at half the nominal operating frequency. For the last portion of each step, the nozzle is printed for every dot -- at the nominal operating frequency. This enables detection of trajectory errors that are frequency dependent. (The Hobbes stability problem results in frequency dependant trajectory errors.) The picture below represents one double stair pattern. The entire pattern is not in the ZAPQT camera’s field of view at the same time. The image analysis for the double stair pattern will be described later. Don Welch Page 12 1/23/2010
- 13. The Black Line Pattern The black line pattern is a combination of the vertical line pattern and the blackout pattern from the Tiger HRAPQT. The vertical line portion of the pattern is significantly different from the original vertical line pattern of the HRAPQT. Instead of printing all of the nozzles in a vertical line. The first vertical line contains only even nozzles, and the next vertical line contains only odd nozzles. Thus, every other line is even, and the rest are odd. This enables a measure of SAD to be made. If the even nozzles are shifted with respect to the odd nozzles then the pen has SAD. (I am using term SAD to describe a more global horizontal trajectory error.) This pattern also enables the vision analysis to sort out the difference between trajectory errors and tails. The following picture is an example of the black line pattern. The entire pattern will not fit in the ZAPQT camera’s field of view. Edge Based Profile Search Most of the APQT vision analysis is simply an attempt to locate the transition from paper to ink. In the past, to find an edge in an image, a row or column profile was created. A threshold was determined by the light levels in the image and the type of edges. Then, the profile was searched for threshold crossings. Anything above the threshold was considered paper, and anything below the threshold was considered ink. Don Welch Page 13 1/23/2010
- 14. This method has several inherent weaknesses. The first weakness is that the algorithm is very sensitive to lighting variation across an image. The second weakness is that the resolution of the algorithm is one pixel. I have implemented a method that I learned from Ken Tubbs for locating edges in a profile. A profile is created. The profile is then scaled to 255(or normalized from 0 to 255). The modified profile is placed in a window. The Intelledex function, vmnedges, is run on the window, and the edges are found with sub pixel resolution. An edge is defined as the zero crossing of the second derivative. This method is much less sensitive to lighting (see the pictures in the Electronic Iris section for verification of this.) Another advantage of this edge based method is that it is significantly faster. Intelledex has optimized the code at the assembly language level. By incorporating this technique, I was able to increase the resolution of the vision system to less than a pixel. (At the end of this report I have included an initial R&R study of the algorithms. Many of the measures had 3 sigma variations of less than a pixel from system to system.) This technique also enabled me to generate parameteric values for the vertical line analysis. 7 Bit Image Processing The vision processing in the past has all been done with 6 bit image processing. With relatively little effort, all of the vision processing can be done with 7 bit image processing. The advantage of this will be most apparent for APQTs that attempt to test pens with light dye loads (Yellowstone), but this should also increase the resolution of the vision processing on other APQTs as well. Double Stair Image Analysis In the past the stairstep pattern has proved to be the most useful. Thus, the pattern that I am using preserves all of the information that had previously been contained within the stairstep pattern. In order to analyze the image, the top and bottom edge of the image are located. Next, the vertical lines within the image are located. Once the widest vertical line is identified the step columns are located. Then, each step is found, first the right half of all the steps is found, then the left half is found. The true position of each step is based on a least square fit to the left side of each stairstep. The odd and the even nozzles are treated independently. (I do not want to measure head alignment at the APQT. I have too many fixtures.) Once the true position is identified, the trajectory error of each step can be determined. At this time, the maximum and rms trajectory error for both the even and the odd nozzles are reported. The minimum, maximum and average height for both the even and the odd nozzles are also reported. The algorithm’s ability to detect spray is not very repeatable at this time. The yellow lines in the pictures represent spray. The ideal position of the nozzles is not displayed in the images below in order to increase the clarity of the images. The following five pictures represents one double stair pattern that is graded from five different registration positions. The purpose of this set of pictures is to demonstrate the flexibility in registration and the importance of printing the pen at both a low and a high frequency. The thick vertical bar in each image is the same bar. These pictures demonstrate the pattern’s robustness to registration. Don Welch Page 14 1/23/2010
- 15. Blackline Image Analysis As with the double stair pattern, the top and bottom edges are located first. Next the edge of the blackout patterns are found. Then, the vertical line region is divided into about 40 horizontal regions. Then all of the edges are found. The algorithm reports minimum, average, and maximum width for both the odd and the even nozzles. It also reports the maximum and rms trajectory error from a least square fit for both the odd and the even vertical lines, and an average measure of SAD is computed. The blackout portion of the blackline pattern is being graded as was done in the past. The blackout pattern has not been useful for the Tiger HRAPQT. Any defect detected in the blackout region has already been detected in the stairstep or vertical line pattern. The following picture represents the black line pattern. The blue lines are the location of the nozzles in regions where all the vertical lines were found, and the small red lines are the nozzle location in regions where nozzles were missing. The long red line indicates the least square fit, and the green line represents the deviation from the perfect line. Don Welch Page 15 1/23/2010
- 16. Don Welch Page 16 1/23/2010