Image Recognition
Download as PPT, PDF0 likes964 views
The document discusses an image recognition plugin that allows macros to search for and interact with images on websites. It provides two commands: IMAGESEARCH to find an image on a page, and IMAGECLICK to simulate clicking on an image. These commands use image recognition technology to locate images based on their visual characteristics rather than coordinates. The document also provides examples of using these commands to automate interactions with flash-based chat applications.
Image Recognition
- 2. Image recognition Plugin IMAGESEARCH: Search for an image anywhere on the website, including inside of Java and Flash applets. Very useful to get state information from Java or Flash-applets. IMAGECLICK: Send mouse clicks to an image, not a fixed X/Y position. This development solves the problem of „moving buttons“ in interactive websites. The coordinates of the image are available in the {{!IMAGEX}} and {{!IMAGEY}} variables for further use. The Image Recognition feature provides two new commands that use state-of-the-art image recognition technology:
- 3. IMAGESEARCH command IMAGESEARCH IMAGE=c:\testimage.bmp CONFIDENCE=99 Two cases: Confidence = 100, a high confidence level (100%) means that an exact copy of the test image has to be found on the website. Extremly fast algorithms (less 1 sec) Confidence <= 99, This command will trigger an advanced image recognition algorithm. The lower confidence levels allow for color changes. Search focuses on shape of object (letters, characters)
- 4. IMAGECLICK IMAGECLICK IMAGE=c:\buttonimage.bmp CONFIDENCE=99 CONTENT=Hello<SP>World IMAGECLICK = IMAGESEARCH + WINCLICK The CONTENT= attribute has the same function as for the normal DS CMD=CLICK command. Use it, for example, to fill an input box inside a FLASH applet. If the image is found, the image recognition module will return the coordinates of the middle of found image to iMacros: The coordinates are in the {{!IMAGEX}} and {{!IMAGEY}} variables
- 5. Test Chat Flash Applet - Blue
- 6. Test Chat Flash Applet - White
- 7. Test Chat Flash Applet - Black
- 8. Test Case Function testing the chat applet: Send a text to the input box 2. Click the "Send" button 3. Verify that text was received and is displayed in main window. This can be achieved with the following command sequence: IMAGE CLICK IMAGE= inputbox.bmp CONFIDENCE=99 CONTENT= Welcome<SP>to<SP>Heidelberg! IMAGE CLICK IMAGE= sendbutton.bmp CONFIDENCE=99 IMAGE SEARCH IMAGE= text.bmp CONFIDENCE=99
- 9. Create Input Images “ inputbox.bmp” “ sendbutton.bmp” “ text.bmp”
- 10. Image Recognition Tips Processing time image recognition: 2-3 seconds typically Processing time increases with input image size Processing time 100% matching: < 1 second False detection if confidence level is set too low MS Paint useful for creating input images. MS Paint is part of Windows OS.
- 11. Demo Write a macro that adds 4 plus 1 at http://www.iopus.com/iim/demo/v6/flash Enter „Hello World“ into banner chat at http://www.123flashchat.com/ Create macro that logs into http://mywallop.com/ and automatically adds a new blog entry.