![case $rtval in
1) rm -f /tmp/dirip.$$ ; return ;;
255) rm -f /tmp/dirip.$$ ; return ;;
esac
mfile=`cat /tmp/dirip.$$`
if [ -z $mfile ]
then
mfile=`pwd`/*
SHELL SCRIPT
else
grep "*" /tmp/dirip.$$
if [ $? -eq 1 ]
then
! mfile=$mfile/*
fi
fi
for i in $mfile
do
if [ -f $i ]
then
! echo "$i Delete?" >> /tmp/finallist.$$
fi!
done
3](https://image.slidesharecdn.com/sikuli-google-100611050723-phpapp01/85/Sikuli-Using-Screenshots-for-GUI-Automation-and-Testing-3-320.jpg)
![HOW TO REFER TO A GUI WIDGET?
<AWTTestScript> Abbot
<launch classpath="src/demo" args="[]"
class="CelsiusConverter" method="main" />
<action args="45" method="actionKeyString"/>
<action class="javax.swing.AbstractButton"
args="Convert..." method="actionClick"/>
<assert value="113 Fahrenheit"
component="113 Fahrenheit"
method="getText"/>
by pre-programmed references
<terminate/>
</AWTTestScript>
11](https://image.slidesharecdn.com/sikuli-google-100611050723-phpapp01/85/Sikuli-Using-Screenshots-for-GUI-Automation-and-Testing-13-320.jpg)
![HOW TO REFER TO A GUI WIDGET?
<AWTTestScript> Abbot
<launch classpath="src/demo" args="[]"
Autoit
class="CelsiusConverter" method="main" />
<action args="45" method="actionKeyString"/>
<action class="javax.swing.AbstractButton"
args="Convert..." method="actionClick"/>
<assert value="113 Fahrenheit"
component="113 Fahrenheit"
method="getText"/>
by pre-programmed references
<terminate/> by absolute locations
</AWTTestScript>
11](https://image.slidesharecdn.com/sikuli-google-100611050723-phpapp01/85/Sikuli-Using-Screenshots-for-GUI-Automation-and-Testing-14-320.jpg)
Sikuli: Using Screenshots for GUI Automation and Testing
- 1. SIKULI USING SCREENSHOTS FOR GUI AUTOMATION AND TESTING Tsung-Hsiang (Sean) Chang, Rob Miller MIT CSAIL Collaborator: Tom Yeh 1
- 2. How to tell computers what to do? 2
- 3. case $rtval in 1) rm -f /tmp/dirip.$$ ; return ;; 255) rm -f /tmp/dirip.$$ ; return ;; esac mfile=`cat /tmp/dirip.$$` if [ -z $mfile ] then mfile=`pwd`/* SHELL SCRIPT else grep "*" /tmp/dirip.$$ if [ $? -eq 1 ] then ! mfile=$mfile/* fi fi for i in $mfile do if [ -f $i ] then ! echo "$i Delete?" >> /tmp/finallist.$$ fi! done 3
- 4. 4
- 5. GUI AUTOMATION 5
- 6. 2 3 1 6
- 7. 7
- 8. click click 7
- 9. SIKULI SCRIPT 8
- 10. SIKULI TEST 9
- 11. OUTLINE • Problem - How to refer to a GUI widget? • Sikuli Script - examples and demos • Sikuli Test - demo • Test By Demonstration • Support of testing practices • Conclusions 10
- 12. HOW TO REFER TO A GUI WIDGET? 11
- 13. HOW TO REFER TO A GUI WIDGET? <AWTTestScript> Abbot <launch classpath="src/demo" args="[]" class="CelsiusConverter" method="main" /> <action args="45" method="actionKeyString"/> <action class="javax.swing.AbstractButton" args="Convert..." method="actionClick"/> <assert value="113 Fahrenheit" component="113 Fahrenheit" method="getText"/> by pre-programmed references <terminate/> </AWTTestScript> 11
- 14. HOW TO REFER TO A GUI WIDGET? <AWTTestScript> Abbot <launch classpath="src/demo" args="[]" Autoit class="CelsiusConverter" method="main" /> <action args="45" method="actionKeyString"/> <action class="javax.swing.AbstractButton" args="Convert..." method="actionClick"/> <assert value="113 Fahrenheit" component="113 Fahrenheit" method="getText"/> by pre-programmed references <terminate/> by absolute locations </AWTTestScript> 11
- 15. Use Screenshots! 12
- 16. TEMPLATE MATCHING click( ) 13
- 17. TEMPLATE MATCHING click( ) 13
- 18. EXAMPLE 1: EMPTY TRASH CAN 2 3 1 14
- 19. EX2: FACEBOOK STALKER 15
- 20. EX2: FACEBOOK STALKER 15
- 21. 16
- 22. RESTRICT SEARCH REGION 17
- 23. EX3: MUTE SPEAKERS 18
- 24. EX3: MUTE SPEAKERS 18
- 25. SPATIAL OPERATORS 19
- 26. SPATIAL OPERATORS 19
- 27. SPATIAL OPERATORS (2) 20
- 28. EX4: SET IP ADDRESS 21
- 29. VISUAL-EVENT-DRIVEN PROGRAMMING 22
- 30. VISUAL-EVENT-DRIVEN PROGRAMMING 22
- 31. SIKULI TEST 23
- 32. VISUAL ASSERTION • Verify appearance 24
- 33. VISUAL ASSERTION • Verify replacement 25
- 34. TEST BY DEMONSTRATION detect changes click 26
- 35. Demo 27
- 36. GENERATE ACTION COMMANDS mouse mouse ‘A’ ‘A’ ‘b’ ‘b’ down up down up down up 28
- 37. GENERATE ACTION COMMANDS low-level events mouse mouse ‘A’ ‘A’ ‘b’ ‘b’ down up down up down up 28
- 38. GENERATE ACTION COMMANDS low-level events mouse mouse ‘A’ ‘A’ ‘b’ ‘b’ down up down up down up high-level commands click type “Ab” 28
- 39. GENERATE ACTION COMMANDS low-level events mouse mouse down up ‘A’ ‘A’ ‘b’ ‘b’ click down up down up high-level commands click type “Ab” combine with screenshots 28
- 40. GENERATE ASSERTIONS click 29
- 41. GENERATE ASSERTIONS click 1. find differences / manually hint 29
- 42. GENERATE ASSERTIONS click 1. find differences / manually hint 2. group together 29
- 43. GENERATE ASSERTIONS click 1. find differences / manually hint 2. group together 3. combine with assertExist 29
- 44. GENERATE ASSERTIONS click 1. find differences / manually hint 2. group together 3. combine with assertExist 29
- 45. jUNIT INTEGRATION 30
- 46. TEST-DRIVEN DEVELOPMENT 31
- 47. READABILITY OF CODE 32
- 48. PLATFORM INDEPENDENCE • Works on ANY GUI that can be displayed on Windows/Linux/Mac • Virtual machines • Remote desktop • Mobile simulators: Android, iPhone • Web: Flash, HTML+Javascript 33
- 49. 34
- 50. 34
- 51. LIMITATIONS Theme Variations Visibility Constraints 35
- 52. DISCUSSION Sikuli programs are written against the user interface instead of an API UI: visible, familiar, always exists API: faster, probably more stable 36
- 53. FUTURE WORK • fast and accurate OCR on screen • more robust against different skins 37
- 54. Thanks! http://sikuli.csail.mit.edu or, just google “sikuli” New Sikuli 0.10 released. Follow us on twitter @sikuli Thanks to our funding sources: NSF and Quanta Computer (as a part of TParty project.) 38