SlideShare a Scribd company logo
10
Most read
11
Most read
12
Most read
RPA Kickstarter Day 3
Ui Automation & Selectors
Lovely Sinha
2
1. Introduction to User Interface
Automation
 What is UI Automation?
 How does UI Automation Work?
2. Selectors and Ui Explorer
3. Input Activities and Methods
4. Output Activities and Methods
Agenda
3
User Interface (UI) Elements :
All graphical user interface pieces that construct an application, be they windows, checkboxes, text fields or drop-
down lists, and so on.
UI Automation:
The process of interacting with graphical elements within applications by simulating mouse and keyboard controls.
There are majorly two interactions we can observe in UI Automation
 Input: Inserting data into an application
 Output: Fetching data from an application or Ui Elements
4
UI Automation Activities
Latest Package Available in
Manage Package Section
Activities can be split into:
•Containers - These are activities that
identify the browsers or apps the
process needs to interact with. All
activities included within a Container
will execute on the same
application. Some examples are Open
Browser, Attach Browser, Open
Application, or Use Application/Browser
•Input Activities - These activities
send input to UI elements. They can be
used to click, check, type into, send
hotkeys, and so on.
•Output Activities - get structured data
or get UiElements containing images.
•Synchronization Activities - They
help you create triggers based on UI
behavior, thus enabling the Robot to
execute certain actions when specific
events occur on a machine.
5
Activity Properties
6
Modern Vs Classic Design Experience
7
Example 1: UI Demo Application
Open Demo UI
Application
Enter Data
Click Accept
Get Transaction Number
Input
Actions
Output
Actions
8
Selectors
Unique identifiers of UI elements. These store UI
element attributes which will help identify the UI
element uniquely during automation.
Wild cards
There are two types of wildcards available in UiPath
 * - Replaces any number of characters.
 ? - Replaces exactly one character.
9
UI Explorer
It is the wizard which is used to indicate and edit selectors. If we are unable to locate the individual
element then we can switch to other Ui Frameworks for better results.
10
Example 2: ACME Add Vendor
Open Browser
https://acme-
test.uipath.com/login
Enter Credentials
Click Vendors -> Add
Vendor
Display added vendor
details
Input
Actions
Output
Actions
11
Input and Output Methods
Input Methods
•Hardware Events
Clicks: the mouse cursor moves across the screen. Typing: the keyboard driver is used to type individual
characters. Does not work in the background. The attended User cannot touch the mouse or keyboard during
the automation.
•Send Window Message
Replays the window messages that the target application receives when the mouse/keyboard is used.
Works in the background. Users can work on other activities during the execution of the automated
processes.
•Simulate
Uses the technology of the target application (the API level) to send instructions. Works in the background.
Users can work on other activities during the execution of the automated processes
12
Output Methods
Output Methods
•Full Text
the default method and good enough in most cases. It is the fastest, it can extract hidden text, it has 100%
accuracy, and can work in the background.
•Native
compatible with applications that use Graphics Design Interface (GDI), the Microsoft API used for representing
graphical objects. It doesn’t extract hidden text and it cannot work in the background; and just like Full Text, it
doesn’t support virtual environments.
•OCR
OCR (or Optical Character Recognition) is the only output method that works with virtual environments and
with “reading” text from images. Its technology relies on recognizing each character and its position. On the
other hand, it cannot work in the background, it cannot extract hidden text, and its speed is by far the lowest.
13
UI Automation is the process
of automating application
user interfaces by
simulating human
input and output actions
through specific UI Activities.
The first step in automating UIs
is understanding the logical
sequence of steps that a
human user would take. The
second step consists in
translating these steps into
UiPath Studio Activities and
configuring them.
The complexity of the UI or
of the application is
irrelevant. All desktop/web
applications can be
automated by using universal
or application-tailored
activities.
Takeaways
14
Assignment: Password Generator
Open Browser and Navigate
to URL
https://www.rpasamples.com/
passwordgenerator
Type Password Length
Generate Password
Get Password
Input
Actions
Output
Actions
Post your Unique 10-digit
password with the following
hashtags :
#UiPathCommunity
#RPAKickStarter
“My Buddy Bot generated a
password – [your password]”
15
Q&A
16
Learning References:
Login to https://academy.uipath.com/
Course Name:
https://academy.uipath.com/courses/ui-automation-with-studio
https://academy.uipath.com/courses/selectors-in-studio
Documentation Links:
https://docs.uipath.com/studio/docs/about-ui-elements
https://docs.uipath.com/studio/docs/about-selectors
17
.
Next Session
https://community.uipath.c
om/events/details/uipath-
learners-presents-rpa-
developer-kickstarter-day-
4-recorders-in-uipath/

More Related Content

PPTX
Ui path| RPA
PDF
UiPath Email Automation | UiPath Tutorial | RPA Training Using UiPath | Edureka
PDF
Introduction To UiPath Studio | Edureka
PDF
UiPath Studio Session 2 - The Fundamentals of UiPath Studio - Final Slides.pdf
PDF
RPA Developer Kickstarter Day 11 Best Practices and RPA Lifecycle.pdf
PPTX
Ui path online training ppt
PPTX
RPA Uipath Presentation.pptx
PPTX
Debugging in UiPath.pptx
Ui path| RPA
UiPath Email Automation | UiPath Tutorial | RPA Training Using UiPath | Edureka
Introduction To UiPath Studio | Edureka
UiPath Studio Session 2 - The Fundamentals of UiPath Studio - Final Slides.pdf
RPA Developer Kickstarter Day 11 Best Practices and RPA Lifecycle.pdf
Ui path online training ppt
RPA Uipath Presentation.pptx
Debugging in UiPath.pptx

What's hot (20)

PPTX
Exception Handling in UiPath.pptx
PDF
Introduction to RPA_SummerSchool _ Welcome to the world of automation using U...
PDF
What is Robotic Process Automation?
PPTX
UiPath Community Event - Build more mature automations with Unattended Robots
PDF
Orchestrator - Practical Approach to host UiPath Orchestrator
PDF
What is UiPath RPA Architecture | UiPath Studio, Robot & Orchestrator | Edureka
PPTX
Getting started with RPA (uipath)
PDF
Variables, Data Types and Activities in UiPath | Basics of UiPath | UiPath Tu...
PDF
What is RPA Lifecycle? Edureka
PPTX
HyperAutomation (3).pptx
PDF
Introduction To UiPath | RPA Tutorial For Beginners | RPA Training using Uipa...
PDF
RPA Tutorial for Beginners | RPA Training Using UiPath | UiPath Training Onli...
PPT
software effort estimation
PPTX
Enhanced Reframework Session_16-07-2022.pptx
PPTX
Introduction to Robotic Process Automation
PPSX
Robotic Process Automation
PPTX
UiPath Development Best Practices.pptx
ODP
Fitts' Law Basics
PPTX
Socio organizational issues ppt
PPTX
Robotic process automation Introduction
Exception Handling in UiPath.pptx
Introduction to RPA_SummerSchool _ Welcome to the world of automation using U...
What is Robotic Process Automation?
UiPath Community Event - Build more mature automations with Unattended Robots
Orchestrator - Practical Approach to host UiPath Orchestrator
What is UiPath RPA Architecture | UiPath Studio, Robot & Orchestrator | Edureka
Getting started with RPA (uipath)
Variables, Data Types and Activities in UiPath | Basics of UiPath | UiPath Tu...
What is RPA Lifecycle? Edureka
HyperAutomation (3).pptx
Introduction To UiPath | RPA Tutorial For Beginners | RPA Training using Uipa...
RPA Tutorial for Beginners | RPA Training Using UiPath | UiPath Training Onli...
software effort estimation
Enhanced Reframework Session_16-07-2022.pptx
Introduction to Robotic Process Automation
Robotic Process Automation
UiPath Development Best Practices.pptx
Fitts' Law Basics
Socio organizational issues ppt
Robotic process automation Introduction
Ad

Similar to RPA Developer Kickstarter | Day 3: UI Automation and UiPath Selectors (20)

PDF
Suncoast Credit Union and Armwood High School - UiPath automation developer s...
PPTX
Human Computer Interaction: User Interface Layer
PDF
UiPath Veterans RPA Studio track program - Session 3: Introduction to Ui Auto...
PDF
Hidden Object Detection for Computer Vision Based Test Automation System
PDF
Application for Data Sync Between Different geo Locations
PDF
RPA summer school session 2.2: Run your first UI automation
PDF
RPA Summer School Session 2.1: Run your first UI automation
PDF
A Modern Dive into QA Automation and Beyond
PDF
Functional Requirements Of System Requirements
PDF
Btec Business Level 3 Unit 14 M1
PPTX
Robotic Process Automation with UiPath Studio
PDF
Real-time Text Audio to Video PPT Converter Tablet App
PDF
A distinct approach for xmotif application gui test automation
DOCX
UI path Interview Question
PDF
Workflow automation i phone application for a construction company
PDF
Password security system for websites
PPTX
UiPath Studiox session I
PDF
Community App for Promoting Cross-Cultural Interaction
PDF
Community App for Promoting Cross-Cultural Interaction
PDF
UI Testing: A Complete Guide With Techniques, Tools, & Best Practices
Suncoast Credit Union and Armwood High School - UiPath automation developer s...
Human Computer Interaction: User Interface Layer
UiPath Veterans RPA Studio track program - Session 3: Introduction to Ui Auto...
Hidden Object Detection for Computer Vision Based Test Automation System
Application for Data Sync Between Different geo Locations
RPA summer school session 2.2: Run your first UI automation
RPA Summer School Session 2.1: Run your first UI automation
A Modern Dive into QA Automation and Beyond
Functional Requirements Of System Requirements
Btec Business Level 3 Unit 14 M1
Robotic Process Automation with UiPath Studio
Real-time Text Audio to Video PPT Converter Tablet App
A distinct approach for xmotif application gui test automation
UI path Interview Question
Workflow automation i phone application for a construction company
Password security system for websites
UiPath Studiox session I
Community App for Promoting Cross-Cultural Interaction
Community App for Promoting Cross-Cultural Interaction
UI Testing: A Complete Guide With Techniques, Tools, & Best Practices
Ad

More from Rohit Radhakrishnan (20)

PDF
Presentation Session 1 - Introduction to Agentic.pdf
PPTX
UiPath Community - Orchestrator Upgrade and Migration Strategies.pptx
PPTX
UiPath Community - Orchestrator Upgrade and Migration Strategies.pptx
PPTX
UiPath Devops.pptx
PPTX
UiPath Test Suite_final.pptx
PPTX
DU PPT (1).pptx
PPTX
UiPath Test Manager Connect Webinar UiPath Planview.pptx
PPTX
uipath_insights_upgrade.pptx
PPTX
UiPath Marketplace - HyperHack 2023.pptx
PPTX
UiPath_Orchestrtor_Upgrade_IAAS_PAAS.pptx
PPTX
Hyperhack_2023_06072023.pptx
PPTX
DevDive_UnleashthFullPotentialofAutomationwithGenAI.pptx
PDF
Hackaton Moonshots - 06222023.pdf
PDF
Dev Dives MAY Deck .pdf
PPTX
Automation Cloud Best practices series - Session 6.pptx
PPTX
Variable and Arguments_4.pptx
PPTX
Automation Cloud Series - Mastering the Automation Cloud Admin experience_Ses...
PPTX
Maximizing efficiency and security in large-scale automation rollouts with Au...
PPTX
Introduction-To-RPA_1.pptx
PPTX
Automation Cloud Series - Session 3.pptx
Presentation Session 1 - Introduction to Agentic.pdf
UiPath Community - Orchestrator Upgrade and Migration Strategies.pptx
UiPath Community - Orchestrator Upgrade and Migration Strategies.pptx
UiPath Devops.pptx
UiPath Test Suite_final.pptx
DU PPT (1).pptx
UiPath Test Manager Connect Webinar UiPath Planview.pptx
uipath_insights_upgrade.pptx
UiPath Marketplace - HyperHack 2023.pptx
UiPath_Orchestrtor_Upgrade_IAAS_PAAS.pptx
Hyperhack_2023_06072023.pptx
DevDive_UnleashthFullPotentialofAutomationwithGenAI.pptx
Hackaton Moonshots - 06222023.pdf
Dev Dives MAY Deck .pdf
Automation Cloud Best practices series - Session 6.pptx
Variable and Arguments_4.pptx
Automation Cloud Series - Mastering the Automation Cloud Admin experience_Ses...
Maximizing efficiency and security in large-scale automation rollouts with Au...
Introduction-To-RPA_1.pptx
Automation Cloud Series - Session 3.pptx

Recently uploaded (20)

PDF
Nekopoi APK 2025 free lastest update
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
PDF
medical staffing services at VALiNTRY
PDF
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
PPTX
Odoo POS Development Services by CandidRoot Solutions
PDF
Navsoft: AI-Powered Business Solutions & Custom Software Development
PDF
2025 Textile ERP Trends: SAP, Odoo & Oracle
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
PDF
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
PPTX
CHAPTER 2 - PM Management and IT Context
PDF
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
PDF
PTS Company Brochure 2025 (1).pdf.......
PPTX
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
PDF
How to Choose the Right IT Partner for Your Business in Malaysia
PDF
Adobe Illustrator 28.6 Crack My Vision of Vector Design
PDF
Design an Analysis of Algorithms I-SECS-1021-03
PDF
Which alternative to Crystal Reports is best for small or large businesses.pdf
PDF
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
PDF
Digital Strategies for Manufacturing Companies
Nekopoi APK 2025 free lastest update
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
medical staffing services at VALiNTRY
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
Odoo POS Development Services by CandidRoot Solutions
Navsoft: AI-Powered Business Solutions & Custom Software Development
2025 Textile ERP Trends: SAP, Odoo & Oracle
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
CHAPTER 2 - PM Management and IT Context
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
PTS Company Brochure 2025 (1).pdf.......
Oracle E-Business Suite: A Comprehensive Guide for Modern Enterprises
How to Choose the Right IT Partner for Your Business in Malaysia
Adobe Illustrator 28.6 Crack My Vision of Vector Design
Design an Analysis of Algorithms I-SECS-1021-03
Which alternative to Crystal Reports is best for small or large businesses.pdf
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
Digital Strategies for Manufacturing Companies

RPA Developer Kickstarter | Day 3: UI Automation and UiPath Selectors

  • 1. RPA Kickstarter Day 3 Ui Automation & Selectors Lovely Sinha
  • 2. 2 1. Introduction to User Interface Automation  What is UI Automation?  How does UI Automation Work? 2. Selectors and Ui Explorer 3. Input Activities and Methods 4. Output Activities and Methods Agenda
  • 3. 3 User Interface (UI) Elements : All graphical user interface pieces that construct an application, be they windows, checkboxes, text fields or drop- down lists, and so on. UI Automation: The process of interacting with graphical elements within applications by simulating mouse and keyboard controls. There are majorly two interactions we can observe in UI Automation  Input: Inserting data into an application  Output: Fetching data from an application or Ui Elements
  • 4. 4 UI Automation Activities Latest Package Available in Manage Package Section Activities can be split into: •Containers - These are activities that identify the browsers or apps the process needs to interact with. All activities included within a Container will execute on the same application. Some examples are Open Browser, Attach Browser, Open Application, or Use Application/Browser •Input Activities - These activities send input to UI elements. They can be used to click, check, type into, send hotkeys, and so on. •Output Activities - get structured data or get UiElements containing images. •Synchronization Activities - They help you create triggers based on UI behavior, thus enabling the Robot to execute certain actions when specific events occur on a machine.
  • 6. 6 Modern Vs Classic Design Experience
  • 7. 7 Example 1: UI Demo Application Open Demo UI Application Enter Data Click Accept Get Transaction Number Input Actions Output Actions
  • 8. 8 Selectors Unique identifiers of UI elements. These store UI element attributes which will help identify the UI element uniquely during automation. Wild cards There are two types of wildcards available in UiPath  * - Replaces any number of characters.  ? - Replaces exactly one character.
  • 9. 9 UI Explorer It is the wizard which is used to indicate and edit selectors. If we are unable to locate the individual element then we can switch to other Ui Frameworks for better results.
  • 10. 10 Example 2: ACME Add Vendor Open Browser https://acme- test.uipath.com/login Enter Credentials Click Vendors -> Add Vendor Display added vendor details Input Actions Output Actions
  • 11. 11 Input and Output Methods Input Methods •Hardware Events Clicks: the mouse cursor moves across the screen. Typing: the keyboard driver is used to type individual characters. Does not work in the background. The attended User cannot touch the mouse or keyboard during the automation. •Send Window Message Replays the window messages that the target application receives when the mouse/keyboard is used. Works in the background. Users can work on other activities during the execution of the automated processes. •Simulate Uses the technology of the target application (the API level) to send instructions. Works in the background. Users can work on other activities during the execution of the automated processes
  • 12. 12 Output Methods Output Methods •Full Text the default method and good enough in most cases. It is the fastest, it can extract hidden text, it has 100% accuracy, and can work in the background. •Native compatible with applications that use Graphics Design Interface (GDI), the Microsoft API used for representing graphical objects. It doesn’t extract hidden text and it cannot work in the background; and just like Full Text, it doesn’t support virtual environments. •OCR OCR (or Optical Character Recognition) is the only output method that works with virtual environments and with “reading” text from images. Its technology relies on recognizing each character and its position. On the other hand, it cannot work in the background, it cannot extract hidden text, and its speed is by far the lowest.
  • 13. 13 UI Automation is the process of automating application user interfaces by simulating human input and output actions through specific UI Activities. The first step in automating UIs is understanding the logical sequence of steps that a human user would take. The second step consists in translating these steps into UiPath Studio Activities and configuring them. The complexity of the UI or of the application is irrelevant. All desktop/web applications can be automated by using universal or application-tailored activities. Takeaways
  • 14. 14 Assignment: Password Generator Open Browser and Navigate to URL https://www.rpasamples.com/ passwordgenerator Type Password Length Generate Password Get Password Input Actions Output Actions Post your Unique 10-digit password with the following hashtags : #UiPathCommunity #RPAKickStarter “My Buddy Bot generated a password – [your password]”
  • 16. 16 Learning References: Login to https://academy.uipath.com/ Course Name: https://academy.uipath.com/courses/ui-automation-with-studio https://academy.uipath.com/courses/selectors-in-studio Documentation Links: https://docs.uipath.com/studio/docs/about-ui-elements https://docs.uipath.com/studio/docs/about-selectors