Integration Group - Robot Framework

Created by Jan Medved
www.opendaylight.org
Getting Started with
Robot Framework
Carol Sanders, Brocade

 Robot Framework is a Keyword Driven automation framework using Python or Java as
it's backend.
 Test cases are automated by writing steps using Robot framework keywords.
 Provides a simple library API for creating customized test libraries.
 Provides a command line interface and XML based outputs for integration into existing
build infrastructure (continuous integration systems).
 Report files are created as a result of every pybot execution: report.html, log.html,
output.xml.
 Implemented with Python
– Runs also on Jython (JVM) and IronPython (.NET)
– Can be extended natively using Python or Java
– Other languages supported via a remote interface
 Open source
– Apache 2.0 license
– Active development and growing community
What is Robot Framework
2

Test Suite Organization
3

Integration CSIT Organization
4
git clone https://git.opendaylight.org/gerrit/integration cd- ./integration/test/
Two directories
CSIT – contains the test suites, resource files containing keyword definitions particular to your
project tests, and testplans which are all needed to successfully run your test suite.
Tools – test scripts specifically written that test features
CSIT Directories (you will always put your files here)
Libraries
Suites
Testplans
Variables

Viewing Test Suites in Robot Framework Ride
5

Collections of test cases are called test suites. Test suites are made of files and directories.
1. Now that you have cloned the integration repository, you have the integration test infrastructure needed to begin
creating your test suite.
2. Open ride on the command line – ride.py&
3. Open directory to Integration test suites directory -- ./integration/tests/csit/suites
4. Create a new directory. This where your test suite and associated files will be located. Use the name of your
project as the directory name.
5. Create a new test suite within the directory just created.
6. Include all elements described in Test Suite Contents: Settings, Keywords, Resource file(s), Variables,
Documentation, Tags, Log, Setup, Teardown
7. Follow best practices when writing test cases
Creating Robot Framework Tests Suites
6

Integration Sample Test Suite
7

Sample Test Case Format
8
*** Settings ***
Documentation Test suite for RESTCONF inventory
Suite Setup Create Session session http://${CONTROLLER}:${RESTCONFPORT} auth=${AUTH}
headers=${HEADERS_XML}
Suite Teardown Delete All Sessions
Library Collections
Library ../../../libraries/RequestsLibrary.py
Library ../../../libraries/Common.py
Variables ../../../variables/Variables.py
Resource ../../../libraries/Utils.txt
*** Variables ***
${REST_CONTEXT} /restconf/operational/opendaylight-inventory:nodes
@{node_list} openflow:1 openflow:2 openflow:3
*** Test Cases ***
Get list of nodes
[Documentation] Get the inventory
[Tags] restconf
Log ${start}
Wait Until Keyword Succeeds 30s 2s Ensure All Nodes Are In Response ${REST_CONTEXT} ${node_list}
Get nodeconnector for a node 1
[Documentation] Get the inventory for a node
[Tags] restconf
Log ${start}
${resp} Get session ${REST_CONTEXT}/node/openflow:1
Should Be Equal As Strings ${resp.status_code} 200
Should Contain ${resp.content} openflow:1:1
Should Contain ${resp.content} openflow:1:2
A test case file, for our purposes, contains the following default sections:
Settings, Variables, Test Cases, Setup and Teardown.
Setup and Teardown usually defined in resources file or initialization file.

Initialization File Format
9
*** Settings ***
Documentation Flow test suite for the OpenDaylight karaf-compatible feature set
Suite Setup Start Suite
Suite Teardown Stop Suite
Resource ../../../libraries/Utils.txt
*** Variables ***
*** Keywords ***
[Teardown]
This section is used to shutdown systems, close connections, and cleanup configurations.

Test Suite Contents
10
Recommended Test Suite Contents
 Setup
 Settings
 Resource file
 Keywords
 Documentation
 Log
 Tags
 Test case step(s)
 Tear Down

Test Suite Contents
11
 A test case file can contain other configuration information to ease the processing and speed
results such as specifying variables, path to external keywords or python functions or other
configuration data.
 When a test directory is executed, the files and directories it contains are processed recursively
as follows:
 Files and directories with names starting with a dot (.) or an underscore (_) are ignored.
 Directories with the name CVS are ignored (case-sensitive).
 Files not having one of the recognized extensions (.html, .xhtml, .htm, .tsv, .txt, .rst, or .rest) are
ignored (case-insensitive).
 Other files and directories are processed.
 If a file or directory that is processed does not contain any test cases, it is silently ignored (a
message is written to the syslog) and the processing continues.

12
Test execution is normally started using pybot, jybot or ipybot runner script.
These scripts are otherwise identical, but the first one executes tests using Python, the second using Jython, and the last one using IronPython.
Regardless of execution approach, the path (or paths) to the test data to be executed is given as an argument after the command.
Additionally, different command line options can be used to alter the test execution or generated outputs in some way.
Specifying test data to be executed
Robot Framework test cases are created in files and directories, and they are executed by giving the path to the file or directory in question to the selected runner
script. The path can be absolute or, more commonly, relative to the directory where tests are executed from.
The given file or directory creates the top-level test suite, which gets its name, unless overridden with the --name option, from the file or directory name.
Example cli command to execute test cases / suites
pybot test_cases.html | pybot path/to/my_tests/ | pybot c:robottests.txt
It is also possible to give paths to several test case files or directories at once, separated with spaces. In this case, Robot Framework creates the top-level test suite
automatically, and the specified files and directories become its child test suites.
The name of the created test suite is derived from child suite names by concatenating them together with an ampersand (&) and spaces.
These automatically created names are often quite long and complicated. In most cases, it is thus better to use the --name option for overriding it:
pybot my_tests.html your_tests.html pybot --name Example path/to/tests/pattern_*.html
Using options
Robot Framework provides a number of command line options that can be used to control how test cases are executed and what outputs are generated.
When options are used, they must always be given between the runner script and the data sources. For example:
pybot -L debug my_tests.txt pybot --include smoke --variable HOST:10.0.0.42 path/to/tests/
Test Suite Execution

Short and long options
Options always have a long name, such as --name, and the most frequently needed options also have a short name, such as -N.
In addition to that, long options can be shortened as long as they are unique. For example, --logle DEBUG works, while --
lo log.html does not, because the former matches only --loglevel, but the latter matches several options. Short and
shortened options are practical when executing test cases manually, but long options are recommended in start-up scripts,
because they are easier to understand.
The long option format is case-insensitive, which facilitates writing option names in an easy-to-read format. For example, --
SuiteStatLevel is equivalent to, but easier to read than --suitestatlevel.
Setting option values
Most of the options require a value, which is given after the option name. Both short and long options accept the value
separated from the option name with a space, as in --include tag or -i tag. With long options, the separator can also
be the equals sign, for example --include=tag, and with short options the separator can be omitted, as in -itag.
Some options can be specified several times. For example, --variable VAR1:value --variable
VAR2:another sets two variables. If the options that take only one value are used several times, the value given last is
effective.
Simple patterns
Many command line options take arguments as simple patterns. These glob-like patterns are matched according to the
following rules:
•* is a wildcard matching any string, even an empty string.
•? is a wildcard matching any single character.
•Unless noted otherwise, pattern matching is case, space, and underscore insensitive.
Examples:
--test Example* # Matches tests with name starting 'Example', case insensitively. --include f?? # Matches tests with
a tag that starts with 'f' or 'F' and is three characters long.
Test Suite Execution
13

Test Suite Reporting
14
Robot outputs three files detailing the results of the test case / suite run
● Output.xml
Output files contain all the test execution results in machine readable XML
format. Log, report and xUnit files are typically generated based on them, and they can also be
combined and otherwise post-processed with Rebot.
● Report.html
Report files contain an overview of the test execution results in HTML format. They have statistics based
on tags and executed test suites, as well as a list of all executed test cases. When both reports and logs
are generated, the report has links to the log file for easy navigation to more detailed information. It is
easy to see the overall test execution status from report, because its background color is green, if
all critical tests pass, and bright red otherwise.
The command line option --report (-r) determines where report files are created. Similarly as log files,
reports are always created unless NONE is used as a value, and their default name is report.html.
● Log.html
Log files contain details about the executed test cases in HTML format. They have a hierarchical
structure showing test suite, test case and keyword details. Log files are needed nearly every time when
test results are to be investigated in detail. Even though log files also have statistics, reports are better
for getting an higher-level overview.
The command line option --log (-l) determines where log files are created. Unless the special
value NONE is used, log files are always created and their default name is log.html.

Robot Test Suite Log File
Example Test Suite Report
15

Summary
16
Test Suite and Test Case Automation Best Practices
● Document each test case. Explain the purpose of the test.
● Use the Log function to return useful information about keyword usage and test case runs.
● Only test one attribute of the feature in a test case instead of many attributes.
● Every test case should have a tag to enable/disable test case run.
● Use ‘wait until’ instead of sleep when waiting for a process to complete.
● Use Resource files if your keywords span more than one test case and when using user created
keywords.
● Use an initialization file ‘__init_.txt to start services or systems needed for the tests.
● It is better to use variables instead of static assignment. Make you test suite more portable and
dynamic.
● Keep your test case Clear and concise.
● Clean up the test environment after each test suite run.
● Setup and Teardown sections in test case can be used for this purpose.
● Define global variables in a variables file.

Open Daylight Integration Reference
https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Test_Tools
Robot Integrate Dev Environment (RIDE)
https://code.google.com/p/robotframework-ride/
https://github.com/robotframework/RIDE/wiki
https://code.google.com/p/robotframework-ride/wiki/InstallationInstructions
Robot Guides
http://robotframework.org/
https://code.google.com/p/robotframework/
http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#executing-
test-cases
Extending Robot
http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#extending-
robot-framework
http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#test-data-
syntax
How to write good test cases:
https://code.google.com/p/robotframework/wiki/HowToWriteGoodTestCases
References
17

Integration Group - Robot Framework

More Related Content

What's hot (20)

Viewers also liked (19)

Similar to Integration Group - Robot Framework (20)

More from OpenDaylight (8)

Recently uploaded (20)

Integration Group - Robot Framework