ElasTest technical presentation

http://elastest.io
ElasTest
Funded by the
European Union
Why, What, Who,
Where, When, and How
ElasTest Overview Presentation | 24/05/17 | Madrid URJC

Why ElasTest?
●
Modern systems are complex
3
Distributed Software / Cloud Native / Microservices

Why ElasTest?
●
We need
tests to
assure
their
quality
4

Why ElasTest?
5
What tools do exist for E2E testing in
complex cloud native apps?

Why ElasTest?
●
The IDE (to develop tests)
6

Why ElasTest?
●
The CI Server (to run tests after changes)
7

Why ElasTest?
●
These tools are focused in managing test
execution
8

Why ElasTest?
●
execution
9

Why ElasTest?
●
execution
10

Why ElasTest?
●
Current testing tools have limitations
when facing E2E tests of complex
distributed software
– Log management
– Test orchestration
– Comparing test executions
– Web testing
– SuT instrumentation
11

Limitations of “testing tools”
●
Log management
12

●
Log management
13
Where in the
source code is the
log of the failing
test?

●
Log management
– How to query logs of the System Under
Test?
14
Elasticsearch Kibana Logstash

15

16
Test logs are in the
IDE / CI, not here

17
Which log entries are
related to failure?
Time filter?

18
Is that the best tool to
know why a test has
failed?

19
OtrosLogViewer is a powerful log analysis tool

●
What if a test wants to know if there are
WARN messages in SuT logs?
20

Why ElasTest?
●
with E2E tests of complex distributed
software
– Log management
– Web testing
21

●
Test orchestration
– Usually we want to execute the same test, but
with different:
●
Input data, browser version, SuT
configuration...
– CI Tools are improving in this domain...
●
Jenkins pipelines
●
Travis Build Stages
●
Cross browser testing services
22

●
Jenkins pipelines
23

●
Travis Build Stages
24
https://blog.travis-ci.com/2017-05-11-introducing-build-stages

●
Cross browser testing services
25

●
Some use cases are still complex

Why ElasTest?
●
software
– Log management
– Web testing
27

●
Comparing test executions
– When there’s a regression, one compares the
failing test log with a previous correct
execution log looking at the differences
– CI Tools only show test management info...
28

●
Diff tools are not designed to compare logs
29

●
JMeter doesn’t compare several executions
30

●
How we create graph comparisons?
31http://bravenewgeek.com/dissecting-message-queues/
Latency in several message queues

Why ElasTest?
●
software
– Log management
– Web testing
32

●
Web Testing
– Selenium is used to control
browsers
– There are problems when
developing tests
●
Browsers are opened during
execution grabbing the focus.
●
You can’t work while tests are
running in your laptop
33

●
Web Testing
– There are problems when
executing tests in CI
●
You have to configure Xvfb for
headless testing
●
You have to configure custom tools
to record test executions
●
Video files are not synchronized with
logs (you have to jump manually)
34

●
Web Testing
– Docker is helping here
●
Official selenium
images
– However
●
A lot of manual
configuration needed
●
No solution for Safari
or Edge
35

●
Web Testing
– Commercial cloud services solve
these problems
36

●
Web Testing
– Testing WebRTC (video) apps
requires new features
●
Quality of Service (QoS)
●
Quality of Experience (QoE)
●
Real-time Latency
●
Testing Network Topologies (ICE)
37

Why ElasTest?
●
software
– Log management
– Web testing
38

Why ElasTest?
●
SuT instrumentation
– Fault tolerance / Resilience testing
– Testing with different network conditions
– Execute the same test with different SuT
conditions (orchestration)
39

Why ElasTest?
●
SuT instrumentation
– No integration with CI systems
– Hard to manage that from tests
– Hard to compare test executions with
different SuT configurations
41

What is ElasTest?
●
An open source tool
●
Designed to ease E2E testing of
complex distributed systems
●
You can install it in your laptop and in
your CI server
43
Apache License 2

What is ElasTest?
●
How ElasTest is controlled?
– Through a web interface (Jenkins like)
– From CI servers plugins
– From IDE plugins (like plugins to deploy
apps in remote servers)
– From a command line tool (Maven like)
44

What is ElasTest?
●
How do you develop your tests?
– With your favorite language
– With your favorite testing framework
– With your usual tools like selenium
– There are libraries for Java and
JavaScript to ease development
45

Who is creating ElasTest?
●
ElasTest is developed as a project
funded by the European Commission
47
http://elastest.eu
ElasTest: an elastic platform for testing
complex distributed large software systems
Duration: From 2017-01-01 to 2019-12-31
Total cost: EUR 5 031 187,50
EU contribution: EUR 4 270 187
Coordinated in: Spain
Topic(s): ICT-10-2016 - Software Technologies
Call for proposal: H2020-ICT-2016-1
Funded by the European Union

●
Project coordinator
48

●
Participant institutions / companies
49

●
We at CodeURJC have
experience in testing
complex distributed
systems
50

Where is ElasTest?
●
ElasTest is developed in GitHub
●
There are several communication channels
52
http://elastest.io
https://github.com/elastest

When ElasTest will be available?
●
Roadmap
54
Milestone Title Date
M1 First ElasTest architecture April 2017
M2 First ElasTest artifacts August 2017
M3 Early ElasTest platform December 2017
M4 ElasTest CI system April 2018
M5 Mature ElasTest platform December 2018
M6 ElasTest fully qualified December 2019

How? ElasTest architecture
●
ElasTest is a microservice application
●
Made up of several components, talking to
each other via
– REST APIs
– Message queues
57

●
Test orchestration and recommendation
manager (TORM)
– Entry point of ElasTest Platform
– Provides API and Web UI for:
●
Test management
●
Test execution
●
Test orchestration
●
Test comparison
58

●
Platform Manager (EPM)
– Abstracts other components from the
underlying infrastructure
– Provides resources for the other components
– Supported platforms
●
Public clouds (AWS)
●
On premise (OpenStack, Kubernetes)
●
Also in your laptop (Docker)
59

●
Data Manager (EDM)
– Provides persistence for other components
●
Management data (MySQL)
●
Logs & metrics (ElasticSearch)
●
Files (Alluxio)
60

●
Intrumentation Manager (EIM)
– Instrument Software under Test (SuT) to:
●
Gather logs and metrics
●
Simulate real network conditions (low
bandwith, high latency, connectivity loss…)
●
Force node failures
●
Simulate node loads with CPU throttling
61

●
Test engines
– Engines complement the functionalities of
ElasTest with services to help in test definitions
●
Orchestrator Engine (EOE): Orchestrate and
compare several tests
●
Cost Engine (ECE): estimates costs of running a
given test suite on a given provider
●
Recommendation Engine (ERE): recommends
additional tests for similar contexts
●
Question & Answer Engine (EQE): Q&A UI for
gathering additional information about tests
62

●
Test services
– Tests need to use some services in E2E
testing
●
Browsers
●
Mobile emulators
●
IoT (devices/actuators/sensors) emulators
●
Security services
●
Big Data services
63

●
Service Manager (ESM)
– A registry for services within the ElasTest
platform
●
OSBA compliant
– Monitors service health status
– Makes ElasTest extensible
64

●
Built-in ElasTest services
– User impersonation Service (EUS): Provides
browsers and mobile emulators for tests
– Device impersonation Service (EDS): Provides
sensors, actuators and devices emulators for tests
– Security Service (ESS): Provides security checks
for tests
– Big Data Service (EBD): Big data is available for
tests for runtime analysis
– Monitoring Service (EMS): Provides test execution
metrics and logs
65

Do you have any question?
66
Thanks!
info@elastest.io

ElasTest technical presentation

More Related Content

Similar to ElasTest technical presentation (20)

More from ElasTest Project (8)

Recently uploaded (20)

ElasTest technical presentation