Event Processing For Dummies

1. Introduction
Event processing is getting more popular concept in several sectors. It can be the
key concept that should be focused in the near feature technology trends. So we
prepared this booklet for to give an idea about event processing, usages and sample
implementations with our Complex Event Processing product EVAM (Event & Action
Manager).

Recently we were doing most of analysis on data that remains in the databases, but
now it is changing; it is time to process steaming data, before it goes to storage. We
call “event” as hot data or we can say that event is moving data, so if we want to
create real time or near real time architectures we should talk about event.

We are facing with big data concepts as well. Currently event producer systems
(hardware, &software) produces huge amount of data and cycle of data is becoming
as follows;

We were waiting to process data after it comes to the storage areas such as
databases, but now we have capabilities to process it as it is produced. This is new
era of processing the events that will be data after some time.

2. What is Event?
An event is any detectable occurrence in a
particular system or domain indicating a
change in the state of the business,
something that happens. Countless events
going on constantly everywhere. We react to
events every day. An event (business or
system) may signify a problem or impending
problem, an opportunity, a threshold, or a
deviation. These events may be happening
across the various layers of an organization as sales leads, orders or customer
service calls. Or, they may be news items, text messages, social media posts, stock
market feeds, traffic reports, weather reports, or other kinds of data. When you
tweet something that is actually an event. Or when you share a photo on Facebook.
Or when you use your credit card to buy something. Or when you call someone, send
text messages. The list is countless.

There are different types of events that we can group with a property such
frequency. Some of them are repetitive. Happens with a certain interval. Lots of
them happen randomly. Or we can group events by domain. Some examples for some
specific domains are as follows:

Bank Events:

Credit card usage, bankcard usage, shopping cancellation, money order, electronic
funds transfer, open/close account, cash sale, installment sale and much more.

E-Business Events:

New Registering customer (register events), login, purchase, shopping basket
updates, shopping card updates, viewing different products of same category,
customer logins from different locations, fraud detection.

Telecommunication Events:

Postpaid subscribers payments from a bank payment method, tariff changes,
account activation, voice calls by prepaid subscribers, voice calls by postpaid
subscribers, voice package usages of subscribers, voice pack, sms or data usages of
subscribers, voice calls or receiving sms from rival Telco firm

Online Gaming Events: Player’s login, player’s movements, player’s strategies.

3. Where are they come from?
An event can originate from any suitable
platform. Today, huge numbers of events come
from databases, software applications, devices
and any other platforms. Directly humans and
systems or devices themselves create events.

Hardware event producers
Hardware producers are used extensively in a number of application areas:
- Medical equipment and personal body sensors (for example, heart-rate monitors)
- Device management (computer systems or industrial equipment)
- Defense and military applications
- Security applications
- Traffic management systems
The archetypal hardware event producer is a sensor that generates events that
report on one or more aspects of the physical environment in which it is situated,
for example, a smoke detector. A sensor can be packaged as a discrete piece of
hardware, such as the smoke detector example; or it can be embedded into another
piece of equipment, for example, a sensor that detects the fan speed on a computer
motherboard. The simplest kind of sensor reports on just one aspect of its
environment, for example:
- Motion of the sensor itself including vibration
- Tilt or angle of orientation of the sensor
- Rotation of a rod attached to the sensor
- Temperature
- Humidity
- Light (intensity or color)
- Infrared or radio waves
- Sound (intensity of frequency)
- Air (or other gas) pressure
- Physical pressure applied to the sensor itself.

More sophisticated detectors use one or more of the physical detection mechanisms
to make specific observations or look for particular kinds of occurrence.
Here is a small list of examples:

- A sensor that detects motion external to itself, such as a passive infrared (PIR)
sensor. This can be used to detect the presence of people in the vicinity of the
sensor, for example, when reporting on room occupancy or when looking for
Intruders.
- A sensor that detects whether the door on the casing of equipment is open or
closed.
- An RFID reader used to detect the presence of an RFID tag. This can be used in
supply chain or many other application areas.
- A seismometer used to detect and report on earthquakes or nuclear tests.
- A traffic speed detector. Apart from their obvious use in penalizing speeding
drivers, speed detectors can also be used in intelligent traffic management systems.
- GPS location devices. These are used in a wide variety of tracking and location
aware services.
Cameras (both still and video), microphones, telephones, and radio receivers can also
be viewed as event producers because the data that they produce can be processed
by event processing applications. For example, you could have a security application
that processes frames coming from a video camera looking for the presence of
unauthorized personnel in a secure area.

Software event producers
Although software is often associated with a hardware producer, some event
producers are made up of software only.
The first category consists of simulated sensors. Simulated sensors are used when
the entire external system is itself a simulation, for example, a flight training
simulator or virtual reality game, and they can also be used to stand in for a real
piece of hardware when testing an event processing application.
An event producer could be a first-class part of a software application. By this we
mean that it is a piece of application logic that explicitly generates an event object
and submits it to the event processing network. This may happen as a result of a
human interaction but sometimes events can be generated less directly. For
example, a financial trading system might include a settlement application that
automatically generates payment events. This application uses a programming
interface to submit the event.
Events can be produced indirectly by a technique known as instrumentation. Here
the events are not generated by application code itself, but instead are produced by
software that is monitoring the application, looking for noteworthy activity. This
kind of producer is sometimes called a monitor or probe. A wide range of events
could be viewed as noteworthy, from tracing program start-up and shutdown or
tracing function calls within the application, through detecting updates that the
application makes to its data, reporting computer performance statistics, or
spotting and reporting on hardware or software errors if they occur.

Instrumentation can be provided by the operating system or the container that runs
the application, or by database or messaging middleware that the application uses.
Examples include workflow engines which can generate events when a particular
workflow goes through a state transition, security subsystems which can generate
alerting events when they detect attempted security violations, and message
queuing systems which can generate events when the number of messages waiting in
a queue exceeds a certain threshold.

More recently web-based feeds that use Atom Syndication Format or Really Simple
Syndication (RSS) have emerged. Literally tens of thousands of them are available
on the Internet.

Human interaction
Some events are generated directly by human interaction, albeit with a bit of
software and hardware assistance. Human interaction can be facilitated by an
application program with a user interface that in effect allows the user to enter the
event. In our example we could imagine that each store has a web application, and
store personnel use a form-style interface to place a delivery order request.
Events can also be generated using a verification or payment device, for example,
the delivery confirmation produced by the driver’s handheld device in our
application, or a purchase event being generated by a till in a retail store.
Some producers detect our presence, for example, the swiping of an identity card
(or having a personal RFID tag scanned) when entering a secure area, the use of an
NFC (near field communications) tag to go through a ticket barrier on public
transport or pass through immigration control. Instant messaging applications (and
increasingly telephony applications) can produce presence events that indicate when
a user has turned a computer or handheld device on or off, and these presence
events sometimes include information about the user’s location or actions.
This brings us to our next area, social communications. As well as providing its
regular web browser interface, the popular Twitter Internet service offers RSS
feeds that can be used to communicate presence (and other) events. Events can also
be produced from other social networking applications.

4. Why Event Processing?
If event processing is such a great idea, why
hasn’t everyone been doing it all along? Event
processing is underutilized partly because
relatively little data on current business events
has been available in digital form until recently.
In the past, many events either were
undetected or were detected but not reporter
in digital form that could be sent over a
network or manipulated by computer. Now, more events are detected and
represented electronically, although, unfortunately, many are still not readily
accessible to people, devices, or IT systems that could benefit from them.

The amount of available event data is rapidly expanding because of the decreasing
cost and increasing speed of computers and networks, and the unifying power of
World Wide Web and its communication standards. We are blessed with an
explosion of event “streams” flowing over corporate networks – data from websites,
enterprise application systems, e-mail systems, cell phones, RFID readers, GPS
systems, and a variety of other sensors and devices. This Wealth of event data will
grow as the cost of the relevant technologies continues to drop and companies
create new sources of event data in their operations and the outside environment.
Our challenge is to make better use of this data.

The other reason that formal event processing has not been widely used in the past
is that competition and customer demands were less urgent. Companies had more
time to respond to events than they have today. A person driving 30 km per hour
doesn’t need as mush advance warning of upcoming curves or obstructions on the
road as a person driving 60 km per hour. Companies today are operating at a faster
pace, so early notification of emerging business threats and opportunities are
evermore important. Companies that know how to leverage event processing have an
advantage over those that don’t.

5. How Does Event Processing Work?
Event processing is a method of tracking and analyzing (processing) streams of
information (data) about things that happen (events), and deriving a conclusion from
them.

Event processing starts with capturing events from source systems. Once you have
built the right architecture any system can be used as an event source. Web sites,
credit cards systems, GSM networks and Call Centers are just a few examples for
event sources in real world. EVAM has both hardware and software components that
listen live systems to capture events.

The next step, Parsing&Filtering is responsible for generating business events from
raw events. In this step filtering, parsing and formatting rules are applied to raw
events before sending them to processing engine.

An Event carries vital information but we have critical information resides in other
systems and is not carried with event itself. In Enrichment phase, business events
are enriched with historical data taken from external sources such as relational
databases, NoSql systems, web services etc…

Event processing step is the heart of the system. It processes streaming events
and decides to take action exactly when appropriate conditions are met. In EVAM

world, event processing engine acts according to defined scenarios. Scenarios are
drawn via drag&drop designer reflecting business needs. Scenarios can be seen as
rulesets but they have capability act according to event history and customer
context. All event processing is done in-memory meaning that you can process
events in real-time.

You have captured, parsed, enriched and processed events.. All these steps are done
for taking actions. Sending sms/mail, calling a web service, mentioning someone in a
tweet, liking some page in Facebook are all examples for actions. Actions are touch
points of event processing with other systems. You can either touch your customer
by sending sms or generate an alarm in a monitoring system via an interface.

All these critical steps should be monitored from an centralized system both for
operational and reporting purposes.

6. Data – Database / Event - Event Processing
A recent question came up regarding the difference between “events” and “data”.
Data is something that is your primitive entity that holds business information in a
tightly coupled SOA or database centric architecture. Events are something carries
around your business data in an EDA. Data is derived from events. Customer data?
Derived from customer registrations and associated checks – all events. Amount
owed? Derived from purchase events. And so forth.

In Complex Event Processing, we are processing this “event data” as it occurs
(within the limitations of our system latency, of course!). Typically this event data is
correlated with previous events, which can be either “relatively recent” (and
considered as stored events or event objects) or “historic” (and considered as
traditional data), although these are necessarily imprecise concepts. Access to
historic data is typically carried out in CEP tools through good-old-database-
services: calling the operational data store associated with legacy systems, for
example.

Event is real-time. So the information carried with them is hot data.

7. Event Processing In Business (Real Time Marketing,
Customer Experience Management, Fraud Detection,
Sensor Alarms)

Business world is a fast-paced environment and becomes harder day-by-day. So,
organizations must adapt this environment faster than rivals. Organizations need to
use more specialized software applications depending on some selection criteria
such as time, speed, budget etc.

Event processing makes dramatic improvements in business processes and IT
systems possible. It has a direct, tangible impact on the lives of businesspeople.
Event processing changes the way they do their jobs by giving them better visibility
into what is happening in their company and its external environment. It also
improves a company's reaction time to unforeseeable situations, reduces the end-
to-end elapsed time of business processes, and improves the quality and availability
of information.

From this point; EDM, which is the new trend, has some advantages over traditional
direct marketing campaigns:

Higher response rate

EDM typically has higher response rates than traditional DM campaigns. Not only is
the client more perspective for your offer because you target him at the
appropriate time and through the channel of choice of client, you also gain by
sending out less communication.

Higher impact

Not only do you leverage the response rate, you also have more impact on the client
resulting in higher profit.

Possibility of testing

In contrast to traditional campaigns where testing is either very expensive or
nearly close to real campaign processes, EDM lets you test new ideas for scenarios
rather quickly because once the scenario is ready you can select a limited numbers
of clients daily for testing, or test manually with hypothetical.

Ongoing adjustment

Because you select a target group and send out messages every day, you can monitor
and adjust both selection criteria, scenario and message content daily. Compare this
to a traditional campaign where you can adjust these variables let’s say once a year.

Homogenous load

With EDM, you level out the work load (both financial as in terms of human
resources). You divert the peaks of load in the startup-phase of a new traditional
campaign to daily operations.

8. Future Of Event Processing
Some people say that event processing will be the next big development in
computing; others say that event processing is old hat with nothing new.

CEP starts becoming an enabler for real time business intelligence in the last few
years, the technologies for implementing real time BI solutions have evolved
drastically making it a viable alternative in the enterprise. A clear example is the
upcoming Microsoft's PowerPivot stack that is able to process millions of records in
a highly optimized multidimensional store that can be accessed from both Excel and
SharePoint environments.

The missing component of real time BI solutions is the mechanisms for collecting
and processing the data in highly efficient ways. This is where CEP could shine. Most
CEP engines are optimized for processing a large number of events using continuous
querying mechanisms. In that sense, CEP technologies should become an essential
component of real time solutions in order to make these type of solutions a viable
alternative in the
Big enterprise.

CEP engines go mobile
Traditionally, CEP applications are based on a server-centric model that is
responsible for hosting, managing and scaling the core components of the CEP
engines. Even though a large number of CEP scenarios can be implemented using this
model, there are native capabilities of CEP engines such as the continuous query
engines or adapters that can be very effective on mobile applications. For instance,
consider a traditional RIFD scenario on which a mobile reader is capturing
thousands of events that need to be filtered based on different patterns. On this

scenario, w e could read the events using a CEP adapter for the mobile RFID reader
and filter them using a CEP continuous query engine. This post does not intend to be
formalized list predictions about the CEP market. Most of the thoughts listed there
are based on tendencies that I have seen in different CEP projects throughout
recent years.

9. Complex Event Processing with EVAM (Event &
Action Manager)
EVAM® is a r e a l time event-processing solution capable of
s e n s e , f i l t e r , e n r i c h a n d response to events as
it is produced. It can process millions of events in seconds and
can create actions that can be customized depending on the
requirements.

EVAM is a highly elastic and easy to use event processing
solution empowered with the ability to react to potential
opportunities or risks in real-time and at an unprecedented scale.

Event loops became a central design principle in programming following the
introduction of graphical user interfaces. Programs listening to various events were
principally designed to accommodate unpredictable users who might do anything in
front of their computers at any given time.

The framework of our solution consists of three main components: Scenario Studio,
Event-listeners, and Processing Engine.

Recognition of patterns in event-streams, enrichment of data by connecting to other
data sources, decision trees and all the governing logic and rules are designed using
simple drag&drop techniques at our Scenario Studio.

Event-listeners are practical low impact components that are installed at various
event sources to capture and send relevant data to the Processing Engine(s).

The scenarios are compiled by the Scenario Studio and loaded up to a cluster of
engines for high velocity processing. The Event-listeners stream data to the
Processing Engine cluster and our innovative architecture ensures persistence of high
volumes and both vertical and horizontal elasticity from a few hundred to billions of
events per day.

Event & Action Manager (EVAM) allows you to abstract and correlate meaningful
business information from the events and data flowing through your information
systems, and take appropriate actions using scenarios. By detecting patterns within
the real-time flow of events, EVAM can help you to detect and understand unusual
activities as well as recognize trends, problems, and opportunities EVAM publishes
this business-critical information in real time to your critical enterprise systems,
dashboards or create response actions. With EVAM you can predict the needs of
your customers, detect frauds, make faster decisions, and take faster action.

There are many application areas for EVAM such as real time offer management,
fraud detection, real time alert generations etc.

Next Generation Marketing Solution concept that aims to take necessary action by
catching the demand when it emerges is with you thanks to Event&Action
Manager(EVAM). Intellica, known by the power of its intellectual capital and its
mastery in high technology in Business Intelligence and Customer Relations
Management presents the Next Generation Event Processing solution to
organizations.

EVAM provides realization of sale and event-driven real-time marketing activities,
especially for sectors of finance, retailing and telecommunication. EVAM broadens
marketing departments’ horizons by integrating into data warehouses and customer
relations management systems of organizations easily.

Product technically capable of processing billion of events from many event source
systems and expand horizontally with new clusters. There are seamless integration
capabilities with event sources in both pull and push methods.

EVAM is, it is business centric tool; with drag and drop type of user interface, a
typical power business user is able to implement new event driven scenarios in EVAM
with minimum IT support. This give opportunity to organizations decrease time to
market dramatically for new marketing ideas.

There is a paradigm shift happening from business intelligence to operational
intelligence. EVAM is one of the very few proven and pioneering products in this
market.

EVAM is one of the newest products in the CEP market with its unique approach to event
processing. It is possible to develop deterministic and nondeterministic scenarios
together.

10 Reason to Use EVAM

1. Your application might need to identify and react to certain situations (either good or
bad) as they occur. An event-driven approach, where changes in state are monitored as
they happen, lets an application respond in a much more timely fashion than a batch
approach where the detection process runs only intermittently.

2. Event processing can give you a way of extending an existing application in a flexible,
non-invasive manner. Rather than changing the original application to add the extra
function, it’s sometimes possible to instrument the original application by adding event
producers to it (for example, by processing the log files that it produces). The additional
functionality can then be implemented by processing the events generated by these event
producers.

3. Intermediary event processing logic can be separated out from the rest of the
application. This can allow the application to be adapted quickly to meet new business
requirements, sometimes by the application business users themselves.

4. The application might involve analysis of a large amount of data in order to provide an
output to be delivered to a human user or another application. This data can be organized
into streams of events which are then distributed to multiple computing nodes allowing
separate parts of the analysis to be performed in parallel.

5. There are potential scalability and fault tolerance benefits to be gained by using an
event-driven approach. An event-driven approach allows processing to be performed
asynchronously, and so is well suited to applications where events happen in an irregular
manner. If event activity suddenly spikes, it may be possible to defer some processing to a
subsequent, quieter time.

6. Create deterministic or nondeterministic scenarios

7. Create more business involvement

8. Integrate Decision Engine with Event Processing

9. Your application might be naturally centered on events.

10. Get high volume event processing capability

Please visit www.evam.com.tr for more information about EVAM or contact
info@intellica.net to test the product on your site.

Event Processing For Dummies

More Related Content

What's hot (20)

Similar to Event Processing For Dummies (20)

More from Evam Streaming Analytics (7)

Event Processing For Dummies