E com conversion prediction and optimisation

E-commerce conversion prediction and optimisation
A data driven approach using supervised and unsupervised learning algorithms
Alexandros Papageorgiou
School of Computing
National College of Ireland
Abstract—E-commerce growth rates continue to climb around
the globe despite low buyer conversion rates remaining a major
hurdle. This is partly due to the lack of systematic analysis
frameworks that enable digital businesses to measure themselves
in order to gain a deeper understanding of the factors driving
conversion metrics and to optimize their efforts in markets. This
study used a widely available web analytics tool to
programmatically collect visitor navigation data. After
transforming the data a selection of supervised and unsupervised
learning algorithms were implemented in order to predict and
optimise e-commerce conversion. The results suggest that the
support vector machines algorithm provides the highest
performance for predicting shopper conversion. Random forests
variable importance suggests that the key factors playing a role in
the process are visitor type, traffic source and operating system,
subcontinent and days since last session. Clustering and key ratio
analyses provide additional ways of understanding key conversion
trends on the website. The study as a whole postulates the
provision of targeted data driven recommendations with special
focus on the digital marketing strategy.
I. INTRODUCTION
A. E-commerce conversion and challenges
E-commerce activity has been rapidly expanding since the
web’s early days when that medium was perceived as a new
powerful outlet for conducting business. Despite the growth
and the continuous improvements in product availability,
personalisation and website design, e-commerce conversion
rates have remained extremely low. Values in the range of one
to three per cent are not considered uncommon[1].
Conversion rate in general is defined as the fraction of
users who complete the purchase process on a website.
Conversion is used interchangeably with similar -but not
identical in meaning- terms such as transaction and purchase.
This work adopts the terms conversion as it is the most
commonly used in the industry.
E-commerce differs from traditional "bricks and mortar"
commerce in many dimensions, one of which is the ease with
which web users can enter and leave a website. This
encourages more digital comparison and hedonistic window
shopping activities. All factors considered however, the fact
that over 95 % of the users on average do not complete a
purchase represents a sizable growth area. This is especially
true for e-commerce websites which are able to gain a deeper
understanding around the factors that drive conversions.
Indeed, modern digital businesses tend to monitor a wide
range of conversion related KPIs such as conversion rate, cost
per conversion and unique converted users among others. This
however is often not enough to provide adequate insight into
the individual purchase behaviour of consumers.
The incentives in any case remain strong as small changes
in the conversion rate can result in significant revenue uplift.
Moreover, targeting users with the right characteristics and a
high probability to convert can represent an area of
opportunity for the digital business.
The problem is a fairly complex one considering the
diversity of the internet population and the multitude of factors
that can impact their behaviour. This is with respect to the
user's own motivations and intents but also in conjunction
with website elements such as its design, prices and product
availability.
B. Literature Review
Researchers have approached the topic of user conversion
with respect to prediction and optimisation from many
dimensions. The reach of one of the key e-commerce
objectives, purchase, involves the examination of several
elements associated with human behaviour, technology and
the interaction between the two. Several studies approach the
question from a behavioural point of view and attempt to
quantify the strengths of various qualitative factors associated
with conversion. These factors include user needs, perceptions
and preferences [2].
Other studies expand this line of research by factoring in
additional parameters that are found to affect the conversion
process. These parameters include perceived consumer risks in
relation to e-commerce, impact of individuals in the social
circle of the users and personality type [3]. Another work
focuses on prior experience of shopping online and preferred
ways of payment [4].These studies use supervised experiments
and observation of a small number of subjects as their main

input. They highlight valuable qualitative insights, but they
can be difficult to reproduce.
An alternative line of research focuses on the analysis of
large amounts of automatically collected web access logs in an
unsupervised setting. The key component is the analysis of
clickstream and granular navigation path data. Within this
area there are no shortage of studies [5, 6] that examine the
question in various specific contexts for example group
buying, social media activity and search engine querying
behaviour.
For the purpose of this study, however, the focus is on
more high level approaches that can have general application,
regardless of the specific type of user context and website.
These studies can be divided in two categories, the ones that
are purely based on analysis of clickstream data and the hybrid
ones that combine it with a number of behavioural
characteristics and site features.
Within the clickstream category there are two main
approaches. An approach that focuses on web path analysis
and one that is primarily based on machine learning and
predictive modelling. The former is typically based on
probabilistic analysis.
1) Web paths and clickstream analysis
Wu et al. [7] use the notion of state of Markov stochastic
process models to study and understand conversion. The
study predicts the most probable paths based on the sequence
of previous steps and thus it is able to predict conversion in
real time. The advantage is that relevant information can be
provided for machine-based decision making at the earliest
possible opportunity
Suh et al. [8] introduce a methodology for real time web
marketing based on association rules with apriori algorithm
implementation. The research classifies pages with a key
corresponding type and then mine the sequences of those
pages to determine whether a conversion took place or
not. Key patterns are subsequently identified based on support
and confidence rules for the associated pages.
While those studies have the capabilities to function and
share information with other systems in real time, they do not
specifically address the so-called cold start problem of
conversion prediction. This refers to the presence of first time
users which is frequently the majority type of users for e-
commerce websites.
To address this Yanagimoto and Koketus [9] suggest that
user profiles are designed based on granular access logs and
matched to neighbouring profiles by cosine similarity from
historical data. Then they associate specific influential web
pages of the site -which they call characteristic pages - with
signals for possible purchase. Subsequently those pages are
ranked using an adjusted PageRank score, producing
customised ranks for different user profile types.
All those approaches are very effective in exploiting the
richness of clickstream data to make informed predictions and
associations. Their limitation however is that they solely
depend on navigation path data. New trends in technology
however have enabled the adoption of advanced machine
learning methods across a high number of dimensions in order
to gain additional insights.
2) Machine Learning approaches
The most extensive study identified suggests a subset of
key variables from a combination of enriched clickstream
data, customer demographics and historical behaviour that
predict next visit conversion [10]. The study uses logit
modelling with best subset selection from a total of 92 initial
predictors. The authors highlight the importance of variables
from clickstream such as the number of products viewed, days
since last visit, supply or not of user information. While very
holistic as an approach, the downside of including a high
number of dimensions is that it has to come from registered
users. For the majority of websites, registered users constitute
only a small fraction of the total user traffic. Moreover, the
study focuses only on linear methods to model the relationship
between predictors and outcome.
Vieria [11] employs click stream data analysis combined with
advance methods of supervised machine learning, including
deep learning, to model the non-linear relationship in many
layers over a deeper architecture The study also uses rich, high
dimensional datasets. The algorithm is compared to logistic
regression and decision tree implementations. The suggested
deep learning algorithm significantly improves performance
and helps to predict purchase in different contexts.
3) Hybrid approaches
A series of studies by Moe and Fader [12] [13] take a
different approach by focusing on timing frequency and user
evolving behaviour. The authors use clickstream data to
extract the temporal association patterns between returning
customer visits and conversion. They additionally address the
heterogeneity of users by classifying them in four categories.
The classification is based on their perceived intention as
derived through their navigation patterns: planned purchases,
hedonic browsing, knowledge building and searching. These
levels are used to adjust the baseline prior probability of
conversion in combination with signals related to historical
visit and purchase trends. Thus according to Moe and Fader
[12] [13] the temporal patterns combined with predefined user
clusters can significantly improve conversion prediction.

Sismeiro and Bucklin [14] use a decomposition of the site
navigation process in sequential tasks, which are required to
take place prior to purchase. Examples include browsing
behaviour, use of interactive decision aids, information search
and input of personal info such as payment details. The
processing of those tasks in a Bayesian setting, results in a
sequence of conditional probabilities further adjusted to
account for different user location and demographics where
available. Results indicate that visitors’ browsing experiences
and navigational behaviour are predictive of task completion
and therefore likely buyers can be identified early in the
process.
While all the previous studies provide interesting
extensions and insightful answers to the conversion question,
the complexity of their implementation makes it challenging
to fully adopt. Additionally, the prerequisite step of the access
to fine-grained clickstream data that are ready to be processed
is practically not within reach (both in terms of access and
analysis) of the vast majority of the websites which typically
lack the resources required for this.
With respect to the breadth of the validity of the results,
the studies are restricted to data from just one e-commerce
provider -typically retailer- and therefore cannot be
generalised across all types of websites. The methodology
cannot be directly adopted to fit other online business models.
In addition some of the most sophisticated studies assume the
availability of key additional information taking for granted
that the users are logged in and have pre-existing profiles.
Another point that has not been examined thoroughly by
the research so far pertains to the fact that conversion is a rare
event that constitutes an imbalanced class. Therefore
specialised methodology needs to be employed to address this
inherent characteristic.
C. Objectives
The objective of this work is to study the process of online
conversion from multiple perspectives and help determine
holistically the major factors that drive conversion in an e-
commerce website. There is a special emphasis on the
evaluation of conversion potential from key marketing media
traffic as well as their components (campaigns and ad-groups).
The study will move gradually from a high level to a more
granular level of conversion analysis. The final objective is to
produce an accurate and stable predictive model which will
enable the systematic prediction of sessions/users who are
likely to convert. Additionally, it will assess the importance of
various marketing channels with respect to conversion. The
model will be tested to prove its effectiveness with unseen
data by using established predictive modeling methods.
The current study has a broad scope. Even though the analysed
data belongs to a specific e-commerce company, the
methodology can easily be generalised regardless of the
specific industry, size and user types of a website. This result
is achieved thanks to the possibility of accessing data
programmatically via the most widely used web analytics
product in the market, Google Analytics.
D. Metrics
With respect to the predictive model, established methods
and metrics will be adopted. Cross validation will facilitate the
selection of the optimal parameters for the model to
avoid over-fitting issues and a validation dataset will be
made available in order to test the performance of the model
using out of sample data. Metrics such as accuracy, area under
the curve, sensitivity and specificity will serve as criteria for
the model evaluation and further improvement.
II. METHODS
A. Basic project design
The basic design of the project highlighted the progress
from the study of general conversion trends to the
identification of specific factors that can describe and predict
conversion. The design accomplished this in three stages.
 The analysis of conversion quality with respect to
key marketing media traffic, with the aim of
capturing a high level picture of the conversion
behaviour across the main traffic sources -which in
the case of digital businesses are almost entirely
composed of various types of digital marketing
channels.
 A more granular level analysis of the performance of
the specific ads of one of the key marketing channels,
with respect to conversion and other related KPIs
such as engagement, volume of transactions and visit
to transaction ratio. This analysis was based on
hierarchical clustering.
 The final stage of the research involved the use
of predictive modelling to predict conversion based
on a multidimensional analysis, while at the same
time evaluating the importance of specific factors that
lead a user to convert.
B. Source data
The dataset refers to the navigation data of users on an e-
commerce website in the retail sector. Custom JavaScript on
the web pages source code transfers user navigation data to
Google Analytics servers. Then the data are made available to

analysts via the Google Analytics user interface and/or the
API. The project was based on data recorded during a period of
6 months. The full unprocessed dataset consisted of over 300
thousand observations and 16 variables (metrics and
dimensions in web analytics terminology). Examples of the
variables included: date and time of session, traffic source, user
location, session page depth, session duration, browser and
operating system. Each one of the three stages of the project
involved different subsets of the initial dataset. The predictive
modelling part, after all filtering and pre-processing was
completed, was based on 36948 examples and 7 predictor
variables both numerical and categorical.
C. Technology
Various tools were deployed for the analysis of the data.
To access Google Analytics data, API functionality was used.
Some of the visualisation for exploratory analysis was
performed in Tableau. However, the more complex graphics
were developed using the R Ggplot library.
In general, the statistical programming language R was
used for most of the data manipulation and modelling. A
major reason for this was the availability of a customised
library called RGA that facilitated almost all aspects of
accessing the API. In order to be consistent and develop an
easy to reproduce study, R was used for the subsequent steps
too.
Due to the size of the dataset, the predictive modelling
proved to be demanding computationally. For this reason
parallel processing was deployed. The parallel package was
used to accelerate the matrix calculations that are typically
required for the execution of predictive models and their
respective parameter tuning operation.
D. Key performance ratio analysis
Key performance ratios are heavily used in business
analysis as they are effective for data exploration in context.
For the purpose of the project conversion, quality index
analysis [15] was performed. This is an exploratory method to
examine the underlying dynamics with respect to conversion
when comparing the performance of several traffic media. It is
thus a valuable way to prioritise the importance of the key
traffic media. The conversion quality index represents the
proportion of conversions that each medium contributes to the
total as a percentage of the proportion of total traffic it
receives.
If, for example, an ad medium receives 30 % of the site traffic
and contributes 30 % of conversions, the ratio equals one. All
other factors being equal, the higher the ratio the better the
relative performance of the given medium with respect to
general website conversion.
E. Clustering
Hierarchical clustering is a very widely used method of
unsupervised learning that enables the discovery of structure
in data based on a chosen similarity criterion. It was employed
in the study in order to create groupings of associated ad-
groups within key advertising campaigns that exhibit similar
characteristics with respect to conversion, both in terms of
volume of conversions and conversion rate.
In stage one, the project only studied the question from a
high level to identify channels of high conversion potential.
However, each channel is the sum of its distinct parts,
typically referred to as marketing campaigns or groups of ads.
The search advertising channel traffic for a fashion web store
for example can consist of traffic from distinct campaigns for
shoes, jackets and accessories. These campaigns can be further
sub-categorised based on target demographics, locations and
interests. The performance of each one of those parts can vary
significantly. The study granularly examined all these under
the surface dynamics.
The adoption of clustering techniques in this context
enabled the systematic performance analysis of a significantly
higher number of observations and variables compared to
stage one. The additional variables incorporated
were engagement, represented by pages per visit, transaction
volume as well as revenue and cost per transaction. The
generated performance-based clusters according to Euclidean
distance were visualised through dendrograms and heatmaps.
F. Predictive modelling
The predictive modelling part aimed to make use of
enriched session level data across multiple metrics and
dimensions in order to address the conversion performance
question holistically.
An initial naive approach was to examine all possible
combinations of the available dimensions in order to identify
the combination of dimensions associated with best
conversion rate performance. While useful to identify
segments with high conversion rate, this approach lacked
generalisability with new data and did not account for possible
interactions between the various dimensions. To overcome
this, several machine learning methods were selected,
implemented and benchmarked against each other.
The nature of the problem of conversion prediction,
naturally led to binary classification algorithms. The standard
and most widely used method in this area is logistic

regression. The nature of the dataset itself however made the
selection of alternative algorithms more appropriate. The first
algorithm implemented was a decision tree.
1) Decision trees
A decision tree follows the divide and conquer method of
recursive partitioning. Its main advantage over logistic
regression is that is has native methods to handle a
large quantity of both numerical and categorical variables,
including ones that have a high number of levels. Moreover,
data preparation steps such as normalisation, creation of
dummy variables and removal of blank values are not
required.
Decision trees are also easy to interpret as they mimic
the human decision making process and are not very
computationally expensive (logarithmic cost as a function of
the number of data points used to train the tree).
However, decision trees are not free of disadvantages. The
main drawback is their high variability. Relatively low scale
changes in the data can have a high impact in the final trees
generated. Moreover, decision trees can generate overly
complex trees, lacking generalisability if they are not properly
pruned.
2) Random forests
Random forests are lacking in interpretability compared to
decision trees but they can address some of the key
shortcomings mentioned above. This is thanks to the ensemble
learning method which is based on the generation of high
numbers of trees with samples -with replacement- from the
available cases and variables. The results of the multiple
predictive models are then aggregated and the final outcome
depends on the majority vote. In this way, lower variance
compared to simple decision trees is achieved.
An additional feature of the random forests is the provision
of a variable importance score. This score can be
calculated according to the amount of predictive accuracy loss
when each of the variables in the model is forced to be absent
from the model generation process. These scores provide an
estimate of the impact of the presence of those key variables.
In the context of the current conversion analysis, this is one of
the defined objectives of the project.
3) Support vector machines
A third option included for comparison was support vector
machines, a popular algorithm which is well known for both
its complexity and its prediction accuracy for classification
and regression problems. However, not unlike random forests,
they cannot be used to intuitively interpret the results.
As part of the methodology, the three selected models
were trained, tuned with cross-validation and tested on
hypothetically unseen data from a test dataset. Key
performance metrics were calculated for each of the models
including accuracy, sensitivity, specificity and area under the
curve. The performance between the models was compared
based on those metrics.
G. Data access
As with the vast majority of websites, the e-commerce site
under study uses Google Analytics to track the visitors’
behaviour on the website. While Google Analytics provides a
high number of functionalities, it typically cannot be used to
access data of a more granular form also known as clickstream
data. It is instead developed to be used via a user interface and
report data in aggregate form. Instead of accessing and
exporting data via the User Interface, Google Analytics Core
Reporting API was used.
There were several benefits in making that choice:
The API provides access to richer datasets by
allowing simultaneous access to multiple dimensions
and metrics compared to the limited amount available in the
UI. It also mitigates the effect of sampled data returned which
is common when large amounts of data are requested. In more
general terms, accessing data via the API enables automation,
reproducibility and easy handling of larger volumes of data. In
terms of authentication and authorisation, to access the data,
the only requirements client and secret ID and the creation of a
project in Google Developers Console.
H. Initial variable selection
For the purpose of this study the capacity of the API was
reached by involving all the 7 possible dimensions and 13
metrics, which correspond to categorical and numerical
variables.
Moreover, the query to the API was made in such a
combination of dimensions with the purpose of segmenting
the data to such a high degree that the final outcome would
essentially be a session based dataset. For example, by using
temporal dimensions such as day -hour- minute combined
with the IP provider, user location and traffic source, it is very
unlikely that there will be more than one session involved for
each of the records returned. In this way, a move from
aggregate data to virtually session -level data was achieved.
For different stages of the analysis, different filters were
applied in the data sets. Where necessary, some of the metrics
columns were removed where multicollinearity issues were

present (for example, between session duration and session
page depth).
III. IMPLEMENTATION
The implementation involved several steps that included the
pre-processing of the data, some degree of feature engineering
and special steps to address the imbalances class challenge.
A. Steps of implementation
-The first stage of ratios analysis required the addition of new
calculated fields for the ratio KPIs, but did not require any
complex operations on the data.
-For the clustering part, the data were broken down by ad-
group level and then scaled before the clustering algorithm
was applied.
-Scaling was also required for the support vector machine
algorithm implementation.
-In general terms, the predictive modelling part was the most
demanding in terms of pre-processing and transformations.
This allowed the data to take the right shape and type to
permit effective application of the learning algorithms.
B. Data pre-processing
Key data preparation activities are highlighted below. The
main purpose of making those transformations was either to
generate additional more relevant predictors or to convert the
existing ones into a shape and form that is required for the
implementation of one or more of the algorithms.
 Session data made "almost" granular
 Invalid sessions were removed
 Highly correlated variables were removed
 Data were split into train and test (0.8 split ratio)
 Day of the week was extracted from date
 Days since last session placed in buckets
 Date converted to weekday or weekend
 Date-hour was split in two component variables
 Geo data were split into sub-continents
 Hour was converted to AM or PM
 Seed was selected to ensure determinism
C. The imbalanced class challenge
One of the key challenges with respect to the methodology
was the presence of class imbalance in relation to the
conversion outcome. In such cases, the usefulness of
prediction accuracy as a metric of performance evaluation can
be limited [16]. If a website has a conversion rate of 98%, then
a prediction that every session will lead to a non-conversion
event will be accurate 98% of the time, which is very high but
with little practical importance.
To mitigate the impact of class imbalance, metrics such
as sensitivity and specificity and their interaction -in terms of
the area under the curve- were calculated. To improve the
modelling outcomes, the algorithms need to identify the rare
cases-which are also the cases of interest. For this purpose, it
is common to oversample the minority class, under-sample the
majority class or penalise outcomes according to the various
types of possible prediction error [17, 18].
A hybrid approach was selected to address the imbalanced
class challenge. The majority class, i.e. non conversion,
represented over 98% of the observations. The page depth
instead was used as a proxy for conversion. This was based on
the observation that the likelihood of conversion tends to
increase in an accelerated way as the number of pages
accessed during a session increase. As displayed in Table 1,
there is a leap in conversion rate when the number of pages
exceeds five.
For the purposes of this project, the proxy for conversion
was set to correspond to sessions with page depth higher than
5. This approach represented a combination of oversampling
the minority class and under-sampling the majority class at the
same time. The aim was to increase the algorithmic sensitivity
to the positive cases of interest.
Table 1 Conversion rate as function of page depth

IV. RESULTS
The analysis involved three stages and the results for each
of them is presented and discussed separately in the sections to
follow. Even though the results refer to the specific website
under study, the methodology is valid for any e-commerce
website that uses Google Analytics, with possibly some minor
adjustments.
A. Ratio Analysis
In the conversion quality index analysis, the three major
traffic channels were explored i.e. display advertising, search
advertising and referral traffic. Figure 1 serves as context by
providing a scatter plot illustrating the percentage of
conversions generated by each medium.
Figure 1 Percentage of conversions by traffic source in time
Figure 2 is a scatter plot representation of their respective
conversion quality. The trend-lines in both cases are modelled
as local regressions and the grey bands around them serve as
their confidence bands. The white dotted line that meets the y
axis at y=1 corresponds to the level where the percentage of
sessions equal the percentage of conversions with respect to
the total.
Display advertising is consistently below the white horizontal
line which suggests that the medium performs lower than
expected or average with respect to the KPI under study.
Search advertising data points are scattered along both sides of
the line suggesting a normal or "as expected" behaviour.
Referral traffic however is visibly above both search and
display which suggests strong performance. “Referral”
represents traffic from other, typically highly relevant,
webpages that include a non-ad link to the e-commerce
website. This can be considered as recommendation. For
example, a blog might contain a referral link to the website
under study along with a comment about its good quality or
having a reference to a promotion etc.- not all incoming links
are of course always positive and not all of them are of equal
worth.
Figure 2 Conversion Quality Index by traffic source in time
The result of this analysis in any case illustrates that digital
"word of mouth" traffic is by a large margin the most effective
in terms of propensity to convert. It is fair to mention however
that this type of traffic tends to be lower in volume compared
to the other two types. This is also evident in Figure 1, where
referral medium accounts for visibly less conversions in
relative terms.
Search advertising not unsurprisingly performs better than
display advertising. Search advertising in general tends to be
more targeted. This is because the user, by inputting a search
query, expresses a specific intent about a specific product or
service - in the case of a commercially driven query. Display
advertising on the other hand often results in increase of brand
awareness which however does not directly translate into a
conversion.
The conversion quality analysis has the benefit of
providing a high level overview of conversion with respect to
traffic sources. This by itself can reveal opportunities and
areas of concern. However, it does not provide any insights as
to what happens under the surface for each of the traffic
channels analysed. Instead, it helps to raise those questions in
a more specific form, by employing more granular types of
analysis and ideally additional inputs such as cost and revenue
related data.
B. Clustering
As evidenced by the conversion quality analysis, search
advertising (which in practice is mainly associated with
Google AdWords) is the medium generating the highest
volume of conversions and its performance can be considered
as fair. Google Analytics provides access to a wide range of
AdWords data including the break down into campaigns and

ad-groups, as well as associated costs and revenue. Given the
importance of this medium and the efficient data integration
with Google Analytics, the clustering part of the project was
centered on a more granular study of AdWords’ ad-group
performance. The data was enriched also with other highly
relevant metrics. The real names of the ad groups have been
masked and instead coded names were used. The scaled
version of the first ten ad-groups is displayed in Table 2. This
operation is a required step in the implementation process in
order to minimise the impact of variables being expressed in
ranges with very different spans.
Table 2 Ad-group values were scaled prior to the clustering
implementation
The hierarchical clustering application was represented
with the dendrogram in Figure 3. The number of clusters is an
arbitrary decision that depends on context. From a practical
perspective, it is preferable to cut the tree where the distance
between branch levels tends to be higher. In Figure 3, three
clusters are highlighted. While knowing about the clusters of
specific ad-groups based on their similarity was useful, it was
considered preferable to also visualise (and colour code) the
scaled numeric variables, based on which the similarity
clusters were generated.
Figure 3 Ad-groups clustered according to Euclidean distance
Figure 4 is the heatmap that provided this additional
insight and allowed for better comparisons. For visualisation
reasons, the largest cluster was separated by additional white
lines.
The first cluster is an individual ad-group associated with
high -green in colour- values across almost all metrics
including transactions, revenue, cost and sessions. This
certainly highlights the high importance of the ad-group and it
is likely that it needs individual attention to ensure continuous
positive performance.
The second cluster represents ad-groups with high
potential: the transaction and revenue figures are high, while
the cost is average. Moreover, the engagement, represented by
pages per session is very high. Those observations
signal opportunity and a possible course of action would be to
increase the associated budget for the given ad-groups in order
to receive more qualified traffic.
Figure 4 Heatmap of ad-group clustering based on 5 key
conversion related features
The third cluster is in many ways the opposite of cluster 2,
as it represents higher than average costs while in many cases
the associated revenue does not reach those heights. A
possible course of action would be to transfer budget from
cluster three to cluster two. Several ad-groups have a high
engagement rate, but this does not translate in high volumes of
conversion. This might be a signal that users who land on the
relevant product pages cannot easily find the product that was
advertised or even that the conversion process faces some
technical issue.

Regarding the interpretation of the results of hierarchical
clustering, it is worth keeping in mind that those results still
have to be validated by further analysis and testing. However,
they can be an excellent starting point for making informed
hypotheses that can lead to opportunities for the business.
C. Prediction
1) Decision trees
Similar types of clustering analysis can be performed for
the other traffic sources as well, as long as it is possible to join
the analytics data with other sources of data. In this case, the
data could contain additional attributes; for example, revenue
and costs for Facebook display advertising. At this stage the
analysis reached a deeper level of granularity; however, it was
still based on a relatively small set of mainly numerical
variables focused on sessions and transactions.
Figure 5 The final decision tree containing 4 splits
The predictive modelling part incorporated many more
features relating to the conversion outcome such as location,
day of week, browser type among several others and
attempted to offer a more holistic approach to what drives
conversion and how conversion can be predicted.
The first among the methods was the decision tree. Table 3
illustrates the cross validated errors based on different values
corresponding to levels of the complexity parameter and
resulting number of splits. The cross validated error was
minimised when the tree had four splits.
Table 3 Cross-validation results for decision tree with respect to
the complexity parameter
The generated tree is visualised in Figure 5. The
interpretation is intuitive and some of the key rules include the
following:
-When the visitor is new, as opposed to returning, the session is
predicted to not convert.
-If the visitor is a return visitor and their operating system is
not in the list displayed below the second node from the top
(which mainly includes mobile operating systems) and also the
traffic source is neither cpc-cost per click (i.e. search) nor
display and the day since last session equals zero, then the
session is predicted to convert.
-If the visitor is a return visitor but the operating system is one
of the ones in the same list as above then the event is non-
conversion.
The tree contains 4 splits and multiple nodes so there are
many other rules. By observing the conditions under which one
branch is selected over another, it is possible to make a further
hypothesis about the factors that are critical as to whether a
session will convert or not.
Already highlighted above is the choice of operating system
and the traffic source. Mobile operating systems are not
associated with conversion events. The absence of search or
display ads as traffic source is associated with sessions that
convert. Combined with the findings of the first part of the
analysis, this suggests that the referral traffic source is the one
that increases the likelihood to convert.
While those rules are simple and intuitive to follow, it is
important to emphasise that they cannot be considered
independently. They are part of a sequence of top down rules
and some relatively small changes in the data can result in the
creation of a different set of rules.
The produced tree is used to predict the data for the test
data set. The confusion matrix below illustrates the
relationship between actual and predicted values. The tree
algorithm succeeds in predicting the non-conversion events
but the prediction of conversion as the positive case is
challenging.

Table 4 Confusion matrices
Decision Tree Random Forest SVM
2) Random forests
To overcome those challenges, a random forest model was
generated with the development of 500 individual trees.
Random forests offer a natural measure for variable
importance thanks to which the observation regarding
important variables can take a more systematic form. In fact,
for every variable used in the model, random forests generate
a comparable score.
Figure 6 ranks the predictor variables according to the
mean decrease accuracy score. Some similarities compared to
the earlier observations are evident, even though the priority is
not the same. The traffic source (medium) is the most critical
variable followed closely by the operating system and visitor
type. Additional predictors that play a role are sub-continent
and days since last session. The subcontinent factor is not
surprising given that the e-commerce site has specific
countries and regions of focus. Regarding the “days since last
session” variable, if this information is cross referenced with
the decision tree, it would suggest that unless the interval
between the last session and current session is no longer than a
day, then chances of conversion deteriorate.
Figure 6 Random forest variable importance dotplot based on
500 trees
3) Support vector machines
The third and last model test was based on the support
vector machines algorithm. This model did not provide any
relevant elements that could be visualised. Despite this,
elements pertaining to its predictive performance were
observed and compared with the respective performance of the
decision tree and random forest algorithm.
4) Model comparison
Table 5 illustrates the performance of the three algorithms
across some of the most widely used performance metrics
Table 5 Learning algorithm performance comparison
Figure 7 Area Under the ROC Curve for the 3 selected models
Figure 7 illustrates that the support vector machine
algorithm is associated with a line of higher area under the
curve compared to the other algorithms. Even though accuracy
is not the highest, given the binary classification nature of the

problem, it was deemed most appropriate to adopt AUC as the
performance metric of reference. In fact, the AUC of support
vector machine is the only one that exceeded the area
referenced by the continuous grey line. This line represents the
Null model or in other words the model that classifies samples
based on random chance. Therefore, for the purpose of this
project, support vector machines was considered as the best
performing algorithm for the prediction of conversion.
D. Summary and evaluation of results
This project examined the e-commerce conversion from
various perspectives and the main finding can be summarised
as follows:
The creation of a conversion quality index is a very
efficient way to get a high level understanding of the traffic
dynamics for the website, with respect to the various traffic
sources. Referral traffic, despite its relatively low volume,
outperforms any other traffic sources with respect to
conversion performance. Search marketing performance is
around the expected average but display lags behind in
performance.
The clustering technique was effective in uncovering
hidden structure among the components of the search
marketing ad groups. This analysis suggested clusters of ad-
groups with high potential which may be worth of additional
investment and attention. It also suggested other clusters that
are under-performing from a cost/revenue analysis point of
view and a third cluster that could be associated with
unresponsive or suboptimal design that does not allow users to
easily reach the product they are interested in.
Predictive modelling allowed for a more holistic approach
to conversion question by involving fine grained observations
and multiple predictor variables. The results suggested that
support vector machines is the best performing algorithm in
terms of AUC score. Decision tree analysis provided an
intuitive way to visualise rules that describe conversion and
non-conversion events. Random forest variable importance
suggested that the key drivers of conversion are the traffic
channel, the visitor type (i.e. new or returning) and the
operating system.
E. Suggested system structure
Based on the outcomes of the project the following system
structure is suggested with the aim of ensuring efficient flow of
data and computation.
 The system will consist of a data pipeline that will
initially retrieve data from the Google Analytics API
and will store them in a relational database.
 Specialised analytics software will access the data,
perform required manipulation and pre-processing
until data are in the right shape and have the required
features.
 A machine learning algorithm will then output class
and probability predictions regarding conversion or
not on a per user/session basis.
V. CONCLUSIONS
The project has suggested new methods for the analysis of
conversion by adopting a data driven analytics approach. This
approach does not depend on the development of observation
experiments or availability of custom web log analysis
software. Unlike previous research that focused mainly on
page path analysis, this project involved many additional
parameters at the user level. However, it did not require the
presence of logged in users. Additionally, it placed a focus on
the examination of traffic channels such as search, display and
referral which are the key acquisition channels for many
businesses on the web.
Moreover, the project: (1) Proposed a methodology to
access the right granularity of data to allow for non-standard
data analyses. (2) Examined thoroughly the process of user
conversion in both a descriptive and predictive sense, by using
supervised and unsupervised learning techniques while
also addressing inherent class imbalance challenges. (3)
Reached all the above goals within a methodology framework
that is reproducible as well as tested and validated on out of
sample data.
As a result of the proposed methodology:
 E-commerce website managers and analysts can
move beyond the “forced” use of aggregate data
provided in the front end of Google Analytics.
 The outcomes of the analysis can support decisions
regarding investment in the right digital marketing
strategies and channels and improvements in website
design.
 The websites can make more informed decisions
regarding the characteristics of the desired potential
customers to target or re-target.
 The websites could even develop a responsive system
that can optimise the website content and navigation
and make simple recommendations based on the
conversion probability of users in real time.
At the same time it should also be acknowledged that the
proposed system faces several limitations.
The predictive ability of the model is marginally
acceptable, as it was seen in the AUC figure. Steps were taken
to address the imbalanced class issue by involving page depth
as proxy for conversion. The objective was only partly
accomplished. The dataset even after transformation was still

not entirely balanced and this is likely to have had an impact on
the final model performance.
One aspect that can certainly be improved is the parameter
tuning process. The project only tested a limited number of
possible parameters but the results would likely be better with a
more extensive parameter tuning operation that would test
multiple combinations of parameters for each of the models.
Similarly, the testing of additional models could further
improve the existing performance.
Moreover, even though the API enables the access to
multiple dimensions, it does not offer access to
all available dimensions at the same time, thereby limiting the
study in this respect.
With respect to clustering, it is fair to recognise that this
can be considered an exploratory method. While it can suggest
reasonable hypotheses, it does not provide the means of
validating them. A possible extension to this research would
be to apply statistical techniques such as analysis of variance in
order to validate the clusters or suggest alternative
formulations.
It is important to note that the conversion analysis was
based on the assumption of last click conversion, i.e. by
attributing a conversion to the last click entirely. While this is
the simplest and possibly most widely used model, it does
not reveal the whole truth. In many cases visitor sessions prior
to the one that led to a conversion can play an important role.
However, this is ignored in the absence of a holistic attribution
model. This area could be the subject of some future research
in the field.
The next step for the project would be to productise this
analysis by developing a custom application that would
integrate with the free Google Analytics product. It would
automatically transform the data and execute on demand all the
modelling parts. A future development can also involve real
time processing of the data that would then feed into
personalisation systems and recommendation engines. This
phase would also require a new level of management of the
data flows and a highly efficient production code framework to
optimise for speed and overall stability of the system.
VI. REFERENCES
[1] K. Gold, “What is the Average Conversion Rate? A 2013
Update - Search Marketing Standard Magazine | Covering
Search Engines,” 22-Aug-2013. .
[2] J. Qiu, “A predictive Model for Customer Purchase
Behavior in E-Commerce Context.,” in PACIS, 2014, p.
369.
[3] H.-F. Lin, “Predicting consumer intentions to shop
online: An empirical test of competing theories,”
Electron. Commer. Res. Appl., vol. 6, no. 4, pp. 433–442,
Dec. 2007.
[4] S. Arulkumar and D. Kannaiah, “Predicting Purchase
Intention of Online Consumers using Discriminant
Analysis Approach.”
[5] Y. Zhang and M. Pennacchiotti, “Predicting purchase
behaviors from social media,” in Proceedings of the 22nd
international conference on World Wide Web, 2013, pp.
1521–1532.
[6] A. Bulut, “TopicMachine: Conversion Prediction in
Search Advertising Using Latent Topic Models,” IEEE
Trans. Knowl. Data Eng., vol. 26, no. 11, pp. 2846–2858,
Nov. 2014.
[7] F. Wu, I.-H. Chiu, and J.-R. Lin, “Prediction of the
intention of purchase of the user surfing on the Web using
hidden Markov model,” in Proceedings of ICSSSM’05.
2005 International Conference on Services Systems and
Services Management, 2005., 2005, vol. 1, pp. 387–390.
[8] E. Suh, S. Lim, H. Hwang, and S. Kim, “A prediction
model for the purchase probability of anonymous
customers to support real time web marketing: a case
study,” Expert Syst. Appl., vol. 27, no. 2, pp. 245–255,
Aug. 2004.
[9] H. Yanagimoto and T. Koketsu, “User intent prediction
from access logs of an online shop,” IADIS Int. J.
WWWInternet, vol. 12, no. 1, 2014.
[10] D. Van den Poel and W. Buckinx, “Predicting online-
purchasing behaviour,” Eur. J. Oper. Res., vol. 166, no.
2, pp. 557–575, Oct. 2005.
[11] A. Vieira, “Predicting online user behaviour using deep
learning algorithms,” ArXiv Prepr. ArXiv151106247,
2015.
[12] W. W. Moe and P. S. Fader, “Dynamic Conversion
Behavior at E-Commerce Sites,” Manag. Sci., vol. 50, no.
3, pp. 326–335, Mar. 2004.
[13] W. W. Moe and P. S. Fader, “Capturing evolving visit
behavior in clickstream data,” J. Interact. Mark., vol. 18,
no. 1, pp. 5–19, Jan. 2004.
[14] C. Sismeiro and R. Bucklin, “Modeling Purchase
Behavior at an E-Commerce Web Site: A Task-
Completion Approach,” J. Mark. Res., vol. 41, no. 3, pp.
306–323, 2004.
[15] B. Clifton, Advanced metrics with Google Analytics,
Third. Wiley & Sons, 2012.
[16] M. Sokolova and G. Lapalme, “A systematic analysis of
performance measures for classification tasks,” Inf.
Process. Manag., vol. 45, no. 4, pp. 427–437, Jul. 2009.
[17] V. Ganganwar, “An overview of classification algorithms
for imbalanced datasets,” Int. J. Emerg. Technol. Adv.
Eng., vol. 2, no. 4, pp. 42–47, 2012.
[18] N. V. Chawla, “Data mining for imbalanced datasets: An
overview,” in Data mining and knowledge discovery
handbook, Springer, 2005, pp. 853–867.

E com conversion prediction and optimisation

E com conversion prediction and optimisation

More Related Content

Similar to E com conversion prediction and optimisation (20)

More from Alex Papageorgiou (11)

Recently uploaded (20)

E com conversion prediction and optimisation