L 13 (17-05-21) data collection methodology

DATA
COLLECTION
METHODOLOGY
& ANALYSIS
L – 13 (17/05/21)
Data Analysis
&
Machine Learning
Presented By
Dr. M Yusuf Alam
Associate Professor
BBIT, Kolkata

There are several methods of
collecting primary data:
--- particularly in surveys and
descriptive researches.
In descriptive research, we obtain
primary data either through observation
or through direct communication with
respondents in one form or another or
through personal interviews.
• These are already available i.e. they
refer to the data which have already
been collected and analyzed by
someone else.
• Secondary data may either be published
or unpublished data. Researcher must
be very careful in using secondary data,
because the data available may be
sometimes unsuitable.
COLLECTION OF
SECONDARY
DATA
COLLECTION
OF PRIMARY
DATA

Methods of Data Collection
Primary Data
1) OBSERVATION METHOD :
Observation method is a method
under which data from the field is
collected with the help of
observation by the observer or by
personally going to the field.
--- In the words of P.V. Young,
“Observation may be defined as
systematic viewing, coupled with
consideration of seen phenomenon.”
TYPES OF OBSERVATION
Structured and Unstructured
Observation
When observation is done by
characterizing style of recording
the observed information,
standardized conditions of
observation , definition of the units
to be observed , selection of
pertinent data of observation then it
is structured observation
When observation is done without
any thought before observation
then it is unstructured observation
3

Non Participant Observation
When observer is observing people without giving any information to
them then it is non participant observation
4
Participant & Non Participant Observation
When the Observer is member of the group which he is observing then it is
Participant Observation
In participant observation Researcher can record natural behavior of group
Researcher can verify the truth of statements given by informants in the context
of questionnaire
Difficult to collect information obtaining through this method but in this way,
researcher may loose objectivity of research due emotional feelings.

5
Controlled & Uncontrolled Observation
When the observation takes place in natural
condition i.e. uncontrolled observation.
It is done to get spontaneous picture of life and
persons
When observation takes place according to definite
pre arranged plans , with experimental procedure
then it is controlled observation generally done in
laboratory under controlled condition.

INTERVIEW METHOD•
This method of collecting data involves presentation or oral-verbal stimuli and
reply in terms of oral-verbal responses.
Interview Method This is Oral Verbal communication .
Where interviewer asks questions( which are aimed to get information
required for study ) to respondent There are different type of interviews as
follows :
PERSONAL INTERVIEWS : The interviewer asks questions generally in a
face to face contact to the other person or persons.
6

7
Characteristics of Personal Interview
Structured Interview
Flexibility in asking questions
Predetermined questions
Standardized techniques
No Predetermined questions recording
No Standardized techniques of procedure laid down i.e. asking recording
questions in form & order prescribed
Interviewer has freedom to ask
Time required for such interview omit.
Non structured manner interview
add questions if any
Ask questions without following sequence
Not necessary of skill or specific knowledge, Deep knowledge & skill
Analysis of data becomes easier required Bcoz information is collected
Analysis of data is difficult in prescribed manner

TELEPHONIC INTERVIEWS
Contacting samples on telephone
Uncommon method may be used in developed regions
MERITS• Flexible compare to mailing method• Faster than other
methods• Cheaper than personal interview method• Callbacks are
simple and economical also• High response than mailing method.•
when it is not possible to contact the respondent directly, then
interview is conducted through – Telephone.
8
Replies can be recorded without embarrassment to respondents•
Interviewer can explain requirements more easily• No field staff is
required• Wider distribution of sample is possible
DEMERITS • Little time is given to respondents • Survey is
restricted to respondents who have telephones • Not suitable for
intensive survey where comprehensive answers are required •
Bias information may be more • Very difficult to make
questionnaire because it should short and to the point

Structured interviews –
in this case, a set of pre- decided questions are there.
Unstructured interviews –
in this case, we don’t follow a system of pre-determined
questions.
Focused interviews -- attention is focused on the
given experience of the respondent and its possible
effects.
Clinical interviews -- concerned with broad
underlying feelings or motivations or with the course of
individual’s life experience, rather than with the effects of
the specific experience, as in the case of focused
interview.
9

Group interviews --
A group of 6 to 8 individuals is interviewed.
Qualitative and quantitative interviews --
divided on the basis of subject matter i.e. whether
qualitative or quantitative.
Individual interviews -- interviewer meets a single
person and interviews him.
Selection interviews -- done for the selection of
people for certain jobs.
Depth interviews -- it deliberately aims to elicit
unconscious as well as other types of material relating
especially to personality dynamics and motivations.
10

11
QUESTIONNAIRE METHOD
This method of data collection is quite popular, particularly in case
of big enquiries.
The questionnaire is mailed to respondents who are expected to
read and understand the questions and write down the reply in the
space meant for the purpose in the questionnaire itself.
The respondents have to answer the questions on their own.
Questionnaire is sent to persons with request to answer the
questions and return the questionnaire
Questions are printed in definite order, mailed to samples who are
expected to read that questions understand the questions and
write the answers in provided space.

12
QUESTIONNAIRE METHOD
 Merits of Questionnaire – It is Low cost even the geographical area is
large to cover
Answers are in respondents word so free from bias
Adequate time to think for answers
Non approachable respondents may be conveniently contacted
Large samples can be used so results are more reliable
 Demerits of Questionnaire -- Low rate of return of duly filled
questionnaire
Can be used when respondent is educated and co operative
It is inflexible Omission of some questions
Difficult to know the expected respondent have filled the form or it is
filled by some one else
Slowest method of data collection

13
Essentials of Good Questionnaire —
It Should Short & simple Questions should arranged in logical
sequence (From Easy to difficult one)
Technical terms should avoided Some control questions which
indicate reliability of the respondent ( To Know consumption first
expenditure and then weight or qty of that material)
Questions affecting the sentiments of the respondents should
avoided
Adequate space for answers should be provided in questionnaire
Provision for uncertainty (do not know, No preference)
Directions regarding the filling of questionnaire should be given
Physical Appearance - - Quality of paper, color

14
HOW TO CONSTRUCT A QUESTIONNAIRE
Researcher should note the following with regard to these three
main aspects of a questionnaire:
• General form
• Question Sequence
• Determine the type the Questions –
• A) Direct Question
• B) Indirect Question
• C) Open Form Questionnaire
• D) Closed Form Questionnaire
• E) Dichotomous Questions
• F) Multiple Choice Questions (MCQ)

15
Secondary Data Sources of data --
Publications of Central, state , local government• Technical and
trade journals• Books, Magazines, Newspaper• Reports &
publications of industry ,bank, stock exchange• Reports by
research scholars, Universities, economist• Public Records.
Factors to be considered before using secondary data --
Reliability of data – Who, when , which methods, at what time etc.
Suitability of data – Object ,scope, and nature of original inquiry
should be studied, as if the study was with different objective then
that data is not suitable for current study
Adequacy of data -- Level of accuracy, • Area differences then
data is not adequate for study

16
Schedules --
Very similar to Questionnaire method
The main difference is that a schedule is filled by
the enumerator who is specially appointed for the
purpose.
Enumerator goes to the respondents, asks them
the questions from the Questionnaire in the order
listed, and records the responses in the space
provided.
Enumerator must be trained in administering the
schedule.

17
Questionnaire Vs. Schedule Questionnaire –
 Q generally send to through mail and no further
assistance from sender.
 Q is cheaper method.
 Non response is high
 Incomplete and wrong information is more.
 Depends on the quality of questionnaire Schedule
 Schedule is filled by the enumerator or research
worker.
 Costly requires field workers.
 Non response is low
 Depends on Honesty of the enumerator.
 Relatively more correct and complete

Rating Scale –
Ratting is term applied to express
opinion or judgment regarding
some situation, object or
character.
Opinions are usually expressed on
a scale of values; rating techniques
are devices by which such
judgments may be quantified.
“Rating is an essence and direct
observation.” -- Ruth Strong
“A rating scale ascertains the
degree, intensity and frequency of
a variable.” -- Von Dallen
Rating techniques are more commonly used in
scaling traits and attributes.
A rating method is a method by which one
systematizes, the expression of opinion
concerning a trait.
The rating is done by parents, teachers, a
board of interviewers and judges and even by
the self as well.
The special feature of rating scale is that the
attitudes are evaluated not on the basis of the
opinions of the subjects but on the basis of
the opinions and judgments of the
experimenter himself.
In rating scale data are collected by; Verbal
behavior, facial expression, personal
documents, clinical type interview, projective
techniques and immediate experiences as
emotions, thoughts and perceptions.
18

DATA ANALYSIS
Kaul defines data analysis as, ”Studying the
organized material in order to discover inherent
facts. The data are studied from as many angles as
possible to explore the new facts.”

DATA ANALYSIS
Data analysis embraces a whole range of activities of both the qualitative and
quantitative type. It is usual tendency in behavioral research that much use of quantative
analysis is made and statistical methods and techniques are employed.
The following are the main purposes of data analysis:
(i) Description: It involves a set of activities that are as essential first step in the development of
most fields. A researcher must be able to identify a topic about which much was not known;
he must be able to convince others about its importance and must be able to collect data.
(ii) Construction of Measurement Scale: The researcher should construct a measurement scale.
All numbers generated by measuring instruments can be placed into one of four categories:
Nominal, Ordinal, Interval, and Ratio Scale.
(iii) Generating empirical relationships: Another purpose of analysis of data is identification of
regularities and relationships among data. The researcher has no clear idea about the
relationship which will be found from the collected data. If the data were available in details it
will be easier to determine the relationship.
(iv) Explanation and prediction: Generally knowledge and research are equated with the
identification of causal relationships and all research activities are directed to it. But in many
fields the research has not been developed to the level where causal explanation is possible
or valid predictions can be made.
20

Types of Data Analysis:
Techniques and Methods
There are several types of Data Analysis techniques that exist based on
business and technology. However, the major Data Analysis methods are:
•Text Analysis
•Statistical Analysis
•Diagnostic Analysis
•Predictive Analysis
•Prescriptive Analysis

DATA ANALYSIS
• Text Analysis
Text Analysis is also referred to as Data Mining. It is one of the methods of data analysis to discover
data sets using databases or data mining tools. It used to transform raw data into business
• Statistical Analysis
Statistical Analysis shows "What happen?" by using past data in the form of dashboards. Statistical
Analysis includes collection, Analysis, interpretation, presentation, and modeling of data. It
analyses a set of data or a sample of data. Two types- Descriptive and Inferential Analysis
• Diagnostic Analysis
Diagnostic Analysis shows "Why did it happen?" by finding the cause from the insight found in
Statistical Analysis. This Analysis is useful to identify behavior patterns of data. If a new problem
arrives in your business process, then you can look into this Analysis to find similar patterns of
that problem. And it may have chances to use similar prescriptions for the new problems.
• Predictive Analysis
Predictive Analysis shows "what is likely to happen" by using previous data. The simplest data
analysis example is like if last year I bought two dresses based on my savings and if this year my
salary is increasing double then I can buy four dresses. But of course it's not easy like this because
you have to think about other circumstances like chances of prices of clothes is increased this year
or maybe instead of dresses you want to buy a new bike, or you need to buy a house!
• Prescriptive Analysis
Prescriptive Analysis combines the insight from all previous Analysis to determine which action to
take in a current problem or decision. Most data-driven companies are utilizing Prescriptive
Analysis because predictive and descriptive Analysis are not enough to improve data performance.
Based on current situations and problems, they analyze the data and make decisions.
22

The Data Analysis Process is nothing but gathering information by using a
proper application or tool which allows you to explore the data and find a
pattern in it. Based on that information and data, you can make decisions, or
you can get ultimate conclusions.
DATA ANALYSIS
PROCESS

Data Analysis consists of the following phases –
Data Requirement Gathering
Data Collection
DataCleaning
DataAnalysis
Data Interpretation
DataVisualization
First of all, you have to
think about why do you
want to do this data
analysis? All you need to
find out the purpose or aim
of doing the Analysis of
data. You have to decide
which type of data analysis
you wanted to do! In this
phase, you have to decide
what to analyze and how
to measure it, you have to
understand why you are
investigating and what
measures you have to use
to do this Analysis.
After requirement
gathering, you will get a
clear idea about what things
you have to measure and
what should be your
findings. Now it's time to
collect your data based on
requirements. Once you
collect your data, remember
that the collected data must
be processed or organized
for Analysis. As you
collected data from various
sources, you must have to
keep a log with a collection
date and source of the data.
Now whatever data is
collected may not be
useful or irrelevant to your
aim of Analysis, hence it
should be cleaned. The
data which is collected
may contain duplicate
records, white spaces or
errors. The data should be
cleaned and error free.
This phase must be done
before Analysis because
based on data cleaning,
your output of Analysis will
be closer to your expected
outcome.
24
DATA REQUIREMENT
GATHERING
DATACOLLECTION
Once the data is collected,
cleaned, and processed, it is
ready for Analysis. As you
manipulate data, you may
find you have the exact
information you need, or
you might need to collect
more data. During this
phase, you can use data
analysis tools and software
which will help you to
understand, interpret, and
derive conclusions based on
the requirements.
DATACLEANING DATAANALYSIS
After analyzing your data, it's finally time to interpret
your results. You can choose the way to express or
communicate your data analysis either you can use
simply in words or maybe a table or chart. Then use the
results of your data analysis process to decide your
best course of action.
DATA INTERPRETATION
Data visualization is very
common in your day to day
life; they often appear in the
form of charts and graphs.
In other words, data shown
graphically so that it will be
easier for the human brain
to understand and process
it. Data visualization often
used to discover unknown
facts and trends. By
observing relationships and
comparing datasets, you
can find a way to find out
meaningful information.
DATA VISUALIZATION

Data Analysis Techniques
There are different techniques for Data
Analysis depending upon the question at
hand, the type of data, and the amount of
data gathered. Each focuses on strategies
of taking onto the new data, mining insights,
and drilling down into the information to
transform facts and figures into decision
making parameters. Accordingly, the
different techniques of data analysis can be
categorized as follows:
25

TECHNIQUES BASED ON
MATHEMATICS AND STATISTICS
•Descriptive Analysis: Descriptive Analysis takes into account the historical data, Key Performance Indicators,
and describes the performance based on a chosen benchmark. It takes into account past trends and how they
might influence future performance.
•Dispersion Analysis: Dispersion in the area onto which a data set is spread. This technique allows data
analysts to determine the variability of the factors under study.
•Regression Analysis: This technique works by modeling the relationship between a dependent variable and
one or more independent variables. A regression model can be linear, multiple, logistic, ridge, non-linear, life
data, and more.
•Factor Analysis: This technique helps to determine if there exists any relationship between a set of variables.
In this process, it reveals other factors or variables that describe the patterns in the relationship among the
original variables. Factor Analysis leaps forward into useful clustering and classification procedures.
•Discriminant Analysis: It is a classification technique in data mining. It identifies the different points on
different groups based on variable measurements. In simple terms, it identifies what makes two groups different
from one another; this helps to identify new items.
•Time Series Analysis: In this kind of analysis, measurements are spanned across time, which gives us a

Techniques based on Artificial
Intelligence and Machine Learning
•Artificial Neural Networks: a Neural network is a biologically-inspired programming
paradigm that presents a brain metaphor for processing information. An Artificial Neural
Network is a system that changes its structure based on information that flows through the
network. ANN can accept noisy data and are highly accurate. They can be considered highly
dependable in business classification and forecasting applications.
•Decision Trees: As the name stands, it is a tree-shaped model that represents a
classification or regression models. It divides a data set in smaller subsets simultaneously
developing into a related decision tree.
•Evolutionary Programming: This technique combines the different types of data analysis
using evolutionary algorithms. It is a domain-independent technique, which can explore
ample search space and manages attribute interaction very efficiently.
•Fuzzy Logic: It is a data analysis technique based on probability which helps in handling the
uncertainties in data mining techniques.

Techniques based on Visualization and
Graphs
•Column Chart, Bar Chart: Both these charts are used to present numerical differences between
categories. The column chart takes to the height of the columns to reflect the differences. Axes
interchange in the case of the bar chart.
•Line Chart: This chart is used to represent the change of data over a continuous interval of time.
•Area Chart: This concept is based on the line chart. It additionally fills the area between the polyline and
the axis with color, thus representing better trend information.
•Pie Chart: It is used to represent the proportion of different classifications. It is only suitable for only one
series of data. However, it can be made multi-layered to represent the proportion of data in different
categories.
•Funnel Chart: This chart represents the proportion of each stage and reflects the size of each module. It
helps in comparing rankings.
•Word Cloud Chart: It is a visual representation of text data. It requires a large amount of data, and the
degree of discrimination needs to be high for users to perceive the most prominent one. It is not a very
accurate analytical technique.
•Gantt Chart: It shows the actual timing and the progress of activity in comparison to the requirements.

Graphs
•Radar Chart: It is used to compare multiple quantized charts. It represents which variables in the
data have higher values and which have lower values. A radar chart is used for comparing
classification and series along with proportional representation.
•Scatter Plot: It shows the distribution of variables in the form of points over a rectangular coordinate
system. The distribution in the data points can reveal the correlation between the variables.
•Bubble Chart: It is a variation of the scatter plot. Here, in addition to the x and y coordinates, the
area of the bubble represents the 3rd value.
•Gauge: It is a kind of materialized chart. Here the scale represents the metric, and the pointer
represents the dimension. It is a suitable technique to represent interval comparisons.
•Frame Diagram: It is a visual representation of a hierarchy in the form of an inverted tree structure.
•Rectangular Tree Diagram: This technique is used to represent hierarchical relationships but at the
same level. It makes efficient use of space and represents the proportion represented by each
rectangular area.

Graphs
•Map
• Regional Map: It uses color to represent value distribution over a map partition.
• Point Map: It represents the geographical distribution of data in the form of points on a
geographical background. When the points are the same in size, it becomes meaningless for
single data, but if the points are as a bubble, then it additionally represents the size of the data
in each region.
• Flow Map: It represents the relationship between an inflow area and an outflow area. It
represents a line connecting the geometric centers of gravity of the spatial elements. The use
of dynamic flow lines helps reduce visual clutter.
• Heat Map: This represents the weight of each point in a geographic area. The color here
represents the density.

DATA ANALYSIS TOOLS
There are several data analysis tools available in
the market, each with its own set of functions. The
selection of tools should always be based on the type
of analysis performed, and the type of data worked.
Here is a list of a few compelling tools for Data
Analysis.
31

1. Excel - It has a variety of compelling features, and with additional plugins installed, it can handle a massive
amount of data. So, if you have data that does not come near the significant data margin, then Excel can be a
very versatile tool for data analysis.
2. Tableau - It falls under the BI Tool category, made for the sole purpose of data analysis. The essence of Tableau
is the Pivot Table and Pivot Chart and works towards representing data in the most user-friendly way. It additionally
has a data cleaning feature along with brilliant analytical functions. E.g. Udemy's online course Hands-On Tableau
Training for Data Science can be a great asset for you.
3. Power BI - It initially started as a plugin for Excel, but later on, detached from it to develop in one of the most data
analytics tools. It comes in three versions: Free, Pro, and Premium. Its PowerPivot and DAX language can
implement sophisticated advanced analytics similar to writing Excel formulas.
4. Fine Report - Fine Report comes with a straightforward drag and drops operation, which helps to design various
styles of reports and build a data decision analysis system. It can directly connect to all kinds of databases, and its
format is similar to that of Excel. Additionally, it also provides a variety of dashboard templates and several self-
developed visual plug-in libraries.
5. R & Python - These are programming languages which are very powerful and flexible. R is best at statistical
analysis, such as normal distribution, cluster classification algorithms, and regression analysis. It also performs
individual predictive analysis like customer behavior, his spend, items preferred by him based on his browsing
history, and more. It also involves concepts of machine learning and artificial intelligence.
6. SAS - It is a programming language for data analytics and data manipulation, which can easily access data from
any source. SAS has introduced a broad set of customer profiling products for web, social media, and marketing

THANKYOU FORYOUR
KIND COOPERATION
Learning makes you knowledgeable and brings satisfaction, and
satisfaction escalations your confidence levels.
– Dr. M Yusuf Alam
33

L 13 (17-05-21) data collection methodology

More Related Content

What's hot (19)

Similar to L 13 (17-05-21) data collection methodology (20)

Recently uploaded (20)

L 13 (17-05-21) data collection methodology