Stock Analyzer Hadoop MapReduce Implementation

Hadoop Project

Stock Analyzer
(Mapreduce and Hive Implementation)
Presented by
Punit Kishore(A13011)
Debayan Datta(A13006)
Sunil Kumar P(A13020)
Maruthi Nataraj K(A13009)
Ashish Ranjan(A13004)
Praxis Business School

AGENDA
 Understanding of the problem
 Technical Architecture
 Basic Structure
 Pseudo Code
 Final Result
 Business Implications

Electronics Template

UNDERSTANDING OF THE PROBLEM
 Objective : To find the adjusted closing price for each

day that a stock not reported a dividend.

 Data Sources :
 NYSE daily prices dataset with the below schema
exchange

stock_symbol

date

stock_price
_open

stock_
price_high

stock_price
_low

stock_price
_close

stock_volume

stock_pric
e_adj_close

 NYSE dividends dataset with the below schema
exchange

stock_symbol

date

dividends

 Isolation of dividend data from total data will give better
picture of the company because sometimes firms avoid
cutting dividends even when earnings drop.
Framework– Mapreduce/Hive

TECHNICAL ARCHITECTURE

Eclipse Indigo 3.7.2
Hadoop 1.2.1 plugin


WinSCP


Putty




Unix Environment /Amazon AWS EC2 Praxis Hadoop Cluster



Sample data - NYSE_daily_prices_AT.csv (Testing is done on sample data only due to
load and time constraints).



Sample data - NSE_daily_prices_BT.csv


Sample data - dividendstest.csv


BASIC STRUCTURE
Input Key Value Pair <Memory Pointer,NYSE,AIT,
12-11-2009,X,X,X,X,X,20.69>

Intermediary Key Value Pair<AIT12-11-2009,1~20.69~0>
<AIT12-11-2009,1~Null~1>

Output/Result Key Value Pair
AIT
12-11-2009
20.69


PSEUDO CODE
import java and hadoop packages

Mapper
Mapper

public static class StockAnalysisMapper extends MapReduceBase implements
Mapper<LongWritable, Text, Text, Text>
{
// declaration of Mapkey and Mapvalue
@Override
public void map(LongWritable key, Text value,OutputCollector<Text, Text> output,
Reporter reporter) throws IOException
{
// declaration of private variables
// switch case to parse the input lines and store the data
// check for null values in the key
// check the header and send the key value to output collector
}

}


PSEUDO CODE
public static class StockAnalysisReducer extends MapReduceBase
implements Reducer<Text, Text, Text, Text>

Reducer
Reducer

{
//Declaration of required private variables
@Override
public void reduce(Text key, Iterator<Text> values,OutputCollector<Text, Text> output, Reporter
reporter) throws IOException
{
//Declaration of sum and flag variables
while (values.hasNext())
{
// Parse the inputs which are count,stock adjusted closing price and check
// Store them as required after parsing
//check for null values of stock adjusted closing price
}
}
}

//Increment the sum
// write to output if sum is 1


PSEUDO CODE
public static void main(String [] arguments) throws Exception
{
JobConf conf = new JobConf(StockAnalyzer.class);
conf.setJobName("Stock Analysis");
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(Text.class);
conf.setMapperClass(StockAnalysisMapper.class);
conf.setReducerClass(StockAnalysisReducer.class);
Path MapperInputPath = new Path(arguments[0]);
Path OutputPath = new Path(arguments[1]);
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
FileInputFormat.setInputPaths(conf, MapperInputPath);
FileOutputFormat.setOutputPath(conf, OutputPath);
JobClient.runJob(conf);
}


Driver
Driver

FINAL RESULT
• NYSE Daily A
– 14 inclusive of
1 header
• NYSE Daily B
– 39 inclusive of
1 header
• Dividends file
– 22 inclusive of
1 header
Total – 75


FINAL RESULT
• Total – 75
• Matching
records – 7
• Headers – 3
• Dividend
records – 21
• Final Output
– 44 records


FINAL RESULT


HIVE
FINAL RESULT HIVE


BUSINESS IMPLICATIONS
 The daily close stock prices are adjusted for dividend distributions/stock
splits because they are a part of total return and affect the historical volatility
estimates .
 The primary use for the adjusted closing price is as a means to develop an
accurate track record of a stock's performance. The comparison of a stock's
historical adjusted closing price to its current price shows the true rate of
return.
 Graphing the volatility history of the target firm simultaneously with that of its
competitors and Market Index can provide unique insights into risk and
comparative advantages(frequency distribution of returns can also be used).
 Historic stock price volatility might have implications to business valuators.


Stock Analyzer Hadoop MapReduce Implementation

More Related Content

Viewers also liked (20)

Similar to Stock Analyzer Hadoop MapReduce Implementation (20)

More from Maruthi Nataraj K (15)

Recently uploaded (20)

Stock Analyzer Hadoop MapReduce Implementation