time series modeling-Decision Science Central Oct 26
Download as DOCX, PDF1 like273 views
This document summarizes time series forecasting of receival, outturn, and inventory volumes for grain storage sites in Australia. Receival, outturn, and inventory data from multiple sites over 5 years was aggregated and decomposed to understand trends, seasonality, and random patterns. Two forecasting models, ARIMA and Holt-Winters exponential smoothing, were used to generate forecasts, and the more accurate model was selected. The results showed receivals and outturns increase seasonally during harvest periods and have decreased in recent years, while inventory follows receival and outturn patterns. Time series forecasting can help optimize site operations and resource planning.
![1
Time Series Forecasting in Grain Storage
Authors: Vinay Mehendiratta, PhD, Director of Research and Analytics, Eka Software
Sishir Kumar Pagada, Senior Software Engineer, Eka Software
Created as part of the Data Science for IoT practitioners course – starting Nov 10 2015
The pdf version of this paper may be downloaded HERE
Abstract
Grain storage operators are always trying to minimize the cost of their supply chain. Understanding
relationship between receival, outturn, within storage site and between storage site movements can
provide usinsightsthatcan be useful inplanningforthe nextharvestreason, estimatingthe throughput
capacityof the system,relationshipbetweenthroughoutandinventory.Thisarticle exploresthepotential
of scanner data in advance analytics. Combinationof these two fields has the potential to be useful for
grain storage business. The study describes Grain storage scenarios in the Australian context.
----------------------------------------------------------------------------------------------------------------------------------------
Introduction
There is sufficient grain storage capacity across most of Australia to cater for a range of seasonal
outcomes. There is about 55 Million Metric Ton (MMT) of bulk handling storage capacity at 623 sites
across Australia. Combined with an estimated 15 MMT of on-farm storage capacity, Australia has the
capacityto store the equivalentof twoyears’average grainproduction.Asaresult,grainstorage feesare
kept relatively low and are falling in real terms [1].
As grainfacilities andportterminals investigate methodstoincrease throughputandgrow revenues,itis
becoming important to maximize resource utilization and understand throughput. A typical busy site
could receive as high as 400 trucks a day during harvest season.
Data collected at individual storage facilities provides visibility of stock across all storage sites. An RFID
Scanneratthe storage site entrance registersinformationontruckarrival time andismappedto tonnage,
grower,grade,commoditytype, andquality. Thisactivityisknownas‘receival’ingrainstorage operations.
RFID Scanner at the storage site exit registers information on truck departure time and is mapped to
tonnage,grower,grade,commoditytype, andquality.Thisactivityisknownas ‘outturn’in grain storage
operations. Storage facilities alsohave access to grain movement information from one site to another
site (‘between-site-movements’) and within a site (‘within-site-movements’).
The questionwe addressis: Could ‘Internetof Things’and ‘Predictive Analytics’help understand receival
and outturn behavior at grain storage sites? Could we make better usage of this data?
Predictive Analytics Platform
The High level flow of dataata grainstorage site is described below.](https://image.slidesharecdn.com/e7ba62ad-ad06-4fe3-ad6a-029f7d5decdc-151026115948-lva1-app6892/85/time-series-modeling-Decision-Science-Central-Oct-26-1-320.jpg)
![2
Figure 1: Internet of Things specific Architecture
We consolidated receivals,outturns informationreceivedby sensors at month level for the last 5 years.
Understanding of those patterns can be used in planning for the site operations at harvest season and
other seasons. The rest of the document is arranged as shown in Figure 2.
Figure 2: Document and Methodology Flow
Input Processing: Receival and outturn is a frequent activity at some sites but not all. Unavailability of
data at regularinterval at those sites made itimpossible toanalyse dailyoperations andatsite level.We
aggregate data across sites, commodities, and grades at month level to overcome this issue of missing
values. We determine inventory snapshot using receival and outturn data.
Inventory attheend of month=Initial Inventory atthebeginning of month +ReceivalQuantity during this
month – Outturn Quantity during this month.
Time Series Modeling – Identify patterns in historical data
Time seriesissequenceof observations(generallyquantitative observations)takenatequallyspacedtime
intervals. An inherent feature of time series is that adjacent observations are dependent/related. Time
seriesanalysisismainlyconcernedwithanalysisof the dependenceamongadjacentobservations. Sensor
data capturedatvariousdiscrete time periodsis atime seriesand suitablefortimeseriesmodeling. There
are variouspackagesavailableinRtodecompose atime series. A tutorial ontimeseriesinRcanbe found
at [8]. Time series decomposition (using software R) breaks observations into three components.
Decomposemethodof Rwas usedtodetermine the trend,seasonality,andrandompatterninthisstudy.
We provide the code snippet below:
tsObject<-ts(QuantitySeries,start=c(fromYear,fromMonth),frequency=frequencyInt)
decomposedTs <- decompose (tsObject, type="additive", filter=NULL)
fromMonth and fromYear is the earliest observation of receival and outturn data. frequencyInt is the
number of observations in a year in this dataset. More information on configuring these parameters is
available at [7]. QuantitySeries is the name of the dataset brought into R by reading a csv file.
Plot function is used to generate all the graphs used in this report.
Scanner Data at Site
Time Series Database
(MongoDB)
PredictiveAnalytics
Engine(R)
Visualization Tool(
Tableau, HighCharts)
Input
Gathering
Input
Processing
Pattern
Recognition
Forecasting
Output
Analysis](https://image.slidesharecdn.com/e7ba62ad-ad06-4fe3-ad6a-029f7d5decdc-151026115948-lva1-app6892/85/time-series-modeling-Decision-Science-Central-Oct-26-2-320.jpg)
![3
plot.ts(tsObject)
Trend: This component looks for the trend in observed data whether observations have increasing or
decreasing or constant trend over time. The function determines the trend component using a moving
average.
trendComp <- decomposedTs$trend
• Seasonality: This component looksfor the cyclic pattern in observeddata over time. It is computed
by averaging, for each time unit, over all periods.
seasonalComp <- decomposedTs$seasonal
• Random: This componentlooksforobservations thatare not explainedbytrendand seasonality. It
isachievedby removingtrendandseasonal figure(recycledasneeded) fromthe originaltime series.
randomComp <- decomposedTs$random
Receival Volumes at sites: Historical receival data is plotted in Figure 3. We used this data and fed to
decompose [2] package of software R to understandthe patterns.One can easilyinterpretfrom Figure 4
that receivalsdoincrease dramaticallyduringharvestseasoneveryyearasshown inseasonalitygraphof
Figure 4. Receivals are increasing every year until 2013. Moving average (plotted in trends section) in
receivals have come down during 2013-2014 as shown in Figure 4.
Figure 3 : Receival Time Series Figure 4: Patterns for Receival Time Series
Outturn Volumesat sites:Historical outturndata isplottedinFigure 5. We usedthisdata and fedto the
decompose package of softwareRtounderstandthepatterns.Onecaneasilyinterpretfrom trendsection
of Figure 6 that outturns do increase steadily everyyearuntil 2013. Outturnsalso do show a verystrong
‘seasonal’ behaviour as shown in ‘seasonal’ section of Figure 6.](https://image.slidesharecdn.com/e7ba62ad-ad06-4fe3-ad6a-029f7d5decdc-151026115948-lva1-app6892/85/time-series-modeling-Decision-Science-Central-Oct-26-3-320.jpg)
![5
There are many packages available within R that can be used to forecast receival, outturn with varying
degree of accuracy.One hasto testandcheck resultsto findthe package thatgivesbetterresults.We did
use two such algorithms: ARIMA and Holt-Winters.
Arima:We use auto.arima() function available in forecastpackage of R[9]. Thisfunctionautomatically
findsbestfittingArima[3] model tothe data. Residuals are the actual valuesminusthe fittedvalues.
tsObject <- ts(QuantitySeries, start=c(fromYear,fromMonth), frequency=frequencyInt)
arimaFit <- auto.arima(tsObject)
arimaResiduals <- residuals(arimaFit)
Holt-Winters: We use Holt Winters Exponential Smoothening model [4] with trend and seasonality.
trend <- isTrend
seasonality <- isSeasonality
hwFit <- HoltWinters(tsObject, beta=TRUE, gamma=TRUE)
beta= TRUE settingwouldinclude the trendcomponent. gammaisthe parameterusedforthe seasonal
component. If set to TRUE, a -seasonal model is fitted. We calculate the sum of squared errors (SSE) for
the in-sample forecasterrors,thatis,the forecasterrorsfor the time periodcoveredbyouroriginal time
series.
arimaErros <- sum(arimaResiduals^2)
hwErrors <- hwFit$SSE
We compare the model accuracy of Arima and Holt-Wintersforgivendata.The model thatprovidesthe
forecast with the minimum value of SSE is chosen to make forecast for the future periods. Code is
provided below:
if (arimaErros<= hwErrors){
print ("Using Arima")forecastedValues<- forecast(arimaFit,h=10)
}
else if (arimaErros> hwErrors){
Print("Using Holt-Winters")
forecastedValues<- forecast.HoltWinters(hwFit,h=10)
}
forecastedValue<- as.data.frame(forecastedValues)](https://image.slidesharecdn.com/e7ba62ad-ad06-4fe3-ad6a-029f7d5decdc-151026115948-lva1-app6892/85/time-series-modeling-Decision-Science-Central-Oct-26-5-320.jpg)
time series modeling-Decision Science Central Oct 26
- 1. 1 Time Series Forecasting in Grain Storage Authors: Vinay Mehendiratta, PhD, Director of Research and Analytics, Eka Software Sishir Kumar Pagada, Senior Software Engineer, Eka Software Created as part of the Data Science for IoT practitioners course – starting Nov 10 2015 The pdf version of this paper may be downloaded HERE Abstract Grain storage operators are always trying to minimize the cost of their supply chain. Understanding relationship between receival, outturn, within storage site and between storage site movements can provide usinsightsthatcan be useful inplanningforthe nextharvestreason, estimatingthe throughput capacityof the system,relationshipbetweenthroughoutandinventory.Thisarticle exploresthepotential of scanner data in advance analytics. Combinationof these two fields has the potential to be useful for grain storage business. The study describes Grain storage scenarios in the Australian context. ---------------------------------------------------------------------------------------------------------------------------------------- Introduction There is sufficient grain storage capacity across most of Australia to cater for a range of seasonal outcomes. There is about 55 Million Metric Ton (MMT) of bulk handling storage capacity at 623 sites across Australia. Combined with an estimated 15 MMT of on-farm storage capacity, Australia has the capacityto store the equivalentof twoyears’average grainproduction.Asaresult,grainstorage feesare kept relatively low and are falling in real terms [1]. As grainfacilities andportterminals investigate methodstoincrease throughputandgrow revenues,itis becoming important to maximize resource utilization and understand throughput. A typical busy site could receive as high as 400 trucks a day during harvest season. Data collected at individual storage facilities provides visibility of stock across all storage sites. An RFID Scanneratthe storage site entrance registersinformationontruckarrival time andismappedto tonnage, grower,grade,commoditytype, andquality. Thisactivityisknownas‘receival’ingrainstorage operations. RFID Scanner at the storage site exit registers information on truck departure time and is mapped to tonnage,grower,grade,commoditytype, andquality.Thisactivityisknownas ‘outturn’in grain storage operations. Storage facilities alsohave access to grain movement information from one site to another site (‘between-site-movements’) and within a site (‘within-site-movements’). The questionwe addressis: Could ‘Internetof Things’and ‘Predictive Analytics’help understand receival and outturn behavior at grain storage sites? Could we make better usage of this data? Predictive Analytics Platform The High level flow of dataata grainstorage site is described below.
- 2. 2 Figure 1: Internet of Things specific Architecture We consolidated receivals,outturns informationreceivedby sensors at month level for the last 5 years. Understanding of those patterns can be used in planning for the site operations at harvest season and other seasons. The rest of the document is arranged as shown in Figure 2. Figure 2: Document and Methodology Flow Input Processing: Receival and outturn is a frequent activity at some sites but not all. Unavailability of data at regularinterval at those sites made itimpossible toanalyse dailyoperations andatsite level.We aggregate data across sites, commodities, and grades at month level to overcome this issue of missing values. We determine inventory snapshot using receival and outturn data. Inventory attheend of month=Initial Inventory atthebeginning of month +ReceivalQuantity during this month – Outturn Quantity during this month. Time Series Modeling – Identify patterns in historical data Time seriesissequenceof observations(generallyquantitative observations)takenatequallyspacedtime intervals. An inherent feature of time series is that adjacent observations are dependent/related. Time seriesanalysisismainlyconcernedwithanalysisof the dependenceamongadjacentobservations. Sensor data capturedatvariousdiscrete time periodsis atime seriesand suitablefortimeseriesmodeling. There are variouspackagesavailableinRtodecompose atime series. A tutorial ontimeseriesinRcanbe found at [8]. Time series decomposition (using software R) breaks observations into three components. Decomposemethodof Rwas usedtodetermine the trend,seasonality,andrandompatterninthisstudy. We provide the code snippet below: tsObject<-ts(QuantitySeries,start=c(fromYear,fromMonth),frequency=frequencyInt) decomposedTs <- decompose (tsObject, type="additive", filter=NULL) fromMonth and fromYear is the earliest observation of receival and outturn data. frequencyInt is the number of observations in a year in this dataset. More information on configuring these parameters is available at [7]. QuantitySeries is the name of the dataset brought into R by reading a csv file. Plot function is used to generate all the graphs used in this report. Scanner Data at Site Time Series Database (MongoDB) PredictiveAnalytics Engine(R) Visualization Tool( Tableau, HighCharts) Input Gathering Input Processing Pattern Recognition Forecasting Output Analysis
- 3. 3 plot.ts(tsObject) Trend: This component looks for the trend in observed data whether observations have increasing or decreasing or constant trend over time. The function determines the trend component using a moving average. trendComp <- decomposedTs$trend • Seasonality: This component looksfor the cyclic pattern in observeddata over time. It is computed by averaging, for each time unit, over all periods. seasonalComp <- decomposedTs$seasonal • Random: This componentlooksforobservations thatare not explainedbytrendand seasonality. It isachievedby removingtrendandseasonal figure(recycledasneeded) fromthe originaltime series. randomComp <- decomposedTs$random Receival Volumes at sites: Historical receival data is plotted in Figure 3. We used this data and fed to decompose [2] package of software R to understandthe patterns.One can easilyinterpretfrom Figure 4 that receivalsdoincrease dramaticallyduringharvestseasoneveryyearasshown inseasonalitygraphof Figure 4. Receivals are increasing every year until 2013. Moving average (plotted in trends section) in receivals have come down during 2013-2014 as shown in Figure 4. Figure 3 : Receival Time Series Figure 4: Patterns for Receival Time Series Outturn Volumesat sites:Historical outturndata isplottedinFigure 5. We usedthisdata and fedto the decompose package of softwareRtounderstandthepatterns.Onecaneasilyinterpretfrom trendsection of Figure 6 that outturns do increase steadily everyyearuntil 2013. Outturnsalso do show a verystrong ‘seasonal’ behaviour as shown in ‘seasonal’ section of Figure 6.
- 4. 4 Figure 5: Out-turn Time Series Figure 6: Patterns for Out-turn Time Series Figure 7: Inventory Time Series Figure 8: Patterns for Inventory Time Series Inventory Volumesat sites: Historical inventorydataisplottedinFigure 7. We usedthisdata and fedto the decompose package of software Rtounderstandthe patterns.One caneasilyinterpretfromFigure 6 that inventory volumeincreasesduringharvesttime everyyear.Inventorybuildwashigherwhenharvest season had more receival tonnage. Inventory is the result of receivals and outturns. It does show seasonalitybehavioursimilartoreceival andoutturnactivitypatternas shownin ‘seasonality’sectionof Figure 8. Time Series Forecasting To make forecast for future periods, time series extrapolates the observed dependence relationships among available observations to future periods. Time series forecasts for future periods are based on analysis of dependence relationships, such as trend and seasonality, among available observations. We dividedthe dataintotwo parts – trainingdataand test data. Training data (from Jan, 2010 to Dec, 2013) wasusedtoallowourmodelslearnfromhistory.Testdata (FromJan,2014 toOct, 2014) wasusedtotest the accuracy of model generatedusingtrainingdata.The objective wastoidentifyanypatterninreceivals (client, grower), outturns,and resultant inventory that can be used to understand grower’s operations, market behavior, and bottlenecks in warehouse efficiency.
- 5. 5 There are many packages available within R that can be used to forecast receival, outturn with varying degree of accuracy.One hasto testandcheck resultsto findthe package thatgivesbetterresults.We did use two such algorithms: ARIMA and Holt-Winters. Arima:We use auto.arima() function available in forecastpackage of R[9]. Thisfunctionautomatically findsbestfittingArima[3] model tothe data. Residuals are the actual valuesminusthe fittedvalues. tsObject <- ts(QuantitySeries, start=c(fromYear,fromMonth), frequency=frequencyInt) arimaFit <- auto.arima(tsObject) arimaResiduals <- residuals(arimaFit) Holt-Winters: We use Holt Winters Exponential Smoothening model [4] with trend and seasonality. trend <- isTrend seasonality <- isSeasonality hwFit <- HoltWinters(tsObject, beta=TRUE, gamma=TRUE) beta= TRUE settingwouldinclude the trendcomponent. gammaisthe parameterusedforthe seasonal component. If set to TRUE, a -seasonal model is fitted. We calculate the sum of squared errors (SSE) for the in-sample forecasterrors,thatis,the forecasterrorsfor the time periodcoveredbyouroriginal time series. arimaErros <- sum(arimaResiduals^2) hwErrors <- hwFit$SSE We compare the model accuracy of Arima and Holt-Wintersforgivendata.The model thatprovidesthe forecast with the minimum value of SSE is chosen to make forecast for the future periods. Code is provided below: if (arimaErros<= hwErrors){ print ("Using Arima")forecastedValues<- forecast(arimaFit,h=10) } else if (arimaErros> hwErrors){ Print("Using Holt-Winters") forecastedValues<- forecast.HoltWinters(hwFit,h=10) } forecastedValue<- as.data.frame(forecastedValues)
- 6. 6 • • • Figure 11: Receival, Outturn, and Inventory Forecast Resultsof time seriesforecastingare shownin Figure 11. Itisimportantto considerpointforecastaswell as forecast with confidence intervals. People responsible for site operations have consider next harvest season related news, weather,and crop yield information to decide the confidence interval that should
- 7. 7 be used for various purposes. Forecast with confidence intervals can be used to devise strategies for various scenarios and assess future uncertainty. Relation between scanner data and internal movement data We also consideredmovementswithin-the storage-site fromone binto another.We foundthat volume of movements within-the storage-site was increasing from year 2012 to 2014 (Figure 12, 13). We are aware thatoutturnsincreasedwhilereceivalsand inventorydecreasedfromyear2012 to 2014. Inventory volume had touched the peak during 2012. We also observed that as inventory volume went down, movement volume within-the storage-site increased while volume moved between-storage-sites decreased. That bodes well for bulk storage handlers trying to reduce the cost of supply chain. Figure 12: Movementswithin-the-sitetime seriesFigure13:Patternsformovementwithin-the-site series Figure 12a: Movements among sites Figure 13a: Patterns for movement among sites
- 8. 8 Figure 14: Client Receivals Figure 15: Patterns for Client Receivals GrowerReceivals –Figure15,16 showusthatpeakvolumereachedduringharvest receivalsfromgrowers isdecreasing. Receivalvolumefromclientsisalsodecreasingsteadily.Impactof thiscause isclearlyvisible indecliningvolume of inventoryatsitesfromyear2012 to year2013 and further. Ison-farmstorage and high cost of storage one of the reason for reduced receivals? Figure 16: GrowerReceivals Timeseries Figure 17: Patterns – Grower Receivals Conclusion Analysisof receival andoutturndataStorage data mightbe beneficial foranorganizationtogaininsights into the behavior of storage sites. One could determine the relationship between throughput and inventory,throughputandinternal movementstomeasure the efficiencyof operations. Thisanalysiscan be useful toplan fornextharvestseason.Itmightbe worthwhileperformingthe time seriesmodelingat daily level during harvest seasonfor major/busysites. Our objective in thisarticle has beento promote
- 9. 9 the use of Advance analytics and scanner data (IOT) has the potential to be useful for grain storage business. Created as part of the Data Science for IoT practitioners course – starting Nov 10 2015 The pdf version of this paper may be downloaded HERE References 1. The cost of Australia’s bulk grain export supply chains, http://www.aegic.org.au/media/22950/140130%20Final%20AEGIC%20Supply%20Chain s%20Report.pdf, Australian Export Grains Innovation Center 2. https://cran.r-project.org/ 3. https://en.wikipedia.org/wiki/MongoDB 4. https://stat.ethz.ch/R-manual/R-devel/library/stats/html/decompose.html 5. https://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average 6. https://en.wikipedia.org/wiki/Exponential_smoothing 7. http://127.0.0.1:20382/library/stats/html/ts.html 8. https://a-little-book-of-r-for-time-series.readthedocs.org/en/latest/ 9. https://cran.r-project.org/web/packages/forecast/index.html 10.