Big Data Synopsis
6 likes2,968 views
The document summarizes a research project using big data to help avoid weather-related flight delays. It discusses how researchers gathered over 10 years of hourly weather and flight data and are using advanced analytics to identify patterns that could help airlines better manage delays. The goal is to allow airlines to anticipate delays before they happen by predicting how weather in one location may impact flights elsewhere. This could help airlines proactively adjust schedules and resources to minimize disruptions. The researchers believe this analysis of massive data sets could significantly improve the travel experience for passengers and airlines.
![PushkarZagade
References
[1] Manyika JChui MBrown Bet al Big data:
The next frontier for innovation,competition,
and productivity McKinsey Global Institute
[2] Research Trends Issue 30 September 2012
[3] The Fourth Paradigm Data-Intensive Scientific Discovery
edited by Tony Hey, Stewart Tansley,andKrist in Tolle pp 12-15
[4] Big Data for Development: From Information- to
Knowledge Societies Martin Hilbert (Dr. PhD.) Hilbert,
Big Data for Dev.; pre-published version, Jan. 2013
[5] S. Consulting. The New York City
Taxicab Fact Book.[Online]. Available:
http://www.schallerconsult.com/taxi/taxifb.pdf, accessed 2006.
[6] Taxi of Tomorrow Survey, New York City Taxi and
Limousine Commission, New York, NY, USA, 2011.
[7] W. Wu, W. S. Ng, S. Krishnaswamy, and A. Sinha, “To
taxi or not to taxi? Enablingpersonalised and real-time trans-
portation deci- sions for mobile users,” in Proc. IEEE 13th
Int. Conf. Mobile Data Manage. (MDM), Jul. 2012, pp. 320 323.
[8] Y. Ge, H. Xiong, A. Tuzhilin, K. Xiao, M. Gruteser,
and M. Paz-zani, “An energy effcient mobile recom-
mender system,” in Proc. 16th ACM SIGKDD Int. Conf.
Knowl. Discovery Data Mining (KDD), 2010,pp. 899 908.
[9] H. Yang, C. S. Fung, K. I. Wong, and S. C.
Wong, “Nonlinear pricing of taxi services,”Transp. Res.
A, Policy Pract., vol. 44, no. 5, pp. 337v348, 2010.
[10] K. Yamamoto, K. Uesugi, and T. Watanabe, “Adap-
tive routing of cruising taxis by mutual exchange of path-
ways,” in Knowledge-Based Intelligent Information and En-
gineering Systems. Berlin, Germany: Springer-Verlag, 2010.
6](https://image.slidesharecdn.com/watermarkytqzlur-150912071958-lva1-app6892/85/Big-Data-Synopsis-6-320.jpg)
Big Data Synopsis
- 1. PushkarZagade A Seminar Synopsis On “BIG DATA TO AVOID WEATHER RELATED FLIGHT DELAYS ” Submitted to JSPM’s JAYAWANTRAO SAWANT COLLEGE OF ENGINEERING PUNE 28 SavitribaiPhule Pune University, Pune In Partial Fulfillment of T.E. Computer Engineering T.E. Semester II Submitted by Mr. Pushkar Girish Zagade Under the Guidance of Prof. Neelima Satpute Department Of Computer Engineering JSPM’s JAYAWANTRAO SAWANT COLLEGE OF ENGINEERING Academic Year 2014-15 1
- 2. PushkarZagade Abstract This paper identifies key aviation data sets for operational analytics, presents a methodology for application of big-data analysis methods to oper- ational problems, and offers examples of analytical solutions using an inte- grated aviation data warehouse. Big-data analysis methods have revolution- ized how both government and commercial researchers can analyze mas- sive aviation databases that were previously too cumbersome, inconsistent or irregular to drive high-quality output. Traditional data-mining methods are effective on uniform data sets such as flight tracking data or weather. Integrating heterogeneous data sets introduces complexity in data standard- ization, normalization, and scalability. The variability of underlying data warehouse can be leveraged using virtualized cloud infrastructure for scala- bility to identify trends and create actionable information. The applications for big-data analysis in airspace system performance and safety optimiza- tion have high potential because of the availability and diversity of airspace related data. Analytical applications to quantitatively review airspace per- formance, operational efficiency and aviation safety require a broad data set. Individual information sets such as radar tracking data or weather reports provide slices of relevant data, but do not provide the required context, per- spective and detail on their own to create actionable knowledge. These data sets are published by diverse sources and do not have the standardization, uniformity or defect controls required for simple integration and analysis. At a minimum, aviation big-data research requires the fusion of airline, air- craft, flight, radar, crew, and weather data in a uniform taxonomy, organized so that queries can be automated by flight, by fleet, or across the airspace system. 2
- 3. PushkarZagade 1 Introduction 1.1 Research In The University Of Michigan The students from the University of Michigan have started a new research which helps in understanding the weather of a particular place. They have taken data of the weather of the past 10 years. The analysis of this data helps in understanding the patterns in the weather. This is a very creative and new process. It could lead to understanding similarities in the weather in the past years. It could be of help in predicting the weather in the future. This can be very helpful for flights. With the help of this data, the flights can be cautious of bad weather in advance. So it will be usefull. 1.1.1 More About Big Data for Predictive analytics The data used in this research is available publicly. Since it the hourly data of the last ten years, the data is huge in quantity. Hence, it has to be managed cleverly and all of it must be taken into consideration. The study of the weather is carried out keeping the flights and their journeys in mind. This enables the researchers to understand the effect of weather on a particular journey. This is a very unique study. It will help in predicting the delays or preventing them in certain cases. 3
- 4. PushkarZagade 2 Brief Working They’ve gathered more than 10 years of hour-by-hour weather observations and domestic fight data, and are using advanced data analytics to spot patterns and help airlines manage more efficiently. While the project uses public data that has been available for years, its size and scope make it unique and it will help in this project. They gather this data in one place and apply this level of computing power to it. ”That enables us to do a very sophisticated analysis of how weather and flight delays are connected and go far beyond individual airports. We can see how the weather in Atlanta may affect flight operations in Detroit later in the day, or exactly how a delayed plane on the East Coast ripples through the system to California.”The chief goal is to enable airlines to anticipate and deal with de- lays before they happen.Today, most airlines compensate for delays by adding slack to the system. They may schedule in extra flight time during the win- ter or keep additional staff members on call. But generally, they don’t look at large-scale weather patterns when they’re building initial schedules. And their ability to shuffle resources to deal with emerging weather patterns is limited. Airlines generally deal with weather delays after they happen.”They want to give them the ability to be a bit more proactive. When they’re able to predict delays further in advance, they’ll be able to do a much better job of communicat- ing with passengers and optimizing resources.This project may be used to build computer-modeling software that could predict the outcome of an infinite number of hypothetical flight and weather scenarios, helping airlines spot likely weather delays in advance. That knowledge could enable airlines to adjust their schedules to account for weather patterns. It may also lead to new options for passengers. For example, air- lines could look several steps ahead to predict a future flight delay, then offer pas- sengers a pre-emptive rebooking to avoid it.Imagine youre scheduled to fly out of Detroit four hours from now and theres a storm in Atlanta.. The airline could use this data to determine that the storm in Atlanta is likely to delay your plane. They could then contact you and offer you a seat on an alternate flight. You save time, and the airline doesnt have to accommodate you on a later flight after the delay happens. Airlines could also use the advance warning to allocate their own resources more efficiently, shuffling ground crews, flight crews and other assets to minimize 4
- 5. PushkarZagade disruption.The project draws on resources from a wide swath of disciplines in- cluding engineering, computer science and others. He says that breadth of knowl- edge has brought a fresh perspective that could hold far-reaching benefits for both airlines and passengers.Our background in industrial engineering and computer science enables us to put existing data together in new ways and ask a whole new set of questions. For me, thats what has been really exciting about this research. Some of the first changes that passengers see are likely to be simple ones, like tweaks to flight times and more proactive communication. 3 Application • We can use big data to collect large set of informations and use it for data ware house purpose. • Using this technique we can predict the weather information of next 2,3 month in advance . Using this we can avoid the delays as well as cancellation of flight due to bad weather. • Big data can also use for data mining purpose. • By using big data we can analyse the large set of information. 4 Conclusion I think these new analytics will enable passengers as well as airlines to better manage the whole travel process. If airlines can offer more options and passengers can educate themselves on how to use those options, well see fewer delays and a less stressful travel experience in the years to come. 5
- 6. PushkarZagade References [1] Manyika JChui MBrown Bet al Big data: The next frontier for innovation,competition, and productivity McKinsey Global Institute [2] Research Trends Issue 30 September 2012 [3] The Fourth Paradigm Data-Intensive Scientific Discovery edited by Tony Hey, Stewart Tansley,andKrist in Tolle pp 12-15 [4] Big Data for Development: From Information- to Knowledge Societies Martin Hilbert (Dr. PhD.) Hilbert, Big Data for Dev.; pre-published version, Jan. 2013 [5] S. Consulting. The New York City Taxicab Fact Book.[Online]. Available: http://www.schallerconsult.com/taxi/taxifb.pdf, accessed 2006. [6] Taxi of Tomorrow Survey, New York City Taxi and Limousine Commission, New York, NY, USA, 2011. [7] W. Wu, W. S. Ng, S. Krishnaswamy, and A. Sinha, “To taxi or not to taxi? Enablingpersonalised and real-time trans- portation deci- sions for mobile users,” in Proc. IEEE 13th Int. Conf. Mobile Data Manage. (MDM), Jul. 2012, pp. 320 323. [8] Y. Ge, H. Xiong, A. Tuzhilin, K. Xiao, M. Gruteser, and M. Paz-zani, “An energy effcient mobile recom- mender system,” in Proc. 16th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD), 2010,pp. 899 908. [9] H. Yang, C. S. Fung, K. I. Wong, and S. C. Wong, “Nonlinear pricing of taxi services,”Transp. Res. A, Policy Pract., vol. 44, no. 5, pp. 337v348, 2010. [10] K. Yamamoto, K. Uesugi, and T. Watanabe, “Adap- tive routing of cruising taxis by mutual exchange of path- ways,” in Knowledge-Based Intelligent Information and En- gineering Systems. Berlin, Germany: Springer-Verlag, 2010. 6