Insight Recent Demo

This document discusses developing a crowd detector system using location data from mobile phones. It proposes collecting location data from phones, performing a random walk on the data, and using k-means clustering with K=500 clusters to identify crowded areas of San Francisco. The goals are to tune parameters like batch processing time and update intervals to have a stable, low-latency system. Challenges addressed include choosing K, limiting input/processing rates, and understanding Spark Streaming complexity. The proposed solution engineers location data from Yelp to simulate real phone data.