Insights Without Tradeoffs Using Structured Streaming keynote by Michael Armbrust
Download as PPTX, PDF5 likes2,286 views
The document discusses insights from Michael Armbrust's presentation at Spark Summit East 2017 on structured streaming, emphasizing the importance of parallelism, developer productivity, and efficiency in data processing. It highlights structured streaming's ability to handle large datasets while providing up-to-date analytical answers and includes several production use cases. Additional resources and office hours information are shared for further engagement with the engineering community.
Related topics:1 of 8
Downloaded 115 times
More Related Content
What's hot (20)
Ad
Viewers also liked (9)
Ad
Similar to Insights Without Tradeoffs Using Structured Streaming keynote by Michael Armbrust (20)
More from Spark Summit (20)
Insights Without Tradeoffs Using Structured Streaming keynote by Michael Armbrust
- 1. Insights Without Tradeoffs Using Structured Streaming Michael Armbrust - @michaelarmbrust Spark Summit East 2017
- 2. Parallelism Split up the problem to harness many machines for computation Complexity Handle cross-machine communication and failures
- 3. Developer Productivity Quickly and concisely express common computations Efficiency Hand-tune code to minimize overheads and process the most data per cycle SQL
- 4. Throughput Process large historical repositories quickly Latency Up-to-date answers as new data arrives Structured Streaming
- 6. Streaming at Collect logs and metrics from a variety of sources to ensure the security, availability and performance of our cloud platform.
- 7. Engineer Office Hours MY HOURS • Spark Core • R • Data Science • ML • GraphFrames, Deep Learning • Databricks • Spark SQL • Structured Streaming • Spark SQL • Structured Streaming • Databricks TODAY 4:30 – 5:15 OTHER ENGINEERS TODAY 1:45 – 5:15 THURS 10:30 – 2:30 @ Databricks Booth
- 8. Thank you! All code available at databricks.com/blog and @michaelarmbrust