Statim, time series interface for Perl.
0 likes537 views
This document introduces Statim, a time series interface for Perl. It provides a consistent interface for working with time series data across different storage backends and protocols. Statim allows adding, getting, and setting time series data using simple commands. It includes conventions for data collection, periods, timestamps, and aggregation. The architecture uses a storage engine like Redis or files with a TCP or REST interface. Statim aims to provide real-time statistics, portability, and easy integration.
![REST
• You can extend to any protocol.
• statim-rest : /method/[args]](https://image.slidesharecdn.com/statim-yapc2011-ptbr-111107110000-phpapp01/85/Statim-time-series-interface-for-Perl-34-320.jpg)
Statim, time series interface for Perl.
- 1. Statim time series interface for Perl Thiago Rondon YAPC::Brasil 2011
- 2. “Science progresses throught observation” -- Isaac Newton
- 3. “The greatest value of a picture is that is forces us to notice what we never expected to see” -- John Tukey
- 4. Where ? • Astronomy • Apps • Medicine • Operating System • Stock Market • Networks • Meteorology • Robotics • Biometrics • Events • Geology • ....
- 5. Wait! let’s see Chart::Cliker examples.
- 6. Line
- 7. StackedLine
- 8. Bar
- 9. StackedBar
- 10. Area
- 11. StackedArea
- 12. Bubble
- 13. CandleStick
- 14. Pie
- 15. PolarArea
- 16. What ? • time series is a sequence of data points, measured typically at successive times spaced at uniform time intervals. • interface define a set of methods and conventions that provides a consistent time series interface.
- 17. Why ? • real time statistics. • simple way to deploy. • support openstandards • distributed input • provides default methods, functions, tools and etc.
- 18. Motivation • independent • many writers • abstract layer on top of other databases to map and reduce. • Portability • easy integration
- 19. But, we need more. • 3D,Visualizing data. • Euclidean distance • Find similar patterns • Anomaly detection • Rule discovery • Frequent pattern discovery
- 20. Architecture TCP TCP Server Client Perl storage schema engine Client REST REST Server Client
- 21. Conventions • data collection • period (time units) • step and timestamp. • type of aggregation • fields
- 22. Engine • AnyEvent • TCP • REST
- 23. Storage • Key Interface • Redis • Memcached • Table Interface • DBI • File • RRD Interface • RRD
- 24. Schema { "collection" : { "period" : "84600", "aggregate" : "sum", "fields" : { "foo" : "count", "bar" : "enum", "jaz" : "enum" } } }
- 25. Command • add, get and set • period, step • version, quit
- 26. Commands • add collection foo:jaz bar:1 • OK 1 • add collection foo:jaz bar:1 • OK 2 • add collection foo:wow bar:10 • OK 10 • get collection foo:jaz bar • OK 2 • get collection bar • OK 12 • set collection foo:jaz bar:2 • OK 2
- 27. Commands with “ts” • You can pass timestamp with “ts”. • add collection foo:bar ts:123467890 bar:1 • You can get a new series with “ts” • get collection foo:bar ts:1234567880-1234567890 bar
- 28. Commands with step • You can pass step with “step”. • add collection foo:bar step:-10 bar:1 • You can get a new series with “step” • get collection foo:bar step:-10..-20 bar
- 29. step vs timestamp • You can get the step in server by: • step 1234567890 • Or make the calc in the client.
- 30. timestamp vs step • Timestamp make easy life to insert data • Step make easy life to get data
- 31. New time series by periods. • get collection bar:jaz ts:$ts1-$ts2 foo:max • get collection bar:* ts:$ts1-$ts2 foo:max • get collection bar:jaz ts:$ts1-$ts3 foo:min • get collection ts:$ts1-$ts3 foo:sum • get collection ts:$ts1-$ts3 foo:avg • get collection ts:$ts1-$ts3 foo:distinct
- 32. New time series by periods. • get collection ts:$ts1-$ts2 foo:trend(5) • get collection ts:$ts1-$ts2 foo:anomaly(100) • get collection ts:$ts1-$ts3 foo:frequency(200)
- 33. Chart ! • You can use any lib to do that. • Server-side, eg. Chart::Cliker. • Client-site, eg. html5 and javascript libs.
- 34. REST • You can extend to any protocol. • statim-rest : /method/[args]