How Edmodo Uses Splunk For Real-Time Tag-Based Reporting of AWS Billing and Usage
- 1. How Edmodo Uses Splunk App for AWS Jack Murgia NetOps Architect jack@edmodo.com
- 2. Edmodo is the largest social learning network in the world 28,000,000+ TOTAL USERS AROUND THE WORLD
- 3. The Edmodo user base has grown REALLY FAST and so has our AWS usage. • New Features 20M+ users 18,000,000 ▸ Require new backend clusters • More Employees 14,000,000 ▸ Launch more development servers 20M+ Registered Users • New Investors ▸ Want assurances we spend money wisely 10,000,000 Investors 6,000,000 4.8M users 1.1M users 2,000,000 225K users Dec 09 Dec 10 Dec 11 Dec 12
- 4. 100’s of production, development, reporting and operations instances are managed on Amazon Web Services Elastic Compute Cloud. 20 employees across 5 teams can launch EC2 instances.
- 5. We tried lots of slow, pricey, hosted apps. None had what we REALLY Needed …
- 6. Real Time RI vs On-Demand Usage Do we have enough RI coverage? Are our RIs being used?
- 7. Getting Usage Data In front of Teams Making Real Time Usage Data Visible Encourages Action Problem: The default Splunk for AWS App configuration is based on billing “subaccounts”, which Edmodo does not use. Edmodo uses tags to differentiate between cost centers.
- 8. Use These Tags: Product, Service, Space Configurable Metadata Available for All AWS Resources • Product ▸ designation is derived from the end user perspective. Where component resources serve overlapping Products, the Product designation is that which uses/requires a resource most or the resource's reason for existing. • Service is the primary function of the resource. Examples are ▸ mysql ▸ apache ▸ active-directory • Space a logical workplace designation. Examples are: ▸ Development ▸ Production
- 9. Edit the Splunk For AWS Scripts for Tags get_instances.py and get_ops.py
- 10. Create Lookup Tables For The Tags Simple .csv files are referenced by dashboard view files
- 11. Personalize Your Dashboard Views The Dashboard XML format is easy to customize
- 12. Drill Down Into Usage Data by Tags Teams Can Now Evaluate Usage For The Services They Manage
- 13. Drill Down Into Usage Data by Tags Teams Can Now Evaluate Usage For The Services They Manage
- 14. Drill Down Into Usage Data by Tags Teams Can Now Evaluate Usage For The Services They Manage
Editor's Notes
- #4: The power of Edmodo has now fuelled record-breaking growth It's no longer about bringing teachers into the 21st century, but how you can leverage the network effort. Up 240% in user sign-ups since last school year (Sept 2011 data)1 in 7 of all teachers use EdmodoIn 85% of top US school districts1 in 7 teacher calculation derivation:- Assume 80% of all 12M are US- 1:10 ratio of teachers to students- 7.2M teachers K-12 in US according to US 2010 Census
- #5: 95%of schools are connected to broadband600 K-12 Districts provide tablets to students55% of students access the Internet at home through mobile devices76% of teachers stream or download videos in classiPad sales in schools have doubled since last year