Elastic at Procter & Gamble: A Network Story
0 likes4,037 views
The document presents an overview of Procter & Gamble's journey in network automation, led by Andrea Fuggetta. It highlights the challenges faced in data management and the implementation of Elasticsearch to enhance data utilization and troubleshooting within the company. The future plans include expanding customer data integration, leveraging machine learning for anomaly detection, and improving data ingestion and alert systems.
Elastic at Procter & Gamble: A Network Story
- 1. 1 Andrea Fuggetta Sr. Software Engineer, Procter & Gamble @andreafuggetta Elastic at Procter & Gamble: A Network Story
- 2. 2 Agenda What are we talking about? Who we are1 Challenges3 The road so far4 What’s next?5 Problem statement2
- 3. 3 Who we are • Founded in 1837 (180+ years) • Superior quality products • More than 180 countries • http://www.pg.com/ Making every day more than ordinary
- 4. 4 Who we are (cont’d) • Andrea Fuggetta • Cincinnati (OH) • Sr. Software Engineer • Network Automation Making every day more than ordinary
- 5. 5 Problem statement • Insourcing • Where is our data? • What data do we have? • What are my processes and use cases? • How do we aggregate the data? • How do we leverage the data? • Where is our data!? Where did we start?
- 6. 6 I have not failed. I've just found 10,000 ways that won't work Thomas Edison
- 7. 7 Problem statement • So many choices • Trial and error • What is Elasticsearch? • Demo sessions • First visualizations • Interest from customers Where to start?
- 8. 8 It works on my machine Every programmer out there
- 9. 9 Challenges • Find the processes that govern your data • Find the people responsible for them • Find the technology to support the business and use cases Where to start?
- 10. 10 Challenges • How much data do we expect? ‒ Do you know all our data? ‒ Throughput • Infrastructure ‒ Cloud or DC ‒ Compute power ‒ Memory ‒ Storage • What product? ‒ Single server ‒ Local cluster ‒ Elastic Cloud ‒ ECE Scaling – Hosting – Product
- 11. 11 Challenges • Define the data sources ‒ Network metrics • Data flow ‒ 500MB/day • Infrastructure ‒ Cloud • What product? ‒ 3 node cluster ‒ Installed manually on VMs Solutions?
- 12. 12 “The important thing is to never stop questioning…” Albert Einstein
- 13. 13 The road so far • Define the data sources ‒ Network devices’ syslogs and metrics • Data flow ‒ 500MB/day ~6TB/day • Infrastructure ‒ Cloud • What product? ‒ Elastic Cloud Enterprise (ECE) • How? Current state
- 14. 14 The road so far Architecture
- 15. 15 The road so far Automation GitHub Azure DevOps Terraform Ansible Elastic ECE Logstash
- 16. 16 VMs Supporting ECE and Logstash pipelines. Hot/Warm/Cold lifecycle Some numbers Terabytes Data coming from network devices i.e. Firewall syslogs Teams Currently leveraging the solution 42 6 6
- 17. 17 Results • Prevented downtime and potential issues • Increased knowledge of our data • One destination for logs and metrics • Easier troubleshooting and forensics • Increased scalability and mobility • All in less than 1 year Long road ahead
- 18. 18 Results • Move infrastructure from Azure to AWS ‒ Load balancers ‒ Kafka-like queue (Kinesis) ‒ Virtual Machines ‒ Storage ‒ Monitoring ‒ Installing and configuring software (ECE, Logstash) • Half day Examples
- 19. 19 What’s next? • More customers – more data ‒ Information Security (SIEM) ‒ Data Science (Search, aggregation, analysis) • ML ‒ Anomalies detection • Cloud data and logs ‒ Function beats ‒ Custom ingestion pipelines • Alerts and actions ‒ Anomalies trigger alerts and scripts to self-heal • Canvas ‒ Executive views ‒ Hallway monitors Near future
- 20. 20 Thank you