Predicting failures on complex machines by Ion Marqués at Big Data Spain 2015
0 likes350 views
Outline NEM Solutions focuses on providing management solutions for complex multi-system assets, addressing client needs for efficient and sustainable operations. They employ a data-driven approach utilizing machine learning and computational units to predict failures and improve reliability. Key insights include the necessity for scalable data processing and efficient algorithm integration in decision-making tools.
Predicting failures on complex machines by Ion Marqués at Big Data Spain 2015
- 2. Predicting failures on complex machines Ion Marqués
- 3. OUTLINE NEM Solutions provides complete management solutions to businesses responsible for the operation and maintenance of multi-system assets.
- 4. OUTLINE NEM Solutions provides complete management solutions to businesses responsible for the operation and maintenance of multi-system assets. Nowadays, we have clients with thousands of assets, generating massive volume of data.
- 5. OUTLINE NEM Solutions provides complete management solutions to businesses responsible for the operation and maintenance of multi-system assets. Nowadays, we have clients with thousands of assets, generating massive volume of data. What we’ll see in the following 15 minutes: 1. The client’s needs 2. Our approach 3. The solution’s overview 4. The engine - the core of the solution. 5. How we did it, what did we learn.
- 6. DEMAND FOR EFFICIENT AND SUSTAINABLE TRANSPORTATION SYSTEMS. HIGH SPEED & URBAN TRANSPORTATION NEEDS ON THE RISE. INCREASING ENERGY NEEDS. ON & OFF SHORE RENEWABLES GROWING. NEED FOR PRODUCTIVITY, RELIABILITY AND CONTINUOUS IMPROVEMENT. THE CLIENTS’ NEEDS REACTIVE APPROACH The business under control Avoid surprises The unexpected happens Business plan fails BUSINESS & KNOWLEDGE
- 7. Normality model definition Normality model Vs = Failure Symptoms Real time data FUTURE PROJECTION FROM DATA KNOWLEDGE GENERATION A.U.R.A: ARTIFICIAL INMUNE SYSTEM
- 8. OUR BIG DATA SOLUTION
- 9. THE WORKFLOW: 1st APPROACH • We translate the calculations to a topology. • Each topology node is a computational unit, i.e arithmetical operations, symptom calculations, machine learning algorithm testings, … • Each node is a Storm bolt. We had around 160 bolts each doing one task.
- 10. THE WORKFLOW: 1st APPROACH • We translate the calculations to a topology. • Each topology node is a computational unit, i.e arithmetical operations, symptom calculations, machine learning algorithm testings, … • Each node is a Storm bolt. We had around 160 bolts each doing one task. • One “master” spout. • If a bolt fails, all the data must be re- emmited!
- 11. THE WORKFLOW: 2nd APPROACH • We translate the calculations to a topology. • Each topology node is a computational unit, i.e arithmetical operations, symptom calculations, machine learning algorithm testings, … • Each node is a Storm bolt. We had around 160 bolts each doing one task.
- 12. THE WORKFLOW: 2nd APPROACH • We translate the calculations to a topology. • Each topology node is a computational unit, i.e arithmetical operations, symptom calculations, machine learning algorithm testings, … • Each node is a Storm bolt. We had around 160 bolts each doing one task. • One spout per variable • Too much communication for our case. • Not efficient enough.
- 13. THE WORKFLOW: CURRENT APPROACH • We translate the calculations to a simple topology. • Non-codependant tasks are grouped into computational units. • We have a few nodes, assigning one executor per task.
- 14. THE WORKFLOW: CURRENT APPROACH • We translate the calculations to a simple topology. • Non-codependant tasks are grouped into computational units. • We have a few nodes, assigning one executor per task. • Same parallelization. • Less communication. • Adapted to small clusters. • Better performance.
- 15. WE HAD: The knowledge about the industries’ needs. The machine learning methodologies to extract useful information. A successful non-scalable product. CONCLUSION
- 16. WE HAD: The knowledge about the industries’ needs. The machine learning methodologies to extract useful information. A successful non-scalable product. CONCLUSION WE NEEDED: o The means to make that product capable of processing massive amount of data. o To solve a key point: Embedding algorithms into a scalable streaming framework.
- 17. • ROI: Industry demands tools that assist in making decisions affecting lots of complex machines. • In order to meet that particular demand, we need more than amazing visualizations and simple data mining methods. LEASONS LEARNED
- 18. • ROI: Industry demands tools that assist in making decisions affecting lots of complex machines. • In order to meet that particular demand, we need more than amazing visualizations and simple data mining methods. LEASONS LEARNED Technically, it is a challenge: • Kafka+Storm+Redis+Hbase can be a winning choice. • There’s no free lunch, and every case is different. • Translate your algorithms into a path the data will cross: A directed graph, a topology. Then simplify. Fail. Try again. • Your team must know your problem: From how heat in a wind rotor behaves to how failures in Storm propagate.
- 19. LISTENING TO YOUR ASSETS NEM Solutions +34 943 30 93 28 info@nemsolutions.com @NEMSolutions Thank you!