NDC London 2014: Erlang Patterns Matching Business Needs
6 likes1,693 views
The document discusses the value of Erlang in meeting business needs for reliability and fault tolerance in software development. It emphasizes the importance of embracing failure and implementing defensive coding practices alongside Erlang's design principles, such as message passing and supervision trees, to enhance system performance and maintain continuous service. Additionally, it provides examples of successful implementations in telecom and messaging applications, highlighting Erlang's capabilities for handling massive concurrency and high reliability.
![Intentional
Programming
a style of programming where the
reader of a program can easily see
what the programmer intended by their
code. [1]
[1] http://www.erlang.org/download/armstrong_thesis_2003.pdf](https://image.slidesharecdn.com/2014-12-ndc-141204083233-conversion-gate02/85/NDC-London-2014-Erlang-Patterns-Matching-Business-Needs-42-320.jpg)
NDC London 2014: Erlang Patterns Matching Business Needs
- 1. Erlang Patterns Matching Business Needs Torben Hoffmann CTO @ Erlang Solutions torben.hoffmann@erlang-solutions.com @LeHoff
- 3. Background Erlanger since 2006 Happiness Mission critical gateway for Tetra Hard work Major learnings
- 4. Why this talk? Encourage you to embrace failure Show how The Golden Trinity of Erlang delivers business value Spread the Erlang love
- 5. Dealing with disbelievers… Source: http://2.bp.blogspot.com/-qNM3LGTtUYM/UIFLJGd_MLI/AAAAAAAAAnU/GCtI5SYfbCs/s320/orc-army.jpg source: http://www.rottentomatoes.com/m/1014027-mission/ source: http://images1.wikia.nocookie.net/__cb20110119125642/villains/images/e/ef/Saruman.jpg source: http://asset3.cbsistatic.com/cnwk.1d/i/tim2/2013/08/12/Larry_Ellison_Oracle_Open_World_2009_610x407.jpg
- 6. What Does Business Need?
- 7. wanted productivity no down-time something that always works
- 8. wanted money money money it’s a rich man’s world! Source: http://zapp5.staticworld.net/images/article/2011/10/cash_bag_of_money-5225858.jpg
- 9. Distributed Systems source: http://www.krug-soft.com/297.html How likely is it that this will “just work”?
- 10. Failure is unavoidable Software Errors Cost U.S. Economy $59.5 Billion Annually (NIST 2002*) * http://www.abeacha.com/NIST_press_release_bugs_cost.htm
- 11. The thinking it took to get us into this mess is not the same thinking that is going to get us out of it.
- 13. Why Erlang?
- 16. Polyglot is Good™ Source: http://www.americanscientist.org/issues/id.3489,y.0,no.,content.true,page.1,css.print/issue.aspx
- 17. Erlang not on the map is the #1 reason to learn it
- 18. Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang.” Robert Virding First Rule of Programming
- 20. Defensive Coding Data Mobility component breakdown Source: h*p://www.slideshare.net/ JanHenryNystrom/produc;vity-‐ gains-‐in-‐erlang 25% Defensive <20% App You are loosing money!
- 21. Heisenbugs Jim Gray - Why Computers Stop Heisenbug = goes away when retrying Bohrbug = a real bug Saw 132 faults - 131 were Heisenbugs http://www.hpl.hp.com/techreports/tandem/TR-85.7.pdf
- 22. Keys to Fault Tolerance Jim Gray - Why Computers Stop software modularity through processes and message passing fault containment through fail-fast software process-pairs to tolerate transient hardware & software faults transaction mechanism to give data and message integrity transaction mechanism combined with process-pairs to ease exception handling and tolerate software faults http://www.hpl.hp.com/techreports/tandem/TR-85.7.pdf
- 23. Separation of Concerns Not embracing failure means you loose the ability to handle failures gracefully! Golden Path Failure Handling GOBAODD!!!!
- 24. The Golden Trinity Of Erlang
- 25. Share Nothing
- 26. To Share Or Not To Share Memory Corrupt Memory Corrupt Memory P1 P2 P1 P2
- 27. Message Passing M P1 P2 P1 sends M to P2. Every process has a mailbox Messages are received: receive {tag, Value} -> Value; N when is_integer(N) -> N + 42 end
- 28. Fail Fast
- 29. Failures Anything that can go wrong, will go wrong. Murphy Programming errors Disk failures Network failures source: http://www.krug-soft.com/297.html Whatever can happen will happen. De Morgan
- 30. Fault In-Tolerance Most programming paradigmes are fault in-tolerant ⇒ must deal with all errors or die source: http://www.thelmagazine.com/BrooklynAbridged/archives/2013/05/14/ should-we-be-worried-about-this-brooklyn-measles-outbreak
- 31. Fault Tolerance Erlang is fault tolerant by design ⇒ failures are embraced and managed source: http://johnkreng.wordpress.com/tag/jean-claude-van-damme/
- 32. . Let It Fail convert(monday) -> 1; convert(tuesday) -> 2; convert(wednesday) -> 3; convert(thursday) -> 4; convert(friday) -> 5; convert(saturday) -> 6; convert(sunday) -> 7 ; convert(_) -> {error, unknown_day}. Erlang encourages offensive programming
- 33. Supervision
- 34. Handling Failure P1 P2 P1 monitors P2. P1 P2 P1 and P2 are linked.
- 35. PROPAGATING EXIT SIGNALS {'EXIT', PidA, Reason} PidA PidB PidC {'EXIT', PidB, Reason}
- 36. TRAPPING AN EXIT SIGNAL PidA {'EXIT', PidA, Reason} PidB PidC
- 37. Supervision Trees © 1999-2012 Erlang Solutions Ltd. 37 worker worker worker worker worker The OTP library is built on this principle supervisor supervisor
- 38. The OTP Supervisor supervisor © 1999-2012 Erlang Solutions Ltd. 38 worker worker worker Specifies a default restart strategy one_for_one one_for_all rest_for_one simple_one_for_one Child spec for how a child is restarted permanent | transient | temporary
- 40. Supervisors
- 41. Simple Manager/ Worker Pattern
- 42. Intentional Programming a style of programming where the reader of a program can easily see what the programmer intended by their code. [1] [1] http://www.erlang.org/download/armstrong_thesis_2003.pdf
- 43. Intentional Dictionary data retrieval - dict:fetch(Key, Dict) = Val | EXIT the programmer knows a specific key should be in the dictionary and it is an error if it is not. search - dict:find(Key, Dict) = {ok, Val} | error. it is unknown if the key is there or not and both cases must be dealt with. test - dict:is_key(Key, Dict) = Boolean
- 44. Saving Money
- 45. Benefits of let-it-fail code that solves the problem Erlang @ 3x Data Mobility component breakdown Source: h*p://www.slideshare.net/ JanHenryNystrom/produc;vity-‐ gains-‐in-‐erlang
- 46. Productivity with Erlang @ Motorola Goal: Development of a mission-critical telecom gateway for TETRA To be developed from scratch using Erlang and some drivers in C The gateway translates between proprietary protocols and the ISI standard Developed using a two – six man team over four year
- 47. Productivity with Erlang @ Motorola Function point analysis Language agnostic measurement of problem size Internal storage input input input input output output output
- 48. Show me the money! Function Point Analysis of the size of the problem Conservative estimation of the number of… inputs, outputs and internal storage Includes design, box test, system test, project management efforts
- 49. Dealing with deadlocks 7 years of coding Erlang Time spent on deadlock issues…. 1 hour (due to lack of experience with OTP)
- 50. Realities of software development Source: http://www.thejournal.ie/readme/lunch-atop-skyscraper-photo-men-irish-shanaglish-518110-Jul2012/ Product Ow?n?e??r
- 51. Business benefits of supervisors Only one process dies isolation gives continuous service Everything is logged you know what is wrong Corner cases can be fixed at leisure Product owner in charge! Not the software! Software architecture that supports iterative development
- 52. Where To Use Erlang
- 53. The Sweet Spot GUI Middleware Coordination Control Drivers
- 54. Erlang History
- 55. There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. - C.A.R. Hoare
- 56. Erlang’s Original Requirements Large scale concurrency Soft real-time Distributed systems Hardware interaction Very large software systems Complex functionality Continuous operation for many years Software maintenance on-the-fly High quality and reliability Fault tolerance Bjarne Däcker’s Licentiate Thesis: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.88.1957
- 57. General vs Domain Specific Telecom C++/Java Erlang Smaller gap = money!
- 58. If the glove fits... needs/fit C Erlang Telecom drivers coordination GUI
- 59. If our basic tool, the language in which we design and code our programs, is also complicated, the language itself becomes part of the problem rather than part of its solution. C.A.R. Hoare
- 60. Good Erlang Domains Low latency over throughput Stateful (in contrast to being stateless) Massively concurrent Distributed Fault tolerant Uses OTP Non-stop operation Under load, Erlang programs usually performs as well as programs in other languages, often way better. @jlouis
- 61. Other Erlang Domains Messaging - XMPP et al ejabberd, MongooseIM Webservers Yaws, Chicago Boss, Cowboy Payment switches & soft switches Vocalink, OpenFlow/LINC Distributed Databases Riak, CouchDB, Scalaris Queueing systems RabbitMQ (AMQP)
- 62. The glove fits! Low latency Stateful Massively concurrent Distributed Fault tolerant Messaging Webservers Soft switches Distributed DBs Queueing systems
- 63. Some Customers 19,000,000,000 reasons to use Erlang
- 64. WhatsApp Real-time Messaging Text and Pictures Group chat
- 65. WhatsApp Numbers 10 Erlang engineers ~500M monthly users 19B msg/day in / 40B msg/day out 147M concurrent connections peak: 324K msg/s in / 712K msg/s out
- 66. WhatsApp Hardware ~550 servers 2x2690v2 Ivy Bridge 10-core (40 threads total) 64-512 GB RAM SSD >11,000 cores
- 67. Bet365 Research team re-did a Java system in Erlang as a POC Results: 5x connected users in load test 4x rate of data change Better utilisation of CPU resources How to convince the developers to switch?
- 68. Bet365 showed that Erlang… makes programming fun scales and is reliable has enough depth to be interesting solves difficult problems with simple code Results: production teams quickly started to appreciate the benefits of Erlang did not want to go back to Java https://www.erlang-solutions.com/resources/webinars/webinar-recording-erlang-gamblingonline-betting
- 69. Learning Erlang
- 71. Learning Erlang ESL training courses Learn You Some Erlang http://learnyousomeerlang.com/ Use the erlang-questions mailing list Do it hands-on Give it time to sink in!!!
- 72. Elixir Built on top of the Erlang VM More Ruby-like syntax Hygienic macros - easy to do DSLs Better support for data handling But… you still have to learn the Erlang programming model
- 73. Cruising with Erlang Understand the failure model Embrace failure! Use supervision patterns to deliver business value Stay in charge!