FunctionalConf '16 Robert Virding Erlang Ecosystem
- 1. The Erlang Ecosystem Robert Virding
- 2. www.erlang-solutions.com Background: the problem β£Ericssonβs βbest sellerβ AXE telephone exchanges (switches) required large effort to develop and maintain software. β£The problem to solve was how to make programming these types of applications easier, but keeping the same characteristics.
- 3. www.erlang-solutions.com 3 β£Handle a very large numbers of concurrent activities. β£Actions must be performed at a certain point in time or within a certain time. β£System distributed over several computers. β£Interaction with hardware. β£Very large software systems. β£Complex functionality such as feature interaction. β£Continuous operation over many years. β£Software maintenance (reconο¬guration etc.) without stopping the system. β£Stringent quality and reliability requirements. β£Fault tolerance both to hardware failures and software errors. Bjarne DΓ€cker, November 2000 - Licentiate Thesis Background: problem domain β£Handle a very large numbers of concurrent activities. β£Actions must be performed at a certain point in time or within a certain time. β£System distributed over several computers. β£Interaction with hardware. β£Very large software systems. β£Complex functionality such as feature interaction. β£Continuous operation over many years. β£Software maintenance (reconο¬guration etc.) without stopping the system. β£Stringent quality and reliability requirements. β£Fault tolerance both to hardware failures and software errors. Bjarne DΓ€cker, November 2000 - Licentiate Thesis Not just telecom
- 4. www.erlang-solutions.com 4 Background: some reflections WE WERE TRYING TO SOLVE THE PROBLEM We were not out to implement the actor model We were not out to implement a functional language
- 5. www.erlang-solutions.com 5 Background: some reflections The language/systemβ¨ evolved to solve the problem β£We had a clear set of criteria for what should go into the language/system βΉ Was it useful? βΉ Did it or did it not help build the system? β£This made the development of the language/system very focused
- 6. www.erlang-solutions.com 6 Erlang and the system around it was designed to solve this type of problem Erlang/OTP provides direct support for these issues Background
- 7. www.erlang-solutions.com 7 βΈ Lightweight concurrency βΉ Must handle a large number of processes βΉ Process creation, context switching and inter-process communication must be cheap and fast. βΈ Asynchronous communication βΈ Process isolation βΉ What happens in one process must not affect any other process. βΈ Error handling βΉ The system must be able to detect and handle errors. βΈ Continuous evolution of the system βΉ We want to upgrade the system while running and with no loss of service. First Principles
- 8. www.erlang-solutions.com 8 Also βΈ High level language to get real benefits. βΈ The language/system should be simple βΉ Simple in the sense that there should be a small number of basic principles, if these are right then the language will be powerful but easy to comprehend and use. Small is good. βΉ The language should be simple to understand and program. βΈ Provide tools for building systems, not solutions βΉ We would provide the basic operations needed for building communication protocols and error handling First Principles
- 9. www.erlang-solutions.com 9 Hard at work developing Erlang
- 10. 10 βΈSimple functional language βΉ With a βdifferentβ syntax βΈIt is safe! βΉ For example no pointer errors βΈIt is reasonably high-level βΉ At least then it was βΉ Still is in many ways βΈDynamically typed! βΉ No user defined data-types! Sequential Language
- 11. 11 βΈTypical features of functional languages βΉ Immutable data βΉ Immutable variables βΉ Extensive use of pattern matching βΉ Recursion rules! Sequential Language
- 12. 12 << IpVers:4, HdrLen:4, SrvcType:8, TotLen:16, ID:16, Flags:3, FragOff:13, TTL:8, Proto:8, HdrChkSum:16, SrcIP:32, DestIP:32, RestDgram/binary >> βΈ IP datagram of IP protocol version 4 Sequential Language: high-level binaries
- 13. 13 Mike Williams: βthree properties of a programming language are central to the efficient operation of a concurrent language or operating system. These are: 1) the time to create a process. 2) the time to perform a context switch between two different processes and 3) the time to copy a message between two processes. The performance of any highly-concurrent system is dominated by these three times.β Concurrency
- 14. 14 βΈLight-weight processes βΉ Millions of Erlang processes possible on one machine βΈAsynchronous message passing βΉ Only method of communication between processes βΉ Necessary for non-blocking systems βΉ Provide basic mechanism βΉ Very cheap βΈSelective receive mechanism βΉ Allows us to ignore messages which are uninteresting now βΈProcesses are isolated! βΈNO GLOBAL DATA! Concurrency: core ideas
- 15. www.erlang-solutions.com 15 Error Handling: basic premise ERRORS WILL ALWAYS OCCUR!
- 16. www.erlang-solutions.com 16 Error Handling: basic goal THE SYSTEM MUST NEVER GO DOWN! Parts may crash and burn BUT The system must never go down!
- 17. 17 Robust systems must always be aware of errors BUT Want to avoid writing error checking code everywhere Want to be able to handle process crashes among cooperating processes Interact well with process communication Error Handling
- 18. 18 We just want to Error Handling Let it crash!
- 19. 19 βΈProcess based βΈIf one process crashes then all cooperating processes should crash βΉ Cooperating processes are linked together βΉ Process crashes propagate exit signals along links βΉ A process receiving an exit signal crashes βΈβSystemβ processes can monitor them and restart them when necessary by trapping exits βΉ exit signals are converted to messages in the process message queue βΉ the process does not crash βΈBut sometimes we do need to handle errors locally Error Handling
- 20. 20 How do you build robust systems? βΈAt least you need to ensure βΉ Necessary functionality is always available βΉ System cleans up when things go wrong βΈMust have at least two machines! βΉ Need distribution Robust Systems
- 21. 21 Supervision trees βΈSupervisors will start child processes βΉ Workers βΉ Supervisors βΈSupervisors will monitorβ¨ their children βΉ Through links and trapping exits βΈSupervisors can restart theβ¨ children when they terminate Robust Systems: necessary functionality Supervisors Workers
- 22. 22 Monitor processes βΈServers monitoring clients βΉ Clean-up after then if they crash βΈProcesses monitoring co-workers βΈGroups of co-workers dying together Robust Systems: system cleanup
- 23. 23 βΈA set of design patterns for building concurrent, fault tolerant systems βΈGeneric behaviours βΉ Implement the design patterns βΉ Extensible to support new patterns βΈLibraries βΈSupport tools for building systems/releases OTP (Open Telecom Platform) There is nothing about telecoms in OTP!
- 24. 24 Systems built with Erlang tend to be very OS like βΈProvides services βΈVery seldom a central thread of execution βΉ At most something which starts βtasksβ Systems
- 25. 25 What is the BEAM? A virtual machine to run Erlang
- 26. 26 βΈ Lightweight, massive concurrency βΈ Asynchronous communication βΈ Process isolation βΈ Error handling βΈ Continuous evolution of the system βΈ Soft real-time βΈ Transparent SMP/multi-core support βΈ Interfaces to the outside world These we seldom have to worry about directly in a language,β¨ except for receiving messages Properties of the BEAM
- 27. 27 βΈ Immutable data βΈ Predefined set of data types βΈ Pattern matching βΈ Functional language βΈ Modules/code βΈ No global data These are what we mainly "see" directly in our languages Properties of the BEAM
- 28. Erlang Ecosystem 28 Languages built/running on top of the BEAM, Erlang and OTP. By following βthe rulesβ the languages openly interact with, and support, each other making the whole system more powerful than any individual language can ever be.
- 29. Erlang Ecosystem 29 The whole system can interact with other systems
- 31. 31 βΈ Languages which keep the basic Erlang execution model and data types βΉ New syntax βΉ Different βpackagingβ βΈ Elixir βΈ LFE (Lisp Flavoured Erlang βΈ Languages which extend the Erlang execution model and data types βΈ Lua βΈ Prolog Extending the system: new skin for the old ceremony
- 32. 32 The thickness of the skin affects how efficiently the new language can be implemented and how seamlessly it can interact Extending the system: new skin for the old ceremony ERLANG BEAM OTP OTP Erlang New Language New Language libraries
- 33. 33 βΈβElixir is a dynamic, functional language designed for building scalable and maintainable applications.β βΈβElixir is influenced by Rubyβ βΉ βElixir is NOT Ruby on the Erlang VMβ βΈElixir has meta programming capabilities using macros βΈMany libraries and interfaces standardised, rewritten and extended βΈComes with extensive set of build tools Elixir Thin skin
- 34. 34 βΈ"LFE, Lisp Flavoured Erlang, is a lisp syntax front-end to the Erlang system" βΈIt's a real Lisp βΈProvides lots of lisp goodies βΉ Real homoiconicity and macros (yay!) βΈSeamlessly interacts with vanilla Erlang/OTP βΉ Be able to freely mix vanilla code and LFE code βΈSmall core language LFE (Lisp Flavoured Erlang) Thin skin
- 35. 35 βΈ"Lua is a powerful, efficient, lightweight, embeddable scripting language. It supports procedural programming, object-oriented programming, functional programming, data-driven programming, and data description." βΈImplements Lua 5.2 βΉ All of it! βΈShared, mutable, global data βΈLua handling of code βΈ... Lua Thick skin