Tew4 Yatce presentation
0 likes1,195 views
This document discusses Yatce, a binding between Erlang and Tokyo Cabinet that allows Erlang programs to access Tokyo Cabinet databases. Yatce uses a linked-in driver approach where a C process communicates with Erlang like a process. This avoids failures separating processes and allows for message passing between Erlang and C. Yatce improved on previous TC-Erlang bindings by using port_control instead of port_command and an interface based on Tokyo Cabinet IDL. With Erlang R13B03, native interface functions were introduced that allow C code to run inside the Erlang virtual machine for faster performance than a linked-in driver.
Tew4 Yatce presentation
- 1. Erlang Tightly coupled Erlang Yatce: Yet Another TC-Erlang binding 4th Tokyo Erlang Workshop in Aoyama, 2010/2/26 @kuenishi http://creativecommons.org/licenses/by-sa/2.1/jp/
- 2. @kuenishi http://www.google.com/profiles/kuenishi : an employee Distributed System Developer, C/C++ OCaml …
- 3. Why YATCE?
- 4. Erlang
- 5. I/O
- 6. mnesia dets
- 10. TokyoCabinet
- 11. TokyoCabinet Erlang Erlang
- 12. Erlang C
- 13. C node
- 14. C node A process behaves like an Erlang node, BERT/TCP connected, Failure doesn’t involve the other. main(){ ... TCP Erlang C node ... node }
- 15. Port Driver
- 16. Port Driver ==Popen3, just an IPC, BERT/stdin/stdout connected, tolerant for Child process’ failure. fork/exec stdin Erlang any CUI process stdout
- 17. tcerl Pioneer in TC-Erlang! Port driver, “I couldn’t make it work”, - matthew, seems not being maintained.
- 18. Linkedin Driver
- 19. Linked-in Driver in-process erlang-port, % acts like an Erlang process % Message can be sent/recv’d, less failure-separeted. Erlang process output(..){ ?DEMO. ... }
- 20. message passing to port Process Process Process ! or port_command/3 ! or return port socket
- 21. Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
- 22. everything wrapped OS driver_alloc, driver_free driver_enq, driver_deq driver_monitor_process erl_driver_select erl_drv_thread_create, erl_drv_thread_join, erl_drv_mutex_lock, ... s/erl_drv_/p/g driver_set_timer, driver_cancel_timer, driver_read_timer R13B04 API
- 23. Toke Most famous in TC-Erlang, Linked-in driver, Simple: just what matthew/LShift wants, to be used in RabbitMQ?.
- 24. Yatce Made in Japan by me, Linked-in driver (0.10) port_control instead of port_command interface based on tokyocabinet.idl, http://bitbucket.org/kuenishi/yatce
- 25. Yatce 0.10 R13B02 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 3000 2250 average response time (us) 1500 750 0 0 1750 3500 5250 7000 Thruput (qps) ets dets yatce tcerl
- 27. port locking
- 28. Python GIL Ruby GVL driver
- 29. Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
- 30. ERTS Worker Worker Worker tcadb
- 31. TC
- 32. driver port port level locking TC DB port …
- 33. R13B03
- 34. Native Interface Functions Since R13B03, Beta, Built-in code runs inside ERTS, easy way to write Erlang module in C, Nothing separated.
- 35. it’s just a function pointer calling inside ERTS nif_func_in_c(...){ call_nif()-> nif_func_in_c(..), hoge, done. return term; } Erlang C
- 36. ERTS Thread Thread Thread tcadb
- 37. Yatce 0.20 R13B03 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 700 525 Response time (us) 350 175 0 0 5000 10000 15000 20000 Throughput (qps) ets dets yatce
- 38. NIF R13B04 erlang-patches erlang-questions GitHub erlang/otp NIF Erlang VM IO linkedin driver I/O pthread driver_select Yatce 0.20 TC TC NIF Yatce Erlang VM
- 39. Choose what fits to you! Slow C node Safe Port driver Linked-in driver port_command/3 port_control/3 Fast NIF Difficult
- 40. Any Questions? or come on to the party and ask me!
- 41. References ERTS: Interoperability tutorial guide http://www.erlang.org/doc/tutorial/overview.html BERT/BERT-RPC http://github.com/blog/531-introducing-bert-and-bert-rpc http://bert-rpc.org/ Erlang external term format http://erlang.org/doc/apps/erts/erl_ext_dist.html NIF http://erlang.org/doc/man/erl_nif.html Toke, vs tcerl http://bit.ly/8nPhm2