How to create multiprocess server on windows with ruby - rubykaigi2016 Ritta Narita-
0 likes636 views
To create a multiprocess server on Windows with Ruby, there are a few key steps: 1. Use the spawn method instead of fork to create worker processes since fork is not available on Windows. 2. Use the fiddle library to call Windows API functions like WSADuplicateSocketA to share sockets between processes. 3. Call rb_w32_wrap_io_handle to convert sockets to file descriptors and allow using them with Ruby socket methods. 4. Use TCPServer instead of Unix domain sockets for inter-process communication due to the lack of UDS on Windows.
![How to use ServerEngine
1. write server module and worker module
serversupervisor worker
module MyServer
def before_run
@sock = TCPServer.new(
config[:bind],
config[:port])
end
attr_reader :sock
end
module MyWorker
def run
c = server.sock.accept
c.write "work"
end
def stop
c.close
end
end](https://image.slidesharecdn.com/rubykaigi2016narita-160914033108/85/How-to-create-multiprocess-server-on-windows-with-ruby-rubykaigi2016-Ritta-Narita-17-320.jpg)
![server worker
spawn
Worker type
module MyWorker
def spawn(process_manager)
process_manager.spawn(env, config[:spawn_cmd])
end
end
ServerEngine.create(MyServer, MyWorker, {
worker_type: 'spawn',
spawn_cmd: cmd
})](https://image.slidesharecdn.com/rubykaigi2016narita-160914033108/85/How-to-create-multiprocess-server-on-windows-with-ruby-rubykaigi2016-Ritta-Narita-21-320.jpg)
![Create Daemon with spawn
serversupervisor
spawn(config[:daemon_cmdline])](https://image.slidesharecdn.com/rubykaigi2016narita-160914033108/85/How-to-create-multiprocess-server-on-windows-with-ruby-rubykaigi2016-Ritta-Narita-31-320.jpg)
![serversupervisor
ruby daemon.rb
execute script for daemon from application side
:
ServerEngine::Daemon.run_server(
:
spawn(config[:daemon_cmdline])
Create Daemon with spawn](https://image.slidesharecdn.com/rubykaigi2016narita-160914033108/85/How-to-create-multiprocess-server-on-windows-with-ruby-rubykaigi2016-Ritta-Narita-32-320.jpg)
![2. ffi
Foreign function interface (FFI) for Ruby
require 'ffi'
module MyLib
extend FFI::Library
ffi_lib 'c'
attach_function :puts, [ :string ], :int
end
MyLib.puts 'Hello, World using libc!’](https://image.slidesharecdn.com/rubykaigi2016narita-160914033108/85/How-to-create-multiprocess-server-on-windows-with-ruby-rubykaigi2016-Ritta-Narita-42-320.jpg)
![2. ffi
typedef :ulong, :dword
typedef :uintptr_t, :socket
typedef :pointer, :ptr
ffi_lib :ws2_32
attach_function :WSADuplicateSocketA,
[:socket, :dword, :ptr], :int
class WSAPROTOCOL_INFO < FFI::Struct
layout(
:dwServiceFlags1, :dword,
)
end
proto = WinSock::WSAPROTOCOL_INFO.new
WSADuplicateSocketA(sock, pid, proto)
Definition
Use](https://image.slidesharecdn.com/rubykaigi2016narita-160914033108/85/How-to-create-multiprocess-server-on-windows-with-ruby-rubykaigi2016-Ritta-Narita-43-320.jpg)
![WSAPROTOCOL_INFO = struct([
"DWORD dwServiceFlags1",
"DWORD dwServiceFlags2”,
・・・
])
proto = WSAPROTOCOL_INFO.malloc
you can use struct
you can get struct pointer by calling malloc](https://image.slidesharecdn.com/rubykaigi2016narita-160914033108/85/How-to-create-multiprocess-server-on-windows-with-ruby-rubykaigi2016-Ritta-Narita-49-320.jpg)
![worker
server
bin = proto.to_ptr.to_s
proto = WSAPROTOCOL_INFO.malloc
proto.to_ptr.ref.ptr[0, size] = bin
send binary
How to share this protocol with worker
protocol → binary
binary → protocol](https://image.slidesharecdn.com/rubykaigi2016narita-160914033108/85/How-to-create-multiprocess-server-on-windows-with-ruby-rubykaigi2016-Ritta-Narita-51-320.jpg)
How to create multiprocess server on windows with ruby - rubykaigi2016 Ritta Narita-
- 1. How to create multiprocess server on windows with ruby Rubykaigi 2016
- 2. Ritta Narita @naritta @narittan Treasure Data, Inc.
- 3. Treasure Data Service Architecture
- 4. Treasure Data Service Architecture As one of solutions for importing data, we use fluentd
- 5. Fluentd is an open source data collector for unified logging layer.
- 6. For example… <source> @type http port 8000 bind 0.0.0.0 </source> in_http out_stdout <match pattern> @type stdout </match> http:8000
- 8. For example… <source> @type forward port 24224 bind 0.0.0.0 </source> in_forward out_stdout <match pattern> @type stdout </match> 24224 fluent-cat: send json to the port
- 10. fluentd uses multiprocess server framework github.com/fluent/serverengine ServerEngine
- 11. What’s ServerEngine? A framework to implement robust multiprocess servers like Unicorn Dynamic reconfig live restart Heartbeat via pipe auto restart serversupervisor worker worker worker
- 12. Auto Restart Heartbeat via pipe auto restart serversupervisor worker worker worker
- 13. Heartbeat via pipe auto restart serversupervisor worker worker worker if worker died unexpectedly…
- 14. It’s detected by heartbeat and server launch worker again. Heartbeat via pipe auto restart serversupervisor worker worker worker if worker died unexpectedly…
- 17. How to use ServerEngine 1. write server module and worker module serversupervisor worker module MyServer def before_run @sock = TCPServer.new( config[:bind], config[:port]) end attr_reader :sock end module MyWorker def run c = server.sock.accept c.write "work" end def stop c.close end end
- 18. 2. write configuration and run How to use ServerEngine serversupervisor worker se = ServerEngine.create(MyServer, MyWorker, { daemonize: true, log: 'myserver.log', pid_path: 'myserver.pid', worker_type: 'process', workers: 4, bind: '0.0.0.0', port: 9071, }) se.run
- 20. server worker server worker thread process Worker type ServerEngine.create(MyServer, MyWorker, { worker_type: 'thread' }) ServerEngine.create(MyServer, MyWorker, { worker_type: 'process' })
- 21. server worker spawn Worker type module MyWorker def spawn(process_manager) process_manager.spawn(env, config[:spawn_cmd]) end end ServerEngine.create(MyServer, MyWorker, { worker_type: 'spawn', spawn_cmd: cmd })
- 22. in_multiprocess plugin you have to use multi sockets and assign each ports port: 24224 port: 24226 port: 24225server worker worker worker conventional multiprocess model on fluentd
- 23. Multiprocess model with ServerEngine port: 24224 using Socket Manager, share one listening socket →can use multicore without any assignment port: 24224 port: 24224 server worker socket manager server socket manager client worker socket manager client worker socket manager client
- 24. What differs from other prefork server like unicorn or nginx
- 25. Ordinal prefork model can know which port before fork only have to pass the socket when fork server workersocket
- 26. In ServerEngine, For example, in fluentd Assuming that you can know which port after fork port information is written in config, it parses config in worker after fork port information server worker
- 27. serverengine prefork model After fork, have to send request from worker firstly, As a response, send socket to worker request socket server worker
- 28. server worker socket request socket manager server socket manager client socket socket information create worker socket http How do you create multiprocess server for windows with ruby? I’ll introduce ruby function and tips to develop for windows.
- 29. server worker socket request socket manager server socket manager client socket socket information create worker socket http To create worker
- 30. To create worker There is no fork in Windows, then I used spawn executes specified command and return its pid pid = spawn(RbConfig.ruby, "-e puts 'Hello, world!'") Process.wait pid
- 31. Create Daemon with spawn serversupervisor spawn(config[:daemon_cmdline])
- 32. serversupervisor ruby daemon.rb execute script for daemon from application side : ServerEngine::Daemon.run_server( : spawn(config[:daemon_cmdline]) Create Daemon with spawn
- 33. server worker socket request socket manager server socket manager client socket socket create worker socket http How to send a socket
- 34. There is a problem about File descriptor
- 35. In unix it can share file descriptor for other process. server worker FD FD FD
- 36. In Windows FD is independent for processes. you cannot share file descriptor server worker FD FD FD
- 37. Then, How do you send a socket in windows?
- 38. process pid:1000 process pid:2000 Windows API : WSADuplicatesocket you can get duplicated socket only for assigned PID socket dup socket for 2000 socket for 2000 pass
- 39. But this API is only for C Almost all of Windows API is for C. It’s important to call API from Others.
- 40. How do you call this API from Ruby?
- 41. 1. C extension The most general way There is no C extension code in ServerEngine. If possible, I want to write with only ruby. void Init_test() { VALUE module; module = rb_define_module("Test"); rb_define_module_function(module, "hello", wrap_hello, 0); }
- 42. 2. ffi Foreign function interface (FFI) for Ruby require 'ffi' module MyLib extend FFI::Library ffi_lib 'c' attach_function :puts, [ :string ], :int end MyLib.puts 'Hello, World using libc!’
- 43. 2. ffi typedef :ulong, :dword typedef :uintptr_t, :socket typedef :pointer, :ptr ffi_lib :ws2_32 attach_function :WSADuplicateSocketA, [:socket, :dword, :ptr], :int class WSAPROTOCOL_INFO < FFI::Struct layout( :dwServiceFlags1, :dword, ) end proto = WinSock::WSAPROTOCOL_INFO.new WSADuplicateSocketA(sock, pid, proto) Definition Use
- 44. This will increase gem dependency. I want to reduce dependency as possible. 2. ffi
- 45. 3. fiddle This is built-in library, then you can call without gem dependency require "fiddle/import" module M extend Fiddle::Importer dlload "libc.so.6","libm.so.6" extern "int strlen(char*)" end p M.strlen('abc') #=> 3 the library to deal with shared library
- 46. how to use fiddle you can just define using extern and c sentence. extern "int WSADuplicateSocketA(int, DWORD, void *)" WSADuplicateSocketA(sock.handle, pid, proto) Definition Use
- 47. how to use fiddle you can get protocol of duplicated socket extern "int WSADuplicateSocketA(int, DWORD, void *)" WSADuplicateSocketA(sock.handle, pid, proto) Definition Use
- 48. how to use fiddle extern "int WSADuplicateSocketA(int, DWORD, void *)" proto = WSAPROTOCOL_INFO.malloc WSADuplicateSocketA(sock.handle, pid, proto) Definition Use
- 49. WSAPROTOCOL_INFO = struct([ "DWORD dwServiceFlags1", "DWORD dwServiceFlags2”, ・・・ ]) proto = WSAPROTOCOL_INFO.malloc you can use struct you can get struct pointer by calling malloc
- 50. server worker socket request socket manager server socket manager client socket socket information create worker socket http To share this protocol with worker
- 51. worker server bin = proto.to_ptr.to_s proto = WSAPROTOCOL_INFO.malloc proto.to_ptr.ref.ptr[0, size] = bin send binary How to share this protocol with worker protocol → binary binary → protocol
- 52. server worker socket request socket manager server socket manager client socket socket information create worker socket http To create socket from sent information
- 53. Get socket from protocol extern "int WSASocketA(int, int, int, void *, int, DWORD)” socket = WSASocketA(Socket::AF_INET, Socket::SOCK_STREAM, 0, proto, 0, 1) Definition Use
- 54. This socket WSASocket is handle what’s handle? Handle is like FD, but we can’t duplicate handle like FD. For UNIX compatibility, There is function to convert handle to FD in Windows CRT
- 55. This handle is C layer we can’t useTCPServer class it’s inconvenient! we have to use ffi for everything like WSASend or WSAReceive for write or read I want to deal with socket in ruby layer
- 56. Ruby API: rb_w32_wrap_io_handle It can convert handle to FD socklist sock ioinfo sock ioinfo sock ioinfo handle you can deal with socket in ruby layer handle fd(7) wrap _open_osfhandle(assign FD) and create io object rb_w32_wrap_io_handle
- 57. Use fiddle to call rb_w32_wrap_io_handle ruby_dll_paths = File.dirname(ruby_bin_path) + '/*msvcr*ruby*.dll' ruby_dll_path = Dir.glob(ruby_dll_paths).first dlload ruby_dll_path extern "int rb_w32_wrap_io_handle(int, int)" you can call with loading ruby dll
- 58. you can deal with socket in Ruby layer
- 59. server worker socket request socket manager server socket manager client socket socket information create worker socket http To send request/response
- 60. There is no UDS in windows, then I used drb as first idea server worker worker drb drb drb socket method call method create socket socket send_io receive_io
- 61. This has race condition problem server workerA workerB drb drb drbmethod call create socket for workerA socketA send_io
- 62. This has race condition problem server workerA workerB drb drb drb create socket for workerB socketA send_io method call socketB send_io
- 63. This has race condition problem server workerA workerB drb drb drb create socket for workerB socketB send_io socketAcall receive_io
- 64. This has race condition problem server workerA workerB drb drb drb create socket for workerB socketBcall receive_io This rarely happens but its impact is significant if it happens.
- 65. server worker worker uds uds uds socket request Against this problem, I used UDS andTCPServer Create several UDS for each worker uds
- 66. To send request/result In Unix, can use Unix domain socket There is no UDS in windows, Then I usedTCPServer for port in 10000..65535 if `netstat -na | find "#{port}"`.length == 0 return port end end Use netstat to find unused port If there is NamedPipe in ruby, I’ll use it.
- 67. server worker socket request socket manager server socket manager client socket socket information create worker socket http Then you can share socket with worker, it can listen with multiple worker
- 68. server worker socket request socket manager server socket manager client socket socket information create worker socket http To deal with http request
- 69. There is no read_nonblock in windows. Used readpartial
- 70. read_nonblock readpartial with O_NONBLOCK raise an exception if there is no data block if there is no data without O_NONBLOCK difference between read_nonblock and readpartial
- 71. readpartial(n) read difference between read and readpartial read less than 2048 bytes read less than n bytes return nil if EOF raise an exception if EOF
- 72. There is no thundering herd problem on Windows? But I implemented accept mutex at first As a result of benchmark test, I found that thundering herd doesn’t occur. worker worker request accept! accept!
- 73. Accept Mutex get mutex accept mutex between processes read and send data to buffer/output server socket get mutex accept read and send data to buffer/output deal with post processing other process can listen while this process is dealing with data detach release mutex attach listening socket to event loop detach release mutex attach listening socket to event loop worker worker
- 74. rotation in order by accept mutex
- 75. Better way using IOCP? completion port thread threadthread socket socketsocket CreateIoCompletionPort Using WindowAPI for IOCP(I/O Completion Port), can implement asynchronous IO in Windows.
- 76. Better way using IOCP? completion port thread threadthread socket socketsocket GetQueued CompletionStatus GetQueued CompletionStatus listen accept create worker thread pool and IOCP manage it
- 77. Better way using IOCP? completion port thread threadthread socket socketsocket GetQueued CompletionStatus GetQueued CompletionStatus listen GetQueuedCompletionStatus accept
- 78. Better way using IOCP? ・Create IOCP in Server process and share socket? CreateIoCompletionPort server worker socket socket ・Or create it in only Worker process? CreateIoCompletionPort
- 80. Benchmark result AWS Microsoft Windows Server 2012 R2 m4.xlarge RPS IO conventional model 1834.01 /sec 385.07 kb/s new model (4 workers) 3513.31 /sec 737.66 kb/s in_http → out_forward
- 81. Conclusion We need to care about some problems using windows API. There is very helpful and niche function to deal with the difference between OS in ruby.