Comredis
0 likes331 views
This document discusses using metaprogramming to automatically generate Redis client functions in Elixir from a JSON specification of Redis commands. It describes parsing the command specification, generating Elixir macro code to define functions for each command, and property-based testing of the generated functions by sending random arguments to a Redis server and checking for valid responses. The automatic generation approach avoids bugs, reduces code duplication, and makes it easy to support new Redis commands as the specification is updated.
![Ruby
# Get the value of a key.
#
# @param [String] key
# @return [String]
def get(key)
synchronize do |client|
client.call([:get, key])
end
end
https://github.com/redis/redis-rb/blob/master/lib/
redis.rb](https://image.slidesharecdn.com/comredis-160430073535/85/Comredis-5-320.jpg)
![Elixir
Redix.pipeline(conn, [
~w(INCR foo), ~w(INCR foo), ~w(INCR foo 2)
])
#=> {:ok, [1, 2, 4]}](https://image.slidesharecdn.com/comredis-160430073535/85/Comredis-8-320.jpg)
![Elixir
Redix.pipeline(conn, [
~w(INCR foo), ~w(INCR foo), ~w(INCRBY foo 2)
])
#=> {:ok, [1, 2, 4]}](https://image.slidesharecdn.com/comredis-160430073535/85/Comredis-9-320.jpg)
![{
...
"GET": {
"summary": "Get the value of a key",
"complexity": "O(1)",
"arguments": [
{
"name": "key",
"type": "key"
}
],
"since": "1.0.0",
"group": "string"
},
...
}
https://github.com/antirez/redis-doc/blob/master/commands.json](https://image.slidesharecdn.com/comredis-160430073535/85/Comredis-19-320.jpg)
![quoteunquote
1 + 1
{:+, [context: Elixir, import: Kernel], [1, 1]}](https://image.slidesharecdn.com/comredis-160430073535/85/Comredis-25-320.jpg)
![Elixir
defp bodies(command, {required_args, []}) do
args = argument_names(required_args)
quote do
def unquote(command.canonical_name)(unquote_splicing(args)) do
List.flatten [unquote(command.name), unquote_splicing(args)]
end
end
end](https://image.slidesharecdn.com/comredis-160430073535/85/Comredis-27-320.jpg)
![Elixir
defp bodies(%Command{canonical_name: :get, name: "GET"},
{[%Argument{canonical_name: :key}], []}) do
args = [{:key, [], Elixir}]
quote do
def unquote(command.canonical_name)(unquote_splicing(args)) do
List.flatten [unquote(command.name), unquote_splicing(args)]
end
end
end
defp bodies(%Command{canonical_name: :get, name: "GET"},
{%Argument{canonical_name: :key}, []}) do
quote do
def get(key) do
List.flatten ["GET", key]
end
end
end](https://image.slidesharecdn.com/comredis-160430073535/85/Comredis-28-320.jpg)
Comredis
- 1. Comredis Iuri Fernandes Redis, Elixir, Ruby, Metaprogramming, Testing and other Buzzwords
- 2. Ruby require "redis" redis = Redis.new redis.set("hello ", "world") # => "OK" redis.get("hello") # => "world"
- 4. “Requests are sent from the client to the Redis server as arrays of strings representing the arguments of the command to execute.” –Redis documentation RESP (REdis Serialization Protocol)
- 5. Ruby # Get the value of a key. # # @param [String] key # @return [String] def get(key) synchronize do |client| client.call([:get, key]) end end https://github.com/redis/redis-rb/blob/master/lib/ redis.rb
- 7. Elixir {:ok, conn} = Redix.start_link # => {:ok, #PID<0.310.0>} Redix.command(conn, ~w(SET hello world)) # => {:ok, "OK"} Redix.command(conn, ~w(GET hello)) # => {:ok, "world"}
- 8. Elixir Redix.pipeline(conn, [ ~w(INCR foo), ~w(INCR foo), ~w(INCR foo 2) ]) #=> {:ok, [1, 2, 4]}
- 9. Elixir Redix.pipeline(conn, [ ~w(INCR foo), ~w(INCR foo), ~w(INCRBY foo 2) ]) #=> {:ok, [1, 2, 4]}
- 11. Ruby redis = Redis.new redis.set("hello ", "world") # => "OK" redis.get("hello") # => "world"
- 12. Elixir {:ok, conn} = Redix.start_link # => {:ok, #PID<0.310.0>} Redix.command(conn, ~w(SET hello world)) # => {:ok, "OK"} Redix.command(conn, ~w(GET hello)) # => {:ok, "world"}
- 13. Elixir {:ok, conn} = Redix.start_link # => {:ok, #PID<0.310.0>} Redix.command(conn, set("hello", "world"))) # => {:ok, "OK"} Redix.command(conn, get("hello")) # => {:ok, "world"}
- 14. Elixir {:ok, conn} = Redix.start_link # => {:ok, #PID<0.310.0>} conn |> Redix.command(set("hello", "world"))) # => {:ok, "OK"} conn |> Redix.command(get("hello")) # => {:ok, "world"}
- 15. Elixir defmodule Commands do def get(key) do ~w(GET #{key}) end def set(key, value) do ~w(SET #{key} #{value}) end # ... end
- 16. Bad idea!
- 17. lib/redis.rb => 2730 LOC* * Includes a lot of documentation * Handles responses * It does not support all the commands Not for the redis-rb
- 18. Imagine if I could generate all these functions automatically! You can! Metaprogramming FTW! Better, at compile time!
- 19. { ... "GET": { "summary": "Get the value of a key", "complexity": "O(1)", "arguments": [ { "name": "key", "type": "key" } ], "since": "1.0.0", "group": "string" }, ... } https://github.com/antirez/redis-doc/blob/master/commands.json
- 21. Macro rules Rule 1: Don’t Write Macros Rule 2: Use Macros Gratuitously
- 22. Elixir defmodule Comredis do use Comredis.Command.Generator ... end
- 25. quoteunquote 1 + 1 {:+, [context: Elixir, import: Kernel], [1, 1]}
- 26. Elixir defmodule Comredis.Command.Generator do defmacro __using__(_options) do for command <- FileReader.load do generate(command) end end defp generate(command = %Command{}) do quote do @doc unquote doc(command) unquote bodies(command, Argument.split_options(command.arguments)) end end ... end
- 27. Elixir defp bodies(command, {required_args, []}) do args = argument_names(required_args) quote do def unquote(command.canonical_name)(unquote_splicing(args)) do List.flatten [unquote(command.name), unquote_splicing(args)] end end end
- 28. Elixir defp bodies(%Command{canonical_name: :get, name: "GET"}, {[%Argument{canonical_name: :key}], []}) do args = [{:key, [], Elixir}] quote do def unquote(command.canonical_name)(unquote_splicing(args)) do List.flatten [unquote(command.name), unquote_splicing(args)] end end end defp bodies(%Command{canonical_name: :get, name: "GET"}, {%Argument{canonical_name: :key}, []}) do quote do def get(key) do List.flatten ["GET", key] end end end
- 30. How to test it? Don’t test code generation, but the generated code Write tests for each function? No, automatically test all of them
- 31. Property-based testing Generator: random arguments compliant to command arguments Property: commands have a valid syntax Use a Redis server as a test oracle github.com/parroty/excheck (triq)
- 33. What is missing? Type checking to use Dialyzer More documentation Something else?
- 34. Who uses similar ideas? Elixir - unicode representation ex2ms - ETS match expressions