Docker + GCE + etcd + ray tracing
9 likes3,154 views
This document discusses using Docker, Google Compute Engine (GCE), and etcd to distribute a ray tracing workload across thousands of nodes. Docker would allow controlling resources for each renderer instance and managing versions. Etcd would enable scalably storing cluster information like node addresses. GCE is suitable due to its CPU-intensive instances and ability to quickly provision them. The proposed architecture involves render nodes pulling tasks from a Redis queue via etcd and processing scenes in parallel. Future work includes leveraging accelerators and automating the process.
![Get redis info
from etcd
http.get(etcdHost + '/v2/keys/redis-server', function(res) {
if (res.statusCode != 200) {
console.log('failed to get redis infor from etcd.');
process.exit(-1);
}
res.on('data', function(chunk) {
var j = JSON.parse(chunk);
var redisURL = url.parse(j['node']['value']);
var redisServerAddr = redisURL['hostname']
var redisPort = redisURL['port']
var redisClient = redis.createClient(redisPort,
redisServerAddr);
Sunday, April 27, 14](https://image.slidesharecdn.com/docker-meetup-2-140427004417-phpapp02/85/Docker-GCE-etcd-ray-tracing-12-320.jpg)
Docker + GCE + etcd + ray tracing
- 1. Docker + GCE + etcd + ray tracing @syoyo Sunday, April 27, 14
- 2. ray tracing Sunday, April 27, 14
- 3. Problem • A lot of computing resource required • Imagine 60fps photorealistic ray tracing for OculusVR • ~ 10K nodes • Efficient deploy, operation for the renderer(ray tracer) Sunday, April 27, 14
- 4. Why docker • Hoping... • Fine resource control • CPU limit, memory limit, etc • Easy versioning management of the renderer • Application sandboxing Sunday, April 27, 14
- 5. My contribution (1/2) Sunday, April 27, 14
- 6. Why etcd? • Hoping... • scalable management of cluster information • node IP addr, redis addr, etc. • up to 10K nodes possible(?) Sunday, April 27, 14
- 7. Why GCE • CPU-intensive instance available • fastVM boot, minute-level charge • CoreOS image avaiable • 282.0.0(Docker 0.10) Sunday, April 27, 14
- 8. Render A Render B Render C Render D Render E etcd Architecture Sunday, April 27, 14
- 9. Container • Busybox(8MB) • Our renderer(single binary. 72MB) • node.js(11 MB) • ~ 90 MBBusybox Renderer(Ray tracer) node.js (application frontend) Sunday, April 27, 14
- 10. Dockerfile FROM syoyo/aobench MAINTAINER Syoyo Fujita(syoyo@lighttransport.com) ADD libdl.so.2 /lib64/ ADD librt.so.1 /lib64/ ADD lte /bin/lte ADD worker.js /home/default/worker.js ADD node /bin/node ADD node_modules /home/default/node_modules Sunday, April 27, 14
- 11. Render A Render B Render C etcd etcd: 172.17.42.1:4001 webfrontend redis Sunday, April 27, 14
- 12. Get redis info from etcd http.get(etcdHost + '/v2/keys/redis-server', function(res) { if (res.statusCode != 200) { console.log('failed to get redis infor from etcd.'); process.exit(-1); } res.on('data', function(chunk) { var j = JSON.parse(chunk); var redisURL = url.parse(j['node']['value']); var redisServerAddr = redisURL['hostname'] var redisPort = redisURL['port'] var redisClient = redis.createClient(redisPort, redisServerAddr); Sunday, April 27, 14
- 13. Task processing function loop() { redisClient.brpop('render-q', timeout, function(err, reply) { // kick the ray tracer exec(...); // resubmit event loop setTimeout(loop, 100); }); } Sunday, April 27, 14
- 15. Future work • ~10K nodes • Leverage PCI-ex accelerator(GPU, Xeon Phi) in Docker container. • Confirmed InfiniBand working. • Automation using Docker API Sunday, April 27, 14