OpenShift Origin Internals

Editor's Notes

• #10: And, once the application is launched within the OpenShift PaaS, OpenShift provides the elasticity expected in a Cloud Application Platform by automatically scaling the application as needed to meet demand. When created, applications can be flagged as “Scalable” (some apps may not want to be scaled). When OpenShift sees this flag, it creates an additional Gear and places an HA-Proxy software load-balancer in front of the application. The HA-Proxy then monitors the incoming traffic to the application. When the number of connections to the application crosses a certain pre-defined threshold, OpenShift will then horizontally scale the application by replicating the application code tier of the application across multiple Gears. For JBoss applications, OpenShift will scale the application using JBoss Clustering which allows stateful or stateless applications to be scaled gracefully. For Ruby, PHP, Python, and other script-oriented languages, the application will need to be designed for stateless scaling where the application container is replicated across multiple gears. The Database tier is not scaled in OpenShift today. Automatic application scaling is a feature that is unique to OpenShift among the popular PaaS offerings that are out there. Automatic scaling of production applications is another example of how OpenShift applies automation technologies and a cloud architecture to make life better for both IT Operations and Development. <next slide>
• #12: OpenShift Origin - Port Proxy Linux handles the loopback interface's 127.0.0.0/8 address block specially: A request from an address in this block can only go to an address in the same block (put another way, a connection on the loopback interface is confined to the loopback interface). OpenShift uses this fact to contain hosted applications: a gear is prohibited by iptables from listening on an external network interface, and so a given gear can only respond to connections that come from processes on the same node. For the common case of Web connections, the system Apache instance acts as a reverse proxy, forwarding requests that come in on the external interface to the appropriate 127.x.y.z address; see the documentation on the node component. However, sometimes gears need to accept other types of connections. The two most common such scenarios are the following: A gear needs to connect to another gear (which may be on the same node or another node). A gear needs to listen for connections on a public interface besides HTTP connections to port 80. For example, a game server needs to expose a port to receive incoming connections from clients, and a database needs to expost a port so that other gears can connect to it. To meet these needs, OpenShift uses haproxy to proxy TCP connections between an external-facing network interface and the loopback interface. Each gear is assigned five exposable ports, and the gear may establish a forwarding rule for each of these ports to forward connections on the the port on the external interface to an arbitrary port on the gear's assigned loopback address. To provide haproxy with adequate ports, we shift the ephemeral port range down to 15000-35530, so that Linux will not use ports outside of this range for connections for which no port is given explicitly. This means that ports 35531-65535 will be available for haproxy's exclusive use. Note: Given that each gear is assigned 5 ports, this imposes a limit of 6000 gears per node. The interaction with haproxy is implemented on the cartridge side in cartridges/openshift-origin-cartridge-abstract/abstract/info/lib/network and: OpenShift Origin - Node Component Hosted applications are run in containers called "gears." These gears are run on hosts (which can be physical hosts or virtual machines) called "nodes." Each node runs a system Apache instance with mod_proxy that listens on port 80 on a public-facing network interface. Each gear is assigned an address in the 127.0.0.0/8 block, and a hosted Web application listens on port 8080 on its assigned private 127.x.y.z address. When a Web client requests a URL for a hosted Web application, the request goes to the node's system Apache instance. The system Apache instance examines the virtual-host header (the "Host:" HTTP header) and dispatches the request to the 127.x.y.z:8080 private address of the appropriate gear. For an explanation of how connections other than regular HTTP connections are handled, see the documentation on the port-proxy.