Designing enterprise drupal

Editor's Notes

• #2: Welcome to our presentation over designing ent drupal environments. This presentation is being given from an infrastructure viewpoint, there are many different aspects to take into consideration from a development perspective and there a lot more qualified folks here that can talk about that aspect. Today we are going to walk you through the software that we have found has the best balance of stability and speed for applications that have to be up all the time and are high traffic sites. All slides will be available online once the presentation is complete and we will be here and available for questions.
• #4: A good network is an absolute requirement in order to run a big Drupal environment, if you plan on getting any press or decent traffic at all, then making sure that you set requirements for your hosting provider is crucial. The main points to require are 1) You should require a gigabit, dedicated connection to their core network 2) A network that has a guaranteed SLA 3) a guarantee that they have the capacity to support your growth.
• #5: This is the semi-unofficial standard Drupal stack. There are a lot of packages here that can be replaced by other packages, some better, some worse. Through our experience we have found that this is currently the most scalable and reliable stack we have deployed. That does not mean we have a problem with Cassandra or nginx, just that we have not gone through enough deployments with those packages to call them “standard” yet. That could change very soon. Quick descriptions: Varnish is the outward most facing cache APC is our opcode cache Apache/PHP is our webserver and interpreter Pressflow is the port of Drupal that we currently use Memcache is our backend caching mechanism MySQL is our database system
• #6: (Directly from fourkitchens.com) Support for database replication Replication in MySQL takes the data from a designated master server and copies it out to any number of other MySQL servers. Pressflow can then use these replicated MySQL servers for time-consuming query operations that would otherwise slow down the central database. Pressflow’s replication model (and much of the same software) is currently in use on Drupal.org and numerous large Drupal sites, but similar support for replication won’t be available in standard Drupal releases until version 7, which includes a new database abstraction layer. Support for Squid and Varnish reverse proxy caching A reverse proxy cache takes most of the load of anonymous browsing off of Drupal, PHP, Apache, and MySQL by placing a high-performance cache in front of the rest of the web application stack. This cache accelerates static content, like CSS, Javascript, and image files, as well as full web pages for anonymous users. Squid and Varnish are the most popular free, open-source reverse proxy caches, and are used to deliver sites as high-traffic as Wikipedia. The key to effective reverse proxy cache deployment is having the content management system inform the cache what it can and cannot cache. Currently, only Pressflow supports this capability. Optimization for MySQL Drupal is designed to run on MySQL, PostgreSQL, and (beginning with Drupal 7) SQLite. While this gives Drupal broad storage support, the vast majority of Drupal sites run on MySQL, especially the largest ones. Pressflow only supports MySQL, which allows it to quickly integrate optimizations that, when proposed for Drupal, often face delays resulting from the need to support multiple database engines. Optimization for PHP 5 Drupal includes “wrapper” functions to provide PHP 5-style operations on PHP 4. These wrapper functions increase the size of the code and are often dramatically slower than their native PHP 5 equivalents. Because Pressflow only supports PHP 5, it can replace these wrapper functions with their high-performance PHP 5 equivalents.
• #7: Varnish is our reverse proxy cache, this is the biggest win available when trying to scale a site to accommodate mass traffic increases.
• #8: Here we are telling Apache to STOP listening on port 80 and instead listen on port 8080, we also inform varnish that it should accept all incoming port 80 requests and utilize apache running on port 8080 to populate it’s cache and serve those requests.
• #9: Apache logs are really only helpful if you want to compare how much traffic is getting handled by varnish, vs how much is making it through to apache
• #10: Varnish cannot cache as effectively when cookies are present. By default, it does not cache anything if a cookie is present in the request headers. It creates a hash from the HTTP headers to look up cached content, and if anything is different (i.e. cookie content), a different hash is generated and the benefits of caching are negated. Vcl_recv is executed on every incoming request Google analytics continues to function, even without the cookies, so we strip these. Any other cookies that are not actually processed by your application are also safe to strip The rest of this function is just cleanup, so we are creating as many consistent hashes as possible Vcl_hash is called when varnish calculates the hash. This code instructs it to hash the cookie headers instead of just skipping it.
• #11: Protect yourself against DOS attacks by only allowing specific network segments to access cron.php. 404-cron.php does not need to exist, drupal will handle it with its own 404 error handling, and this entry will be logged so that you can be aware of any unwanted access attempts to cron.php This same procedure can be used to protect other sensitive URLs, such as update.php or install.php
• #12: Because varnish is able to serve pages from cache so quickly, it can handle a much larger number of incoming HTTP connections than a standard apache server. As such, it needs to be able to open and maintain a large number of file handles to manage its HTTP connections and cache content. Here we are just telling to kernel to allow more file handles than is enabled by default.
• #15: Again, every application is different, so these values will need to be tuned accordingly, but a good starting point is 120MB In addition to the php.ini configuration, the kernel needs to be configured to provide access to enough shared memory
• #16: Every table in Drupal mysql that starts with cache_ can go into memcache and it’s common to setup a separate bucket for each table. A bucket is a term used to identify an allocation of memory that is set aside for a portion of the cache. Each bucket is actually a separate memcache instance.
• #17: Multiple bins require separate instances on different ports
• #18: Setting up the Drupal memcache module to utilize our newly created bins is pretty straight forward. You provide the memcache server in the memcache_server array, Assign each of the memcache instances to a drupal table in the foreach loop, then assign the tables to buckets in the memcache_bins array.
• #21: This is what we will assume for the rest of the presentation
• #22: Varnish can use it’s own server depending on your configuration.
• #23: This is fairly standard for a starting site.
• #24: Phase 1, beta|testing
• #25: LVS, F5, Cisco CSS 11501
• #26: Failover for redundancy, not load balanced
• #27: Multiple servers to one place
• #34: Just a few items that we have come across.