Designing and developing challenging Drupal sites

Agenda
9.00 Opening words Janne Kalliola, Exove

9.10 Drupal performance through Kalle Varisvirta, Exove
modules
9.25 Case: Suomi24 Mikko Ristimäki, Suomi24
9.40 Continuous Availability and Joonas Iivonen, Exove
Continuous Change
9.55 Case: Nelonen.fi Jonne Laakso, Sanoma
Entertainment Finland
10.10 Break

10.20 Designing and Developing Jani Tarvainen, Exove
Multilanguage Websites
10.25 Case: CSR-Kompassi Linda Piirto, Ministry of
Employment and the Economy
10.50 Discussion

How to design and develop
big and challenging sites with
Drupal?

Know what you / your
business / your customers
want.

Check that Drupal is the right
choice for you.

Pay close attention to concept
and visual design.

Implement the system using
the strengths of the platform.

Design. Implement. Analyse. Fix.

When in doubt, ask
professionals for help.

Exove is one of the leading Nordic and
Baltic companies specialising in open
source web services design and
development.

We enable companies to conduct
better business on the Internet
through best-of-breed personnel
and solutions

Our Approach

Understanding Power of open
your business source

Our expertise

Results

Beautiful,
functional &
business-
driven
services

55 people, over 120 customers,
over 3 MEUR revenue 2011, profitable

Platform
  Since Drupal is a PHP software, all platform
enhancements regarding PHP in general will
usually work for Drupal as well

Platform
  Always use a opcode cache (e.g. APC) with
optimized settings
  Nginx/FPM uses less memory than Apache/
mod_php and results in better performance
especially with a lot of concurrent users
  Nginx is also way more powerful in serving binary
files
  Drupal uses MySQL always, so make sure your
MySQL settings are optimal (they vary with different
data)

Drupal settings
  Drupal has some internal settings concerning
performance
  In Configuration/Performance you can set the
Drupal internal anonymous page cache on
  You can also enable JS and CSS aggregation
  In Configuration/Logging you can also tune
logging settings

Drupal settings
  Always use Drupal page cache unless you’re
using an external anonymous cache (or you
have something special preventing it)
  JS/CSS aggregation will make the browsing
experience better, but won’t affect your backend
load level
  Logging, in some cases, may cause serious
load to the database, so keep your logs short
and sweet

Drupal performance
modules
  Drupal has a lot of performance –related modules
  This doesn’t mean Drupal’s especially slow, it’s
comparable to other CMSs, but that Drupal is used
a lot on really high volume sites
  I’ll go through some of the key modules next
  There’s a lot of different modules available, for more
info follow the High Performance group on
Drupal.org (http://groups.drupal.org/high-
performance)

Replacing Drupal cache
backend
  Drupal uses it’s only storage engine MySQL for also
storing cache, sessions and locks
  Using different modules you can make it use other
available storage backends
  Memcache (http://drupal.org/project/memcache) is
a module that will make Drupal use Memcached as
the cache, sessions and/or locks backend
  It makes Drupal’s internal cache way more effective
and lessens the burden on the MySQL
  There are similar modules for many different
backends, such and APC, MongoDB and Redis

Optimizing Drupal for
anonymous users
  To gain maximum performance for anonymous
users you can use an external cache
  If you can’t use an external caching software,
there are still solutions for using something other
than Drupal for caching
  If you can use an external software then Varnish
is the best choice
  Varnish is an external http caching software that
can serve 10 000 requests per second

Using an external cache
  With no extra software the right way is to use Boost
(http://drupal.org/project/boost) which stores all
ready-made html pages to the disk and serves
them from there
  If you can install Varnish, that would be even faster
  To use Varnish, you need to use some module
enabling you to tune the cache headers on Drupal’s
end, Varnish (http://drupal.org/project/varnish) and
Cache Control (
http://drupal.org/project/cache_control) should both
do the trick

logged in users
  After you’ve optimized your site for anonymous
users, it’s time to look into logged in users
  If your site doesn’t have logged in users, that’s
great and makes things easy
  If your site is only accessible through a login,
then this should get really interesting for you

logged in users
  When a user logs in to Drupal, the Drupal
internal page caching turns off
  That’s to enable users to have personalized
content
  It also makes the site significantly slower and
burdens the back-end
  There are solutions for making things better

Cache Control
  Already mentioned Cache Control module is a
module that enables you to cache most of a page
being served to a logged in user
  The user will get the same cache stored page from
Varnish that the anonymous users get but it will be
enriched with personalized content using AJAX
  Cache Control can give a significant boost to logged
in user peformance
  There’s also the ESI module, which uses the Edge
Side Includes functionality, a technology that has it’s
downsides compared to pure Javascript

Front themer
  Front themer (http://drupal.org/project/front_themer)
is a module for moving theming partially to the front-
end
  Drupal’s own theming engine user a lot of CPU
cycles when theming small snippets of personalized
content for users
  Front themer can be used to use those CPU cycles
on the user’s computer instead of the server

Certain case-specific
optimizations
  When using Drupal for a big site, you might
bump into a special problem, that will only occur
with large amounts of data
  For example, if your menus grow into tens of
thousands of items, Drupal’s node edit page
becomes unusable
  Or, using Views with a large database can
cause problems in searching the rows to be
displayed

Menus
  To optimize very large menus, you should use
Menuperformance (
http://drupal.org/project/menuperformance)
  It splits the Drupal internal menu selection
dialogues to smaller bits and only queries the
database part by part
  It makes the node edit page actually work with
large menus

Views backend
  When a large node table is searched for a View,
it might cause problems with the MySQL
  Search API (http://drupal.org/project/search_api)
enables you to switch the backend from MySQL
  With submodules, you can switch to backend to
Apache SOLR or MongoDB
  This will make small Views calls to large
databases faster

Performance and your
product roadmap

Mikko Ristimäki
June 1, 2012

June 1, 2012 | 40

It’s a place where people come to
Discuss and chat about stuff

Find new online friends

Date online and offline

Read and send emails

Check news and tv programming

Buy and sell stuff

Read comics and horoscopes

Etc...

June 1, 2012 | 41

And it’s relatively big place
Over 12 million pages

Hundreds of gigabytes of DB’s

Around 30 TB storage

Millions of lines of code

And so on...

June 1, 2012 | 42

Traffic and usage patterns

In addition to plain requests consider
-  Static or dynamic content

-  Personalized features for authenticated users

-  Amount of simultanous users

-  Read or write intesive features

-  API traffic

-  3rd party systems

-  SSL

June 1, 2012 | 43

Key aspects for mastering the performance dilemma

1.  Know your current traffic and usage patterns

2.  Know the solutions and hacks that give you the current performance

3.  Know your performance bottlenecks

4.  Understand if a change or a new feature affects the balance
between 1, 2 and 3

5.  Refactor, optimize, repeat

6.  Test performance during development cycle (CI, acceptance, release)

7.  Have a scaling plan ready for the infrastructure

8.  Make sure that your team and subcontractors is aware of all the
above

June 1, 2012 | 44

How to keep Drupal performing

1.  The basic stuff
-  Choose the right caching solutions matching your traffic and usage patterns
-  Run Pressflow and use slave reads
-  Offload applicable data to Solr or NoSQL database
-  Keep the codebase clean and don’t use excess or heavy modules

2.  Look at your product roadmap
-  Is there something that will change the load or usage patters?
-  Try to estimate beforehand when do you need more capacity or refactor
performance related solutions

3.  Look at your site usage and data growth
-  Should we remove stuff from the DB?
-  How much growth we can take with these features and services?

June 1, 2012 | 45

Tips for scaling the infrastructure

•  Make sure you have nicely clustered and virtualized infrastructure

•  and that all the clusters can be scaled independently

•  and maybe you can scale to the cloud?

•  You can also automate scaling!

•  Network is hardest part to scale. Make sure both internal and internet
bandwidth have plenty of room to grow.

•  Consider fluid storage if you have a lot of data

June 1, 2012 | 46

Kiitos!

mikko.ristimaki@s24.fi
@mpristim

June 1, 2012

June 1, 2012 | 47

Continous availability
  Continous availability means that our servers
have to be allways on and serving the content
  Traditionally this means…
  multiple servers,
  backups and disaster plans,
  software version upgrades with detailed plans
  But it is also linked to changes in content

Inconsistency

  Changes in linked content

Changes in linked content
This page is being This page is being
This page is being
edited and has edited and has
edited and has
links to other links to other
links to other
pages. pages.
pages.
-  Link 1 - -  Link to aapage
Link to page
-  Link 2 - -  Link to aapage
Link to page
-  Link 3 - -  New link to…
New link to…

Atomic change

  All has changed or nothing has changed
  Prevent inconsistency during the change by
changing everything on one go

  Content version upgrade with detailed plans
  Editing as draft, publishing with schedule

Inconsistency


  Changes on one of the several servers

Changes on one server…
Editor

Server
Server

Server

Managing the sessions

  Access content only from servers in same state

  Manage the load balancing based on sessions

Inconsistency


  Changes on one of the several servers

  Does this scale?

Content from users

Server
Server

Server

Huge amounts of changes
1  Lorem ipsum dolor sit
amet, consectetur
2  adipisicing elit, sed do
eiusmod tempor incididunt
3  ut labore et dolore
magna aliqua. Ut enim ad
4  minim veniam, quis
nostrud exercitation
5  ullamco laboris nisi ut
aliquip ex ea commodo
6  consequat. Duis aute
irure dolor in reprehenderit
7  in voluptate velit esse
System X

cillum dolore eu fugiat
8  nulla pariatur.
Excepteur sint occaecat
9  cupidatat non proident,
sunt in culpa qui officia
10  deserunt mollit anim id
est laborum. Lorem ipsum
11  dolor sit amet,
consectetur adipisicing
12  elit, sed do eiusmod
tempor incididunt ut labore
13  et dolore magna aliqua.
Ut enim ad minim veniam,

Server
14  quis nostrud
exercitation ullamco l
15  aboris nisi ut aliquip ex
ea commodo consequat.
16  Duis aute irure dolor in
reprehenderit in voluptate
17  velit esse cillum dolore
eu fugiat nulla pariatur.
18  Excepteur sint
Server
occaecat cupidatat non
19  proident, sunt in culpa
qui officia deserunt mollit
20  anim id est laborum.
21  …
22  …

Server

Challenges

  Updating a node takes only a little while.
  Updating thousands of nodes takes a lot longer.

  When we have thousands of nodes, which are
in continous change, this can be a problem for
continous availability.

Caching as facade

  Keep old version in cache during the big change

  Clear (and reload) the cache after the change

What kind of availability?

  Does everyone need to get the newest content?

  Are some changes linked to each other?

  Plan the order of priority

  Know your use case

  Plan your content change

  Embrace the change

Television and Internet will merge
Jonne Laakso, Development Manager, Nelonen Media

Nelonen Media is part of Sanoma Group

Sanoma – strong European media company
Aikakauslehdet
Televisio
Sanomalehdet

Oppimateriaalit
Norja Suomi
Verkkoliiketoiminta
Ruotsi
Viro Jakelu

Liettua
Venäjä

Englanti
Hollanti

Puola
Belgia
Tsekki Ukraina

Unkari

Slovenia
Romania
Kroatia
Serbia
Bulgaria

Lisäksi Esmerk toimii Suomessa, Englannissa, Ruotsissa, Saksassa, Ranskassa, Venäjällä ja Malesiassa.
AAC Global toimii Suomessa, Ruotsissa, Tanskassa, Norjassa, Englannissa, Venäjällä ja Kiinassa.

67

Nelonen Media is a diverse and strongly developing
broadcasting group reaching 3,7 million Finns weekly

3 ad-funded ja ruutu.fi –web-tv +
2 radio stations
5 pay-tv channels channels websites

TV is not dead

http://www.flickr.com/photos/schmilblick/

Teens currently watch almost four hours of TV a day

http://www.flickr.com/photos/davet/

Hot topics on TV business
  Social TV/2nd Screen
  Mobile

Three levels of program sites
1.  Fully auto-generated based on ERP-system
2.  Header picture change, promoboxes and news
3.  The Voice of Finland

Editors
  Nelonen's producers
  Production companies
  News editors
  Radio hosts
  Trainees
  PR-people

Drupal
  Performance
  Permission handling

Kiitos!

Jonne Laakso
jonne.laakso@nelonenmedia.fi

@jontte

DESIGNING AND
DEVELOPING
MULTILANGUAGE
WEBSITES

Designing and developing
multilanguage websites
  The internet is a global tool
  Keep this in mind from the very beginning
  A single content management solution can host
multiple websites (or a group of websites)
  They do not come off the assembly line
  Common components can be utilized
  Differences in the same product across regions

Opportunities
  Save time and money
  Keep a consistent feel
  Efficient reuse of technical elements, ideas,
campaigns, content…
  Developing internal competence
  Improving Search Engine visibility through cross
linking

Challenges
  Design by committee
  Communication
  Avoid wasted efforts
  Ongoing development
  One size does not fit all
  The solution needs to be modular and maintainable
  For some functionalities (e.g. forums) use a separate
product
  Who is in control internally?

Pre-project phase
  Sufficient planning before the RFQ
  Reduces surprises
  Use consultants if you need to
  Define and communicate the project goal internally
  We are making a public site
  Bulgaria has this password protected section where…
  Map legacy features
  Use your local resources to map the current features
  Prioritize language versions and features

Design phase
  Defining the content structure
  Less is more
  Which data is common to all languages?
  Features
  Which features do we have and which sites will use
them?
  What are the priorities?
  Domain strategy
  How will the sites be structured?
  Prepare early for country domains

Implementation phase
  Technical development
  Make features independent
  Assume everything should be translatable
  Content production & entry
  Prioritize the main language
  Translations
  Sourcing this internally or externally?
  Provide worst cases early on
  Prepare for life after initial deployment

Maintenance phase
  Once the project is complete, the work begins
  Minor changes
  Subprojects
  Keeping things under control and everyone happy
  Change management is key
  Deploying updates
  Who is responsible?
  Who can order changes?
  Surprising costs

Monikielisten sivustojen toteutus:
Case CSR-kompassi

Linda Piirto 1.6.2012

Case CSR-kompassi
•  What s CSR?
•  Why do we bother?
•  What s the CSR-kompassi?
•  Why does the site need to be bilingual?
•  Offering a bilingual service by using Drupal done a) the hard way or
b) the easy way
•  Thanks!

Designing and developing challenging Drupal sites

More Related Content

Similar to Designing and developing challenging Drupal sites (20)

More from Exove (20)

Recently uploaded (20)

Designing and developing challenging Drupal sites