Migrating to Drupal 8: How to Migrate Your Content and Minimize the Risks

Migrating to Drupal 8
How to Migrate Your Content and Minimize
the Risks
Acquia Webinar, 11-May-2015

The 5 phases of migration
→ Auditing the source site
→ Planning the new data model
→ Prepping for migration
→ Planning the migration
→ Launching the migration

Phase 1: Source site audit
Taxes? Nope, understand your beginnings.

Know your source site
Primary for a smooth migration:
→ What does my site contain? (data structures)
→ What does my site use? (contrib and custom modules)
→ What does my site look like? (contrib or custom
themes)

Simple or Complex
Questions to ask yourself:
→ Are you looking to essentially migrate everything?
→ How complex do you believe your site to be?
→ Do you have time to try a basic and direct migration?
→ Do you have cleanup you wish to perform?

Data structure: bundles, fields, entities
→ Content types
→ Fields (migrate_tools’ migrate-fields-source Drush
commands can help with this)
→ Menus
→ Vocabularies
→ Taxonomy terms
→ Users

Visualize your entities and fields

Features/Modules
→ Advanced search
▪ i.e. Search API + Search API Solr + Acquia Search
→ External Authentication
▪ SimpleSAMLphp, LDAP, OAuth
→ Rebuilding views used in D6 or D7
▪ Manual process
▪ Missing contrib modules?
→ Commonly used modules:
▪ Pathauto, Redirect, Purge, et cetera.
→ Simple Sitemap module to generate Sitemap.xml file

Are the contrib modules required for
your site available yet on D8?
→ Check out Drupal 8 Contrib Porting Tracker
▪ https://www.drupal.org/project/contrib_tracker

More Questions
→ Are the skills available to rewrite custom modules?
→ Are those custom modules still necessary or does D8
introduce better solutions in Core or Contrib?
→ Is there a cutover window?
→ Is new content being added and are site alterations
being made while the site content is being migrated?

Discuss some of the challenges
→ Learning D8
▪ We started assembling module building lessons:
• https://docs.acquia.com/articles/building-drupal-8-modules
→ Find skilled assistance to fill skill gaps
→ Identify impacted stakeholders
▪ Including stakeholders in the launch and post-
launch process
▪ Insuring they are there to test.
▪ Insuring continued buy-in and collaboration

Phase 2: Data model planning
Congrats!
You’ve chosen a complex migration/cleanup

Plan, consult, resolve, goto 10
10 Plan out what content you expect to see
20 Consult with interested parties [time boxed]
30 Resolve any issues in your model
40 goto 10

What’s in a name?
When building it’s important to communicate as much as
possible, as concisely as possible with structure.
→ i.e. Authors: field_article_author became field_authors
→ i.e. Drupal Projects: field_terms_project became
field_projects_drupal
Often there isn’t a single right answer, consistency is key.

Issues we encountered
1. Typos and misspellings in source
2. Merging fields
3. Processing fields

Review Date (date-time)
→ field_date_review
→ field_review_date (content type A)
▪ date
→ field_revew_date (content type B)
▪ date-time
▪ required using hours:minutes + date
▪ possible bug requiring fix?
Trouble merging two similar fields

Simple wins
1. Coordinating the machine names
2. Type and spelling fixes in machine names
3. Opportunity to change data-type

Phase 3 - Migration prep or:
How We Learned to Stop Worrying and Love the Migrate API

A quick overview of contrib modules
Module Description
migrate_upgrade Drush commands for running upgrades from Drupal 6 or 7
to Drupal 8
migrate_tools General-purpose Drush commands and basic UI for
managing migrations
migrate_plus Extends the core migration framework with additional
functionality (e.g. migration from XML and JSON sources)
migrate_ui UI on top of Migration entities in core
migrate_manifest Drush command for running template-based SQL
migrations from a manifest file

Dataset and data model, please.
→ Are we migrating one thousand or one million items?
→ Do you have any data model change?
→ Migrating via the UI is almost never a good solution
▪ Doesn’t allow for custom migrations
▪ Drush scales better
▪ Contrib has powerful and flexible Drush commands

Get started with migration modules
$ drush @site.env en -y migrate_drupal, migrate_upgrade,
migrate_tools
The following extensions will be enabled: migrate_drupal,
migrate_upgrade, migrate_tools, migrate, migrate_plus
→ Use branch 8.x-2.x for all migration modules
→ Get the DB connection settings for the source site
→ Get familiar with the new Drush migration commands
$ drush | grep migrate

Configure your initial migration
$ drush @site.env migrate-upgrade --legacy-db-url=mysql://
{USER}:{PASSWORD}@{HOST}:{PORT}/{DB} --legacy-root={URI} --
configure-only
→ All necessary YAML files should now be exported
→ Audit with Drush Configuration Management commands
$ drush config-list | grep -c migrate
62

Confirm your migration key is set
>>> Drupal::>>> Drupal::state()->get('migrate_drupal_6');
=> [
"key" => "upgrade",
"database" => [
"driver" => "mysql",
"username" => "{USER}",
"password" => "{PASSWORD}",
"host" => "{HOST}",
"port" => "{PORT}",
"database" => "{DB}",
"prefix" => null,
],
]

Run a full migration
$ drush @site.env migrate-upgrade --legacy-db-url=mysql://
{USER}:{PASSWORD}@{HOST}:{PORT}/{DB} --legacy-root={URI}
→ Straightforward migration...but not extremely flexible
→ You probably need control at all levels of the migration
→ Enter the migrate-import Drush command

Create a custom module (it’s easy!)
→ Create your *.info.yml file
→ Add your exported migration files to config/install
→ Enable your module. That’s it.
$ drush @site.env config-export -y sync ; cp
/path/to/config_dir/sync/migrate_plus.migration*
/path/to/custom/module/config/install/

Run a custom migration
$ drush @site.env migrate-import --group=migrate_drupal_6 --
limit=10000 --feedback=1000
$ drush @site.env migrate-import d6_url_alias,d6_block, d6_file,
d6_user,{...} --limit=10000 --feedback=1000
→ Use --limit to limit on the number of items to process
→ Use --feedback to be notified of progress
→ You can even arbitrarily restrict by source plugin

Migration status and messages
$ drush @site.env migrate-status
$ for i in upgrade_d6_node_type upgrade_d6_user {...} ; do drush
@site.env migrate-messages $i ; done
→ Use migrate-status to have an overview of (last)
imported, unprocessed and total number of items, etc.
→ Use migrate-messages to catch migration errors

What about really custom migrations?
→ It still mostly is the Migrate you know and like
▪ prepareRow()
▪ query()
▪ ...
→ Extend default core Migrate source plugins
→ Write your own source Plugins

Sample custom source Plugin
<?php
namespace Drupalmigrate_webinarPluginmigratesource;
use DrupalpathPluginmigratesourced6UrlAlias;
/**
* @MigrateSource(
* id = "webinar_d6_url_alias"
* )
*/
class WebinarUrlAlias extends UrlAlias {
public function query() {
$query = parent::query();
$query->condition('src', 'user/%', 'NOT LIKE');
return $query;
}
}
Core source Plugin
Plugin annotation
Custom logic via the
query() method

Migration tips and tricks
→ The Migration API is still in flux. Check drupal.org!
▪ Major changes in Drupal 8.1.x - Check http:
//mikeryan.name/blog/mikeryan/migration-
update-for-drupal-8-1
→ See a d6 or d7 string in a migration filename?
▪ It’s specific to this Drupal version
▪ If you don’t see anything, it works for both
→ Start backwards with custom migrations
▪ Use a minimalistic set of migration files first
▪ Add the required dependencies as you go

Phase 4: Migration planning & strategy
The devil is in the...

Stages of migration
1. Initial development - Used Dev and Test
2. Working through the migration bugs
3. Prepping prod (check and triple check)
4. Code “almost ready” - test migration manually
5. Switch to continuous content migration (failed)
▪ dev for active migration work
▪ stage for QA
▪ prod for lead up to launch
6. Test until your eyes and fingers bleed (joke, !joke)

Why continuous content migration
→ This allows code fixes to be rolled into place until very
shortly (48 hours) before launch.
→ Why did we fail?
▪ Tokens in content
▪ Most in place but some had to be manually moved
→ Result -> migration was semi-continuous but we
needed longer window for move (one week).
▪ If “no-go called” -> redo the manual migration
▪ Generated pressure

Continuous migration of content
$ ssh -i /home/{USERNAME}/{SSH_KEY} -f -L 1111:{DB_SERVER}:3306
{USER}@{WEB_SERVER} -N
→ Can you afford a hard cutover/downtime?
→ Use SSH tunneling for maximum security
→ https://docs.acquia.com/articles/ssh-tunneling-server-
side-applications

Now’s your chance
Take the migration to D8 as an opportunity to get rid of
bad old habits!
→ PHP filter
→ Full HTML
→ Improve editorial Workflow
▪ Multiversion (must be in place prior to migration)
▪ Workbench Moderation
However, Don’t fall down the rabbit hole!

Phase 5: Launch
Where the rubber meets the road!

Plan for go/no-go decision
→ This is probably one of the hardest processes to plan.
→ Complete buy-in from stakeholders is needed.
→ Clearly record and communicate.
→ Set a reasonable and firm timeline.
→ Set the conditions for why to delay.
→ If possible stick to the timeline when launching.
→ Try to mitigate risk where possible
▪ sometimes this means delaying a launch
▪ sometimes this means going and fixing post launch

Our plan: what worked
→ Content Freeze
→ Window for migration
→ Focusing on content as the priority
→ Iterative development with Minimal Viable Product
▪ identifying key needs
▪ addressing non-critical path issues after launch
→ Replaced a lot of older code just by using D8 core

Our plan: what didn’t work
→ Misaligned expectations
▪ incomplete go/no-go plan
→ Working against a hard deadline (Drupal 6 EOL)
→ Not enough time planned for encountering bugs in
custom migration
→ Missing contrib modules
→ Finding time to contribute to porting modules

Ensure assets consistency
mysql> SELECT COUNT(fid) FROM file_managed;
$ find . ! -path "*styles*" ! -path "*ctools*" ! -path "*css*" !
-path "*js*" ! -path "*xmlsitemap*" -type f | wc -l
$ rsync -avz --progress --exclude=styles --exclude=ctools --
exclude=css --exclude=js --exclude=xmlsitemap -e "ssh -i /home/
{USERNAME}/.ssh/id_rsa" {SOURCE_SERVER}:{SOURCE_PATH} .
→ Not all files are necessarily managed by Drupal
→ Use rsync when the Migrate API can’t help

We hope you learned about...
1. How to audit: bundles, fields, and entities
2. Planning steps: stay, go, and modify
3. Migrate API and contrib migration modules
4. Process: near-continuous migration of content to
minimize downtime
5. Effectively creating, implementing a risk mitigation plan

Migrating to Drupal 8: How to Migrate Your Content and Minimize the Risks

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Migrating to Drupal 8: How to Migrate Your Content and Minimize the Risks (20)

More from Acquia (20)

Recently uploaded (20)

Migrating to Drupal 8: How to Migrate Your Content and Minimize the Risks