Drupal8 migrate

Drupal 8 Migrate
12/14/20
17
1

Who are we?
Full service provider of
branding, marketing, website
design & development, and
strategic communication
services
Proven methodology for
positioning companies to
scale and succeed in an
increasingly digital
environment
Trusted agency partner to
dozens of recognized
associations and
corporations

My Background
・Credentials: B.S. Computer Science
from Virginia Tech
・Position: Chief Technology Officer @
Bluetext
・Interests: IoT, Gaming, Snowboarding,
Hiking, Fishing, etc…

Why are we here?
・Overview of Drupal 8 Migrate
・Migrate Source & Destinations
・Migrate Processors
・Examples!
12/14/20
17
10

Overview of Drupal 8 Migrate
・Complete rewrite & moved into core in D8
・OOTB support for D6 and D7 to D8
・Nodes, Users, Comments, Profiles, Taxonomy
・Configuration & Content
・Support for custom source and destination classes
・Several processors for working with data
12/14/20
17
12

Drupal 8 Configuration Migration
・Completely new feature in Drupal 8
・Will migrate data structures such as node types and
vocabularies
・Support is limited in the contrib space
12/14/20
17
13
Image from Drupal.org

Drupal 8 Content Migration
・Core modules such as User, Node, Comment, etc… come with
d6 and d7 migration templates
・Migrate sources and destinations are extensible
・Similar to Drupal 7, typically build a migration for each entity
type/bundle combination
12/14/20
17
14

D8 Migrate Module Overview
・Provides powerful API for all migrations
・Provides extensible object-oriented base classes and interfaces
for migration plugin components
・source & destination plugins
・process plugins
・config migration mappings
・Provides configuration entity types to migrate configuration
12/14/20
17
15

Configuration Files
・ Config files provide the blueprint for the migration
・ Two types of migration config files:
・ migrate_plus.migrate_group.<name>.yml
・ migrate_plus.migrate.<name>.yml
・ Config file overview
・ id: Unique system name of the migration (same as the last part of the file
“d6_node”)
・ source: Defines the source plugin that is used (d6_node or custom blog_node)
・ process: Mapping of fields and processors used in the migration
・ destination: Define where this content is being migrated to
・ dependencies: Other migrations that need to run before this one
12/14/20
17
16

Contributed Space
・Contributed space still provides significant value
・Migrate Plus: The Migrate Plus project provides extensions to
core migration framework functionality, as well as examples.
・Migrate Tools: The Migrate Tools module provides tools for
running and managing Drupal 8 migrations.
12/14/20
17
17

Anatomy of a Migration
・Migration Group
・Migration Definition
・Migration Source
・Migration Destination
・Migration Mappings
12/14/20
17
19

Migration Group
・ Similar to hook_migrate_api in
Drupal 7
・ Defines a group of migration
classes
・ id – unique identifier
・ shared_configuration – Defines
shared configuration between all
migration classes that are part of
this group. **Example: Setting
the source database to use
・ dependencies – Sets the
dependencies for this set of
migration classes to function
12/14/20
17
20
id: btwp
label: Custom migrations
description: Custom data migrations
from BT WP.
shared_configuration:
source:
key: legacy
dependencies:
enforced:
module:
- demo_migrate

Migration File - Definition
・ Similar to a Migration class in
Drupal 7
・ Defines the metadata for the
migration in Drupal 8
・ id should match the file name
・id: blog_post
・filename:
migrate_plus.migration.blog_
post.yml
12/14/20
17
21
id: blog_post
label: Blog Post Migration.
migration_group: btwp
migration_tags:
- blog
- node
migration_dependencies:
optional:
- blog_tags
dependencies:
enforced:
module:
- demo_migrate

Migration File - Source
・Defines the source plugin
class for the migration
・Tells the migration where the
data is coming from
・Different sources allow
different configurations
・Allows definition of constants
to be used in field mappings
12/14/20
17
22
id: blog_post
migration_tags:
- blog
- node
source:
plugin: wp_post
constants:
format: rich_text
optional:
- blog_tags
dependencies:
enforced:
module:
- demo_migrate

Migration File - Destination
・Defines the destination
plugin class for the
migration
・default_bundle: Defines
the bundle we want the
entity:node plugin to map
data to
12/14/20
17
23
id: blog_post
migration_tags:
- blog
- node
source:
plugin: wp_post
destination:
plugin: 'entity:node'
default_bundle: blog
optional:
- blog_tags
dependencies:
enforced:
module:
- demo_migrate

Migration File – Field Mapping
・Defines field mappings and
any processing that should
be performed on source
data before being mapped
to destination
・Default process plugin used
is ‘get’
12/14/20
17
24
id: blog_post
migration_tags:
- blog
- node
source:
plugin: wp_post
constants:
format: rich_text
destination:
process:
title: post_title
sticky: 0
promote: 0
uid: 1
'body/value': post_content
'body/format': constants/format
optional:
- blog_tags

Migration File – Processors
・Utilize processors to
manipulate data before it
gets mapped
・Several processors available
OOTB
・Very powerful
12/14/20
17
25
id: blog_post
migration_tags:
- blog
- node
source:
plugin: wp_post
constants:
format: rich_text
destination:
process:
title: post_title
sticky: 0
promote: 0
uid: 1
'body/value':
plugin: wp_vc_parser
source: post_content
optional:
- blog_tags

Using Process Plugins
・ Processor plugins provide flexibility in working with data being
migrated
・iterator– Provides an iterator to loop through multiple values in a
source field
・default_value– Provides the ability to set a default value to a field
・entity_generate – Generates entities for reference fields if they don’t
exist
・entity_lookup – Looks up entities migrated through another migration
・ Very powerful and can be combined together
・Ex. Iterate over all of my items and perform an entity_lookup to find the
nid mapping to the new system
12/14/20
17
26

Customizing Your Migrations
12/14/20
17
27

Customizing Your Migrations
・99% of Enterprise migrations will require customizations
・Custom requirements and mappings likely required
・Have to deal with the “innovative” way that the previous
developer built the source site
12/14/20
17
28

Scenario: WordPress -> Drupal
・Migrating a website from WordPress to Drupal 8
・Website uses Visual Composer to manage content
・Website consists of both Posts and Pages
12/14/20
17
30

MigrationGroup: BTWP
・id: btwp
・shared_config: legacy (legacy
is the source WP database
configured in settings.php
that all of the migrations will
read from)
・dependencies: modules that
are required for these
migrations to run
12/14/20
17
31
id: btwp
description: Custom data migrations
from BT WP.
source:
key: legacy
dependencies:
enforced:
module:
- demo_migrate

WPPost.php - MigrateSourcePlugin
・ Custom MigrateSourcePlugin
that handles reading from the
wp_posts table in WordPress
・ Defines MigrateSource as
wp_post
・ This is used in .yml
migration mappings
・ Extends SqlBase source class
since our datasource is MySQL
12/14/20
17
32
<?php
/**
* @file
* Contains Drupaldemo_migratePluginmigratesourcewp_post.
*/
namespace Drupaldemo_migratePluginmigratesource;
use DrupalmigrateRow;
use DrupalmigratePluginmigratesourceSqlBase;
/**
* Source for CSV files.
*
* @MigrateSource(
* id = "wp_post"
* )
*/
class WPPost extends SqlBase {

12/14/20
17
33
・Basic function definitions:
・query() – Defines the base query to pull the data.
・fields() – Defines the fields that get pulled from the source
database.
・getIds() – Defines the ID mapping for source value.
・baseFields() – Helper function to define base fields

・query() - Defines query to
load all data from the
wp_posts table to be
processed.
・fields() – Calls the
baseFields() method to
load fields from
wp_posts table for
mapping purposes.
12/14/20
17
34
/**
* {@inheritdoc}
*/
public function query() {
return $this->select('wp_posts', 'wpp')
->fields('wpp', array_keys($this-
>baseFields()))
->condition('wpp.ID', 0, '>');
}
/**
* {@inheritdoc}
*/
public function fields() {
$fields = $this->baseFields();
return $fields;
}

・getIds() – Defines ID field
and settings so the
migration knows how to
map the data in your
source to the data in your
destination.
・prepareRow() –
Implements prepareRow
and calls parent.
12/14/20
17
35
/**
* {@inheritdoc}
*/
public function getIds() {
return [
'ID' => [
'type' => 'integer',
'alias' => 'wpp',
],
];
}
/**
* {@inheritdoc}
*/
public function prepareRow(Row $row) {
return parent::prepareRow($row);
}

・baseFields() – Defines the
base fields from the
wp_posts table so that
they can be used in the
migrations.
12/14/20
17
36
/**
* Returns the user base fields to be migrated.
*
* @return array
* Associative array having field name as key and description as value.
*/
protected function baseFields() {
$fields = [
'ID' => $this->t('Post ID'),
'post_author' => $this->t('Post Author'),
'post_date' => $this->t('Post Date'),
'post_date_gmt' => $this->t('Post Date GMT'),
'post_content' => $this->t('Post Content'),
'post_title' => $this->t('Post Title'),
'post_excerpt' => $this->t('Post Excerpt'),
'post_status' => $this->t('Post Status'),
'comment_status' => $this->t('Comment Status'),
'ping_status' => $this->t('Ping Status'),
'post_password' => $this->t('Post Password'),
'post_name' => $this->t('Post Name'),
'post_type' => $this->t('Post Type'),
];
return $fields;
}

So How Does This Actually Work?
12/14/20
17
37

Migration File – Blog Post
・group – Defines the source
for the migrations.
・Tells all migrations in this
group to utilize the
“legacy” database
12/14/20
17
38
migrate_plus.migration.blog_post.yml
id: blog_post
migrate_plus.migration_group.btwp.yml
id: btwp
description: Custom data migrations from BT WP.
source:
key: legacy

・source_plugin – Maps to the
source WPPost.php
MigrateSource that we
created based on the ID
defined.
12/14/20
17
39
source:
plugin: wp_post
constants:
format: rich_text
WPPost.php
/**
* Source for SQL files.
*
* @MigrateSource(
* id = "wp_post"
* )
*/

・process – Provides the field
mappings for Drupal fields :
WP fields
・Default process plugin: get
12/14/20
17
40
process:
title: post_title
sticky: 0
promote: 0
uid: 1
'body/value': post_content
WPPost.php
protected function baseFields() {
$fields = [
'ID' => $this->t('Post ID'),
'post_author' => $this->t('Post Author'),
'post_date' => $this->t('Post Date'),
'post_content' => $this->t('Post Content'),
'post_title' => $this->t('Post Title'),
'post_excerpt' => $this->t('Post Excerpt'),
'post_status' => $this->t('Post Status'),
'post_name' => $this->t('Post Name'),
'post_type' => $this->t('Post Type'),
];

Results
・Well that didn’t work as well as we hoped!
・Each CMS has its own unique quirks and way of storing
information
・In our WP build, we happen to be using Visual Composer
which creates a majority of the site with shortcodes
・So what now?
12/14/20
17
42

MigrateProcessPlugin
・Create a custom MigrateProcess
・Parse through the WordPress Visual Composer shortcodes
・(Simple) Replace shortcodes with markup
・(Advanced) Repurpose shortcodes into paragraphs
・ Sorry, this is for another time!
12/14/20
17
43

・@MigrateProcessPlugin –
Defines the ID of the plugin
that will be used in .yml file
・Class name (WPVC) must
match the filename
WPVC.php
・Extends ProcessPluginBase
12/14/20
17
44
<?php
namespace Drupaldemo_migratePluginmigrateprocess;
use DrupalmigrateProcessPluginBase;
use DrupalmigrateMigrateExecutableInterface;
use DrupalmigrateRow;
/**
* Handles Visual Composer Markup.
* @see
DrupalmigratePluginMigrateProcessInterface
* @MigrateProcessPlugin(
* id = "wp_vc_parser"
* )
*/
class WPVC extends ProcessPluginBase {

・Define regex expressions to
be used in the parsing of
content
・Implements transform() –
required method that does
the actual processing of the
source value
12/14/20
17
45
class WPVC extends ProcessPluginBase {
// Regex101 reference:
https://regex101.com/r/pJ7lO1
const SHORTOCODE_REGEXP = "/(?P<shortcode>(?:(?:s?[))(?P<name>[w-
]{3,})(?:s(?P<attrs>[wd,s="'-
+#%!~`&.s:/?|]+))?(?:])(?:(?P<content>[wd,!@#$%^&*()s
="'-+&.s:/?|<>]+)(?:[/[w-_]+]))?)/u";
// Regex101 reference: https://regex101.com/r/sZ7wP0
const ATTRIBUTE_REGEXP =
"/(?<name>S+)=["']?(?P<value>(?:.(?!["']?s+(?:S+)=|[>"']))+.)["']?/u";
/**
* {@inheritdoc}
*/
public function transform($value,
MigrateExecutableInterface $migrate_executable, Row
$row, $destination_property) {
if ($value) {
$value = $this->replaceShortcodes($value);
}
else {
throw new MigrateException(sprintf('%s is not
an array', var_export($value, TRUE)));
}
}

・Helper method to convert all
shortcodes into an array and
replace strings
・Could get very complex
depending on the setup of
your site
12/14/20
17
46
/**
* Replace all shortcodes w/ markup.
* @param $content
* @return mixed
*/
protected function replaceShortcodes($content){
// String Replacement
$shortcode_arr = $this->parse_shortcodes($content);
// For every shortcode
foreach ($shortcode_arr as $key => $value) {
switch ($value['name']) {
case 'vc_row':
$this->replaceVCRow($value);
break;
case 'vc_column':
$this->replaceVCColumn($value);
break;
default:
echo $key . ' is not mapped yet.';
break;
}
}
//Now replace all closing tags.
$content = str_replace('[/vc_row]', '</div>', $content);
$content = str_replace('[/vc_col]', '</div>', $content);
return $content;
}

Current State of Things
・Drupal 8.4 has several issues/dependencies
・Requires Drush 9.0
・Must be running migrate_tools 4.x with this patch for Drush 9.0
to work
・Drush 9.0 changed some migrate commands slightly:
• drush mim (migrate:import) instead of drush mi (migrate-import)
12/14/20
17
48

Useful Code Snippet
・Must delete
configuration file and
reimport to get
changes to .yml files
・Typically handle this
through hook_uninstall
・ drush pm-uninstall <mymodule>
・ drush en <mymodule>
12/14/20
17
49
/**
* Implements hook_uninstall().
* Removes stale migration configs during uninstall.
*/
function demo_migrate_uninstall() {
$query = db_select('config', 'c')
->fields('c', array('name'))
->condition('name', db_like('migrate_plus.') .
'%', 'LIKE')
->execute();
$config_names = $query->fetchAll();
// Delete each config using configFactory.
foreach ($config_names as $config_name) {
Drupal::configFactory()-
>getEditable($config_name->name)->delete();
}
}

Check us out:
www.bluetext.com

Drupal 8 References
・Upgrading from Drupal 6 or 7 to Drupal 8
・Upgrade using Drush
・Known Issues when migrating from Drupal 6 or 7 to Drupal 8
・Migrate Process Overview (Processor Plugins)
・Webinar – Migrating to Drupal 8
・Carlyle Example Migrate Project
5
2

Drupal8 migrate

More Related Content

What's hot (20)

Similar to Drupal8 migrate (20)

Recently uploaded (20)

Drupal8 migrate

Editor's Notes