Princeton University Press to Drupal 8: Migration case study by Evolving Web
- 1. Our expertise. Your digital DNA | evolvingweb.ca | @evolvingweb DRUPAL 8 MIGRATION CASE STUDY
- 2. ALEX DERGACHEV • Co-founder at Evolving Web • 10 years! • @dergachev on twitter + github • Co-organizer of Drupal Montréal meetups
- 3. MATT CORKS • Technical project manager at Evolving Web • Using Drupal since 2004 • mvc on Drupal.org, IRC, slack • @mvc447 on Twitter
- 4. • 9 years of great Drupal dev. • Enterprise level projects • Extensive training program • Office in Old Montréal • We’re hiring!
- 5. OUR WORK
- 6. .ca .ca OUR APPROACH Training & Knowledge Transfer MVP, Iterative approach Expertise through the technology stack. Collaboration tools: Redmine/JIRA, Basecamp, Google Suite, Slack Great design and theming Large scalable infrastructure Multilingual CMS Search UI Content migration and sync Custom Drupal development OUR EXPERTISE
- 7. Princeton University Press is an independent publisher with close connections, both formal and informal, to Princeton University. As such it has overlapping responsibilities to the University, the academic community, and the reading public. Our fundamental mission is to disseminate scholarship (through print and digital media) both within academia and to society at large.
- 8. • 9,000 published titles • 15,000 pages in Einstein Papers • 3,000 quote pages • 2,000 sample chapters • 2,000 ToCs pages • 600 bird section pages • 6,000 PDFs • 6,000 blog pages • 18,000 content pages, catalogues, sales pages, videos, and other ancillary materials. CURRENT SITE
- 9. • Minisites • Wordpress Blog • phpList newsletter • Search (CGI + Google Books) • Two external shopping carts (US/UK) • ONIX XML feed + images • (distributor FTPs) • videos + audios, moving to vimeo MOVING PARTS
- 10. WHAT WE DID • Analysis of the SQL Server DB, VB scripts, and current website • Content audit - identify pages to drop • Deploy Drupal CMS to organize the content • Updated responsive design • Nightly synchronization of book data from SQL Server • Deploy to Pantheon
- 13. LEGACY VB SCRIPTS • 30-50 scripts in MS Access • Take days to run • Code duplicated, out-of-date • “Special” books • (edited in Dreamweaver) • “Ann’s Biblio” MsSQL DB • HTML files moved to W: • COMPLEX! (5k-10k Lines of code)
- 14. VB SCRIPT AUDIT
- 15. CURRENT WORKFLOW Static HTML Dynamic HTML Works Dynamic HTML Pages Website PDFs Illustrations Cover Images Biblio SQL Server VB Scripts
- 16. NEW WORKFLOW Biblio SQL Server Dynamic Views Works Basic Pages PDFs Illustrations Static HTML Migrations (sync) CSVs on FTPExport Cover Images Drupal Static Files on FTP One-time import
- 17. BIBLIO QUERIES - GENERATE CSV
- 19. CONTENT CLEAN-UP Category Number of Files Dynamic Pages - Books and Book Lists 20,163 Static Pages to Keep 2,478* Pages to Delete 53,929 Total 76,570
- 21. IMPLEMENTATION
- 22. • Modernized, responsive design • Accessible and good UX • SEO • Stable, secure, maintainable platform • Easy to update web content, WYSIWYG • Images are auto-resized • Clear navigation • Automatic nightly sync BENEFITS OF NEW SITE
- 23. • Existing branding + look-and-feel • Adapted to standard modern responsive template • Changed side menu to consistent drop-down UI • Clean, accessible footer • Search box collapsed HOMEPAGE REFRESH
- 26. • Simplified UI (less in header) • Displays books consistently • Adapts based on the fields available for each book • Integrates with shopping carts, Google Book search • Renders videos + materials • Collapses long sections of text BOOK TEMPLATES
- 27. • Includes migration of ~8,000 books and ~4,000 jpgs* • Process is optimized to prevent downtime/inconsistency on the live site • Migrations can be run manually or automatically • Out-of-print books are removed Biblio SQL Server CSVs on FTP Export to CSV Migrate to website MIGRATING BOOK DATA
- 28. • Includes migration of works, editions, reviews, contributors, etc. • Handle source deletion • Multi-CSV “join” • SFTP + FTP support, with source change detection • Nightly cron via queue API • Fixed encoding • Resolved many issues with legacy data and legacy book logic ONLY CUSTOM MODULE: pup_migrate $ drush migrate-status Group: PUP Biblio (pup_biblio) Status Total Imported Unprocessed pup_math_subjects Idle 30 30 0 pup_textbooks_by_author_redirects Idle 7 7 0 pup_catalog_by_math_subject_order_by_author_redirects Idle 30 30 0 pup_subjects Idle 50 50 0 pup_work_chapters Idle 4241 4242 -1 pup_chapters_by_subject_redirects Idle 50 50 0 pup_contrib_roles Idle 37 52 -15 pup_contribs Idle 10356 10357 -1 pup_editions_ebook Idle 4285 4292 -7 pup_region_restrictions Idle 131 131 0 pup_editions_physical Idle 11386 11407 -25 pup_prizes Idle 3096 3094 0 pup_series Idle 225 225 0 pup_textbook_cats Idle 7 7 0 pup_trans_languages Idle 45 45 0 pup_work_contribs Idle 16058 15941 117 pup_work_covers Idle 7540 7560 -23 pup_work_illustrations Idle 125 125 0 pup_work_interviews Idle 569 357 0 pup_work_interview_by_author_redirects Idle 5 5 0 pup_work_interview_by_title_redirects Idle 5 5 0 pup_work_links_header Idle 125 124 0 pup_work_links_mixed Idle 2872 1677 95 pup_work_reviews Idle 42328 42220 35 pup_works Idle 8219 8231 -12
- 29. • admin_toolbar • context • context_active_trail • devel • entity_reference_revisions • metatag • migrate_source_csv • paragraphs • token • ctools • entity_browser • inline_entity_form • migrate_plus migrate_tools • pathauto • views_slideshow • editor_file • redirect • redis • stage_file_proxy CONTRIB MODULES
- 30. • ISBN • Number of Pages • Price UK/US • Region • Season • Status • Type • Affiliation • Audible • Book Club • Sub-authors • Book ID • Co-publisher • Sub-title • Table of Contents • Cover Caption • Cover Image • Description • Tagline • Title • Need Textbooks • Edition Notes • Title (formatted) • Volume • Primary Authors • Math Subject • Prior Editions • Reviews • Links • Illustrations • Chapters • Prizes • Interviews • Contributors • Subjects • Translation language Biblio SQL Server CSVs on FTP Export to CSV Migrate to website IMPORTED BOOK FIELDS
- 31. • PDF links rewriting (assets) • Merged denormalized records for ebooks and print books • Added publication type for book apps • Encoding problems BOOK DATA CLEANUP
- 32. • Imported most-visited static pages • Can be easily cleaned up using the Drupal website admin interface by client team • Left other lower priority pages can be migrated manually STATIC HTML MIGRATION
- 33. • Lists of books, dynamically updated after Biblio sync • Search pages • In future, can replace some static listings with Drupal views (eg. books published in the last month, have won a prize) DYNAMIC LISTINGS (Views)
- 34. GOING LIVE
- 35. DEPLOYMENT CHECKLIST (1/2) • Deploy on • Set legacy asset domain to assets.press.princeton.edu • Test migrations / cron sync on Pantheon • Ensure uploaded files stored in subdirectories per field • Google Analytics, etc. • Enable/disable extensions: admin_toolbar, devel, … • Ensure that the live site domain is white-listed as trusted host in $settings • Change redirects to use live domain in settings.site.inc
- 36. DEPLOYMENT CHECKLIST (2/2) • Performance audit (frontend + backend) • Enable page cache, CSS/JS aggregation • Broken link checker, Fast 404 • Soft launch: DNS, remove HTTP Auth • Pantheon Launch Checklist: https://pantheon.io/docs/ guides/launch • Setup backups & recovery • Monitoring (Pingdom)
- 37. POST-LAUNCH • Support agreement • Friendly URLs & redirects • Advanced search • Convert static pages to dynamic pages, Webforms • More admin training + UI improvements • SEO enhancements - social media tags, XML sitemap • Annual sale (discounts), shopping cart simplification • Additional homepage content (eg. news in slideshow, events) • Migration to VirtuSales Biblio • Design overhaul
- 39. .ca .ca Alex Dergachev Business lead Jigar Mehta Backend dev Jorge Diaz Front-end dev Matt Wahba Designer Matt Corks Technical PM Suzanne Dergacheva Training PROJECT TEAM Dave Vasilevsky Backend dev Alex Parker QA (Intern)
- 40. CLOSE COLLABORATION • 7 calendar days of onsite meetings in Princeton • 5 demos (in person or Zoom) • Single point of contact (PUP + EW) • Backlog of tasks with estimates for client prioritization • No client-initiated scope creep
- 41. TIMELINE
- 43. SUMMARY / CHALLENGES • Phases: • Scoping, Design, Dev, Migration, Content, Deployment, Maintenance • Gotchas • Migrations in cron, Legacy complexity, Content cleanup, Editor interface • PM • Timeline, Scoping phase, Budget expectations
- 44. .ca .ca