Case Study: A Complex SharePoint Migration

Online Conference
June 17th and 18th 2015
WWW.SPBIZCONF.COM
Case Study:
A Complex On-Premises SharePoint Migration

WWW.SPBIZCONF.COM
Matthew J. Bailey
MCT, MCSE, Independent Contractor
Email : sharepointmatthew@gmail.com
Twitter : @matthewjbailey1
Website : http://www.matthewjbailey.com
LinkedIn :
http://www.linkedin.com/in/matthewjbailey1
I consider myself a “SharePoint All-Rounder”. My job tasks have varied and
included Administration, Development, Training, Analysis, UAT and Project
Management. My job changes often but it keeps things interesting!
Currently, I am a MCT (Microsoft Certified Trainer) & MCSE (Microsoft
Certified Systems Engineer) in SharePoint. I have worked with SharePoint for
almost seven years. I don’t like to use the word “expert” but I have a fair
amount of knowledge with the technology and currently use it daily in my
career.
If I don’t know an answer to one of your questions, I will try to find it out or
point you in the right direction!

WWW.SPBIZCONF.COM
Before I Begin - Thanks To…
Although I worked hundreds of hours on this
project, it absolutely would not have been
successful without the help of others too.
Special thanks go out to:

WWW.SPBIZCONF.COM
Before I Begin - Thanks To…
• The engineers at our new hosting provider
• The internal network & infrastructure teams at the
company
• My previous supervisor who helped me push the project
through the "red tape" and get the project approved
• Trevor Seward (MVP) / Independent consultant who
helped troubleshoot claims issues.

WWW.SPBIZCONF.COM
Agenda & My Role in Project
1. Business Case
2.Project Plan Creation
3.Technical Migration Phases
a. Phase 1
b. Phase 2
c. Phase 3
4.Final Testing & Launch
5.Case Study Review, Rewards & Lessons Learned
•

WWW.SPBIZCONF.COM
Business Case

WWW.SPBIZCONF.COM
Business Case Summary
• The current infrastructure setup is not working well
• Staff changes, budgets, organizational restructuring and
insufficient resources have occurred
• Dedicated staff and centralized hosting location are
needed
• Marketing (whom controls the site) needs full
transparency into the platform

WWW.SPBIZCONF.COM
Business Case Summary (cont.)
In other words, things aren't working the way they are now
so let's grab all the servers and move them to our own host
where we can control everything 

WWW.SPBIZCONF.COM
Existing SharePoint Environment
A previously developed & architected SharePoint
environment consisting of:
– 3 SharePoint farms (authoring, staging, production)
– 1 AD domain
– 1 Forest
– 2 AD Domain Controllers
– 1 SQL Server
– 12 SharePoint servers (mixture of WFE & application)
– 3 web applications in each farm
– 18 language packs
– SharePoint 2010 Enterprise SP1, SQL 2008 Enterprise, Windows 2008 R2

WWW.SPBIZCONF.COM
Existing SharePoint Environment (2)
Additional notes:
• The environment has had multiple consulting companies &
employees who no longer work for the company develop
and manage it. It is now being inherited by the Marketing
department and newly hired employees dedicated to
working with these applications (myself being one of these
people).
• There is a development environment , however it was
separate & not part of this project.

WWW.SPBIZCONF.COM
Existing SharePoint Environment (3A)
1. A public/internet facing SharePoint 2010 site in 18 languages with
publishing, several custom features and .wsp files. Functionality includes:
• Content deployment to keep servers in sync between environments,
shared managed metadata service
• Custom built workflows for multi-author publishing, many custom web
parts to provide advanced lookups and data pulls similar to that of a
CQWP, custom page layouts for visual effects, slideshows and video
• An Active Directory trust to enable single sign on capabilities for
content authors
• Customized web parts and page layouts to accommodate country
specific requirements for sub-sites written in non-English languages

WWW.SPBIZCONF.COM
Existing SharePoint Environment (3B)
2. An extranet for partners of our organization. This includes:
• Required authentication via FBA (Forms Based
Authentication), LDAP, Secure Token Service & claims
• Extremely detailed levels of item specific permissions for
content based on login identity, custom developed
management portal to allow creation and management of
partner accounts
• A pricing task that imports information from another
computer system and is modified and imported into the
extranet

WWW.SPBIZCONF.COM
Existing SharePoint Environment (3C)
3. A blogging platform consisting of:
• A few different blogs
• Based on the CKS (Community Kit SharePoint) CodePlex
project

WWW.SPBIZCONF.COM
Issues with Existing Environment (a)
Overcome Current Hosting Provider's Constraints
• Production environment has single points of failure (i.e., only one SQL server)
• Staging environment is rarely used yet has excessive resources that could be applied to
production or eliminated to reduce cost
• The current hosting provider appears to have only one SharePoint specialist on duty and
has restricted hours of 9am to 6PM EST. This has limited us to when we can deploy
changes and must be done during business hours affecting user’s ability to perform work.

WWW.SPBIZCONF.COM
Issues with Existing Environment (b)
Overcome Current Hosting Provider's Constraints (continued)
• No advanced monitoring software for hardware issues and site outages (they do have
basic). This has caused hardware and web page issues taking more time then needed to
resolve.
• Security lockdown (beyond reasonable security concern lock outs) that prohibit us from
viewing server OS based reports (i.e., Windows Event Log, CPU usage, etc.)
• Inability to install troubleshooting software on the current hosting provider's servers
without paying a fee for evaluation of the software first.

WWW.SPBIZCONF.COM
Issues with Existing Environment (c)
Resolve internal support issues
• Security lockdowns that cause excessive time to troubleshoot and deploy changes
• Lack of standards on what is and is not allowed to be completed and of transparency on
why errors, issues and limitations occur
Resolve non-unified environments issues
• Keeping Term Store in sync due to firewall / network issues & allowing users to make Term
Store changes
• Quickly obtaining copies of databases for backup and restores due to size and time
needed to upload to a place that is shared so we may obtain the db copy
• Time delays in troubleshooting Content Deployment issues easily as we have to make
formal requests to 2 different environments to obtain logs to investigate issues

WWW.SPBIZCONF.COM
New Vendor Requirements (1)
Infrastructure
• The provider must be able to offer a multi layer disaster recovery and site outage
replacement plan including solutions for short term outages, mid term outages and total
losses of website and data.
• The provider must provide an advanced outage process and notification system that
includes both hardware, software and web server features to shorten the period of
outages.
• The new provider must provide monitoring services to suggest alterations and correct
performance issues on the servers.

WWW.SPBIZCONF.COM
Services & Support / ROI
• The provider must allow code based changes and deployments to be made during non
working hours to avoid constant interruptions to employee’s work and provide the ability
for us to deploy solutions more than once a month due to massive backlog of projects.
• The provider must have multiple layers and a larger team of expertise, experience &
consultation services that have worked with other mid to large sized customized
corporate SharePoint environments.
• The new selection must reduce the amount of time it takes to document, plan,
troubleshoot and deploy even basic changes. It should allow for less people to be
involved in the process full process of build to deploy.

WWW.SPBIZCONF.COM
SLA
• The provider must have a minimum of a 99% uptime guarantee.
• The provider must have a hardware outage replacement guarantee of less than 4 hours.
• The provider must have competent & conscious staffing available 24/7 in case issues
arise.

WWW.SPBIZCONF.COM
Architectural Changes
Infrastructure
Although the temptation was great due to many challenges with the current architecture, we chose
not to attempt to redesign and re-architect the system at the same time of migrating it due to the
high level of risk involved. However, we have made some improvements to create a more stable,
better performing & fault tolerant environment. This includes:
• Current environment only has one SQL server per farm which creates a single point of
failure for environment. The new environment will have a virtualized SQL cluster with an
active/passive failover sequence. *NOTE: It is not recommended to virtualize your SQL
servers unless you have extremely powerful hardware to run them on, in this case we did.
• Current environment did not have a dedicated, hardware based load balancer. New
environment will.
• New environment will have faster servers, more space, more CPUs and more memory.
• Our staging environment was used very little and we decided to lessen the number of
SharePoint servers on this farm to give our production environment more resources.

WWW.SPBIZCONF.COM
Project Plan CreationProject Plan Creation

WWW.SPBIZCONF.COM
Project Plan Creation
Our project plan needs to be extremely detailed and take into consideration:
• Migrating Data (all SharePoint databases and other files on OS)
• Installing Software
– Exact version match on SQL, SP, Windows
– Language packs
– AD trust
• Setting up Hardware (deciding which servers will be on which Hypervisors so
that the servers are evenly balanced across the 4 physical boxes)
• Our company's business needs, exceptions and all stakeholders involved with
the project as well as their availability and potential conflicts
• Emergency backup plans

WWW.SPBIZCONF.COM
Project Plan Creation - Hardware
• New DAS (Direct-attached storage)
• 4 dedicated physical services (that will be virtualized to allow many
different servers to run), 64GB RAM each, 2 x 8 core CPUs each (16 cores
each total)
• F5 load balancer
• Cisco firewall
• Cisco VPN

WWW.SPBIZCONF.COM
Project Plan Creation – Software
Since this will be a "one for one" migration and our expected outcome is to
have the same environment functioning in a different location, we want to
replicate the software versions identically to avoid any potential issues.
These include knowing and confirming:
• Exact version of Windows Server OS
• Exact features & roles enabled on each Windows Server
• Version of SQL Server
• Exact version numbers of SharePoint
• All language packs needed

WWW.SPBIZCONF.COM
Project Plan Creation – Data/Code Migration
• Partner accounts in AD for extranet
• OS level files (i.e., DLL, .resx, web.config, etc.)
• Extracted .wsp files / features
• Content databases from SQL for all three web applications
• Custom Scheduled Tasks on Windows Server we created
• Managed metadata databases from SQL
• URL rewrite rules from IIS
Additional guidance and best practices regarding content migration from Microsoft:
http://technet.microsoft.com/en-us/library/cc287899(v=office.14).aspx

WWW.SPBIZCONF.COM
Project Plan Creation – People & Company
• Find out when everyone will need time off to schedule around them
• Find out special events at your company (i.e., end of month lock ups for IT or end of quarter
earnings releases freeze periods for us, audit times, etc. , holidays, users in other countries so
their holidays are different than ours, conferences employees have to attend)

WWW.SPBIZCONF.COM
Final Meeting Stakeholders Signoff
• We want to review and get everyone's agreement to our detailed project plan.
During our conference call with all stakeholders present, we reviewed the
spreadsheet below to confirm all timelines, tasks, risks, teams responsible for
each task and any other details.
Of course, even with the best of planning, things didn't go as hoped, but that's why
we are here today! What fun would a project be that didn't have any issues (and
why would we have jobs? grin).

WWW.SPBIZCONF.COM
Emergency Backup Plan
• Our existing web host will still be active. Although the prices have dramatically
increased, not switching the DNS records over to our new hosting provider is an
option if we cannot get the environment working.
• Since we pursued a "phased" approach, we at least knew that some parts of the
website were working farm before the cutoff date. In this case, should it have
happened, we could have gone live with a "mostly working" site if it was
needed. Fortunately we did not encounter this situation.

WWW.SPBIZCONF.COM
Technical Migration Phases
and now the fun really begins…
Technical Migration
Phases

WWW.SPBIZCONF.COM
Technical Migration Phase Begins
Items to remember as we review the technical phase:
• Each company functions differently. Limited funds prevented expertise assistance being hired on to
ensure a smoother transition.
• A lot of learning on my part "happened along the way". Although there was a great deal of due
diligence done prior to migration beginning, there will always be the "I don't know what I don't know"
aspect of a project.
• I am human, some items may seem like an obvious red flag or could have had a simpler solution, but
everyone did the best they could with the budget and resources we had available.
• There will be a lot of information condensed into this presentation. We are covering 5 months of
technical work in a one hour review. It’s ok to come back to these slides later and review them!
• Since our previous host was managed at a far more restrictive manner, we did not have as much
insight as we thought in regards to trying to replicate some of the settings needed to configure at the
new hosting provider.
• Eventually, the project was completed successfully, it just had some bumps along the way and was still
very much worth the effort of going through.

WWW.SPBIZCONF.COM
Technical Migration - Phase 1 (a)
• 3 week estimate from hosting provider was given to setup hardware environment & install
SharePoint per our specific needs for our "private cloud" or dedicated/managed
environment.
• Create web applications on SharePoint servers Deploy solution files we have as of that
date Use PS to extract all .wsp files from current environment
• Copy any .dll files or .resx files over & match up Timer Jobs - MANUAL PROCESS
• Attach / restore content databases (which may be a bit old), we have empty shell dbs now,
mount db, add service accounts to security & roles and Term Store files.
• We will bring over entire Term Store now and sync with the task instead of bringing over
the other part of the Term Store later for extranet (using PowerShell script)
• Run Test-SPContentDatabase to review potential errors

WWW.SPBIZCONF.COM
Technical Migration - Phase 1 (b)
• Create service applications for web application (done as part of install by hosting provider)
• Compare Central Admin between 2 servers (existing and new)
• Compare Central Administration setup and config files for each web applications using text
comparison software
• Configure email on all servers & configure SMTP role in Windows Server
• Pass along software license numbers to new host (our company has an Enterprise
Licensing Agreement, licenses were already paid for)
• Configure State Service
• Create, start & restore managed metadata service
• Create the service using the same name as previous installation and restore MMS
database
• Configure search in Central Admin (using SharePoint 2010 so much of this was manual
work)

WWW.SPBIZCONF.COM
Technical Migration - Phase 2
• Create VPN Tunnel from our company to new hosting provider
Delays starting to happen from the hardware replacement not having arrived and
configured at the hosting provider yet.
Lesson Learned: I contacted our account manager at the new hosting company to
inform them our management was growing concerned of the delays in our
environment setup (although, in my opinion there wasn't much on their part they
could do and they did not bill us for anything until our environment was up and
running).

WWW.SPBIZCONF.COM
• Work with Infrastructure & new hosting provider to establish One way
trust between our company and new host
After our hardware and delays started to happen and a couple of weeks had passed, it
was finally discovered that the name of the Active Directory domain & NetBIOS name
of, which was asked what to be by host, was the same name as our internal company
Active Directory domain name. It is not possible create a domain trust between two AD
domains that have the same name. It had been suggested that the hosting provider
just create 2 new AD VMs and rejoin them to the rest of the farm but the hosting
provider had seen previous issues with SharePoint not working after doing this, thus
the decision was made a completely new install of all software on all servers was
needed. All software had to be reinstalled from scratch on the entire environment with
a different AD domain / NetBIOS name at the hosting provider. Also, as with most
companies, the "blame game" started to occur.
!

WWW.SPBIZCONF.COM
Lesson Learned: Be willing to absorb a lot of other's frustration and keep pursuing
forward with the project. Don't disregard their opinions or comments but don't allow it
to become so negative that it just creates more issues with the project instead of
accepting responsibility and moving on.
If you are ever going to create a domain trust between AD servers, make sure they
never have the same NetBIOS name!

WWW.SPBIZCONF.COM
• Create detailed plan for AD trust
In response to the issue above and the realization that we were starting to have
challenges establishing an AD trust.
Lesson Learned: I needed to create a detailed plan & document trail to
help resolve the issue. I also contacted our account manager at the hosting
company to arrange a call with their engineers and ours to create a formal
plan on how we were going to clean up the situation and establish new
time lines. There should have also been a discussion prior to the project
starting about this with our Infrastructure team in more depth as in this
situation the previous host had performed more configuration than our
new host and there was more "as we go along" learning that created
delays.

WWW.SPBIZCONF.COM
A Dose of Reality…

WWW.SPBIZCONF.COM
• Obtain final copy of blog db from current hosting provider. Install CKS (codeplex blog project), restore content
database
• New hosting provider creates a secondary AD tree for Partners of new hosting provider who use the extranet
for FBA (our extranet was created to use FBA with LDAP and a separate OU in the AD domain at the hosting
provider. This kept our extranet users/external partners login identities completely separate from our own
internal company's AD domains and servers)
• Content freeze on our blogs for users (selected the web application of least impact to use as our first test)
• Run full search crawls & pretest all new hosting provider environments (will have some old data on it but most
functionality should be there)
• Install PDF Foxit filter, add file type in Central Admin (2010 task required)
• Import existing search suggestions / type-ahead from previous environment (New-
SPEnterpriseSearchLanguageResourcePhrase)
• Set Up term store sync process (we used a PowerShell script ran on a scheduled task that always ran before
the scheduled content deployment job of the day that exported our MMS databases from one server to
another. Content deployment jobs that run to another server where a term from the Term Store does not
exist will create the term with a new and different ID which will create issues the next time a job is ran)

WWW.SPBIZCONF.COM
• Ask current hosting provide for any final usage reports we may use (as we are not reimporting usage log databases to new
environment) - Just in case anyone wanted to know statistics from searches done on our website for analytics purposes as
after the migration was complete this data would no longer be available. Search query reports help us to determine when
we should create Best Bets to assist users (now referred to as promoted results in SharePoint 2013)
• CONTENT FREEZE - PUBLICWEBSITE.com & Term Store NO CHANGES - turn off CD, turn SQL databases for
PUBLICWEBSITE.com to read-only at current hosting provider / Shut off CD (may be turned back on after freeze for
emergencies maintain dual content entry)
• Obtain final set of databases, Term Store export and any other data from current hosting provider for PUBLICWEBSITE.com
(1st web application)
• Attach last/most updated content databases from authoring to new hosting provider servers
• Import final Term Store file to new hosting provider servers, re-verify web.config is the same at new hosting provider as it
was at current hosting provider
• Take VM snapshots for backup purposes (now that we achieved a working state that has relative purpose we want to be
able to revert back to it if needed)
• Make sure firewall rules are setup similar to current hosting provider (i.e. we had to make a special rule for one of our
customers in Canada to access our site)
• Configure Anonymous authentication, other zones and security similar to existing environment
• Make sure Central Admin has diagnostic logging and reporting functioning & analytics reports

WWW.SPBIZCONF.COM
• Configure Blobcache, ULS and IIS logging to data drive (http://technet.microsoft.com/en-
us/library/cc770229(v=office.14).aspx)
• Make sure all .resx files are copied / match on ALL SharePoint servers (since we 18 different languages and
some custom alterations this was a needed step)
• Configure content deployment (add jobs, server names and extend timeout settings in config file and Central
Admin)
• Set up & test content deployment jobs
• Pretest mobile compatibility (we had a few mobile views that had been built with jQuery to be tested)
• Give infrastructure's team a list of DNS names that will need to be modified
Realization that the project timeline is in jeopardy and that even with everyone working extra time to
try and catch up, it is starting to appear that the current timelines may not be met
Meeting and agreement reached with stakeholders that the current timelines are not going to be
met. Extending project another month or two

WWW.SPBIZCONF.COM
Due to continued challenges, unavailability of resources and delays, our project plan has been
changed to launch all 3 web applications at the same time at our new host which differed from
our phased approach of launching each web application separately (in hopes of having less
issues). We were able to test everything at the new hosting provider using hosts file tricks and
such before we changed our DNS records.
Lesson Learned: "Agile" definitely has a new meaning to me at this point…lol. Being able to adapt to
change, especially when you cannot control the exterior factors affecting your project, is a necessary
skill.
As project timelines have changed, reorganizing availability with everyone involved for testing and
support needed to recompleted as well. Employee burn out and other work commitments start to
pull stakeholder's full attention away from project.

WWW.SPBIZCONF.COM
Realization that our internal networking IP schemes and how our new host had configured their IP schemes were quite
different. Our internal network relies heavily on NAT IPs. The new host had assigned all public IP addresses to our hardware
and needed now to have NAT IP addressing configured
Lesson Learned: We should have included an IP mapping plan and requirements for IP security architecture section to our project
plan.

WWW.SPBIZCONF.COM
• Create service applications for web application including STS (secure token service). Ensure the token in
the web.config is the same from previous environment.
Employee on Infrastructure team leaves company. VPN tunnel & AD trust project comes to a
halt. Time to pull another resource from the Infrastructure team and bring them up to speed on
the project is needed.
• Configure search (not sure if each box has be done individually or not)
Our extranet has SSL, this creates a unique configuration for our search crawl to use a different
port. The hosting provider recommends we use a different port for this crawl or it will not work.
Our internal IT team wasn't in agreement, however we proceeded with suggestion and search
worked (mostly)
Lessons Learned: Research special issues that are going to occur with SharePoint in SSL situations.
Understand, once again, IT architecture is an "art" not a "science" and that some people may
recommending doing things differently than others.

WWW.SPBIZCONF.COM
Final Testing & Launch
Final Testing &
Launch

WWW.SPBIZCONF.COM
UAT - Final Testing & Launch
• UAT Scripts for all content authors must be completed
• Technical review of Central Admin and other "daily IT duties" performed by administrator (myself) & developer
• Pretest all functionality on the extranet (email sending, downloads, new users, logging in as customer or employee)
• Notify Infrastructure's team and make appointment for DNS changeover and testing next week
• Pretest load balancing with scripts
• Infrastructure's team does security test on extranet (2 days)
• FINAL STEP: Have IP addresses (mail servers, RDP) and DNS routing changed for domains and other related URLs to
propogate new IP around the globe
After Launch
• Ask user's to do a bit of retesting the following Monday when they come back into work, remove content freeze and
allow users to return to their normal work
• The extranet launches at new hosting provider
• Make sure Google Analytics is still functioning as normal for web properties (I don't believe there should be impact but
would like to verify)
• Take snapshots for backup reasons (as long you have the disk space! although some systems will retain delta changes of
snapshots and this can slow down your server)

WWW.SPBIZCONF.COM
Case Study Review,
Rewards & Lessons LearnedCase Study Review,
Rewards &
Lessons Learned

WWW.SPBIZCONF.COM
FINALLY!!!! – The Payoff
• Between $55k – $100k annual savings from previous host’s billing compared with new host
• 40X Increase GB Increase in storage
• 95% reduction in ability to deploy urgent code & server (72 hour minimum mandated wait vs.
approximately 3 hours) (sometimes longer depending on approvals from hosting company
needed)
• 80% decrease in time to turn around medium priority requests
• 3% increase in business hours uptime
• Change in ability to solve P1 (critical outage) issues
• 15 minute maximum of notice of soft or hard failures on server - Better monitoring scraping
• Database failover no single point of failure
• 8.2 % reduction in Average Page Load Times
• 34.9% reduction in Server Response Times
•

WWW.SPBIZCONF.COM
Lessons Learned
What did we do well?
• The shared project tracking list with daily updates for all stakeholders to view gave everyone a sense that the project
was not being neglected, problems were being worked on and no one was left in the dark about issues that were
occurring.
• Even though our project plan wasn't perfect, the easy to read color coded team assignment format made it easy for
each team to understand their tasks.
What could we have done better / didn't work so well?
• The project went over budget and time by approximately 60 days. This incurred an extra expense of approximately
$12k that was needed to be paid to the current host that had not been expected.
• Understanding the depth of your own infrastructure and things you don't know. My learning experience of how
network configuration at every company can be so different and come from years of multiple persons working at that
company who have come and gone leaves room for many surprises.
• Don't be afraid to ask people involved doing tasks you are not familiar with to create a document on how they will be
doing it. This could spark a conversation far sooner about needing help or a lack of knowledge rather than having it
unfold as things progress.

WWW.SPBIZCONF.COM
Lessons Learned - Continued
What have we learned ?
• When many people are involved, and responsibilities shift, go out of your way to get the same person and continue to
persist on them working on that task.
• Infrastructure is more of an art than a science.
• Never try to create an AD trust between to environments with the same NetBIOS name.
• If we had known that creation of the AD trust would have caused so many challenges and such an amount of delay,
we probably would have stopped to do an actual evaluation on the usability of the site running 100% independently
from our internal organization. Our company does not have a very large number of users that need to actually
authenticate to the environment since it is mostly used as an informational source as an internet & extranet site.
• Unexpected delays from hardware and networking that is beyond your control can always pop-up.
• NAT IP addressing & VPN tunnels are quite a bit to learn for someone unfamiliar with this.
• Always prepare to be agile for the “What you don’t know you don’t know” such as employee turnover, company
changes, staffing and technical challenges. You can never have too many backup plans or prepare too much (IMHO).

WWW.SPBIZCONF.COM
Please fill in my session feedback form available
from the ‘Session Resources’ tab
on my session window.

Case Study: A Complex SharePoint Migration

More Related Content

What's hot (20)

Viewers also liked (12)

Similar to Case Study: A Complex SharePoint Migration (20)

More from Matthew J. Bailey , MCT (9)

Recently uploaded (20)

Case Study: A Complex SharePoint Migration

Editor's Notes