Mage Titans USA 2016 M2 deployment

Editor's Notes

• #5: Deployment processes can be divided into two types: “Production deployment”. And “development deployment”, which is also deployment as a developer needs to deploy a Magento instance in order to start working with it. And, as we can see, the requirements are different for different types of deployment: Production is focused on final result: the working store. And so it must be secure, reliable and perform well. Usually, in order to reach all these and to reach specifics of the project, flexible deployment settings are necessary. Form the other hand, Development deployment must be simple, so that the developer could focus on the deployment itself instead of playing with deployment settings. And also it needs developer and debugging tools to simplify the development process.
• #6: Each of these types can be divided more into sub-types. Deployment process for production highly depends on the project’s needs and abilities. It can be: Self-hosted, which gives the most flexibility Cloud hosted, which gives flexibility in resources, but still highly depends on the hosting features from flexibility perspective Shared hosting, which gives a simple solution for simple installations. Here I’d recommend to work with hosting providers that provide in-built Magento installation. This guarantee that you have all resources necessary for your Magento store to run. On other hand, development procedures are also different basing on the project’s purposes: This can be an extension development Or contributing development for Magento
• #7: Basing on your deployment type, how do you obtain the code? Here are our recommendations: In most cases you should use Composer-based installation, where you run a Composer command and download whole project in “vendor/” directory. Only in case you’re a contributing developer and are working on a contribution to the Magento core code, please, use Git repository. This is the only case, when it should be used. There are also zip/gzip/bzip archives for your convenience. It is an alternative to the Composer-based installation. Just that the Composer commands are pre-run by Magento release tools and the archives are packaged with downloaded code in them. The result should be the same as if Composer is used. This would make sense to use if you just want to try Magento and don’t want to install additional tools or/and if you want to deploy Magento on a shared hosting with no access to console where you’d run the composer commands. Another alternative solution would be to use ready Vagrant or Docker boxes with pre-installed Magento. Or even w/o Magento in it, but with environment ready.
• #8: There are three application modes in Magento 2: “developer”, “production” and “default”. Developer mode allows to see errors right on the page or in web API request. Also it makes the application auto-generate necessary static view files. Production mode is faster and safer, as it hides all error in log files and is intended for usage with pre-generated files. Default mode is a compromise between developer and production modes. It still hides errors, but it performs auto-generation of static view files, when necessary. It’s recommended to use either developer or production mode.
• #10: There are two recommended ways of setting up file permissions for Magento files. A “single-user setup” is a simplified one. It’s when you have a single user you use for any actions with the files – for the web-server, for console access. It may be the only option with shared hosting. It’s simple in usage because there can’t be any conflicts in access to the files, as all files are created and accessed by the same user. A more flexible approach to permissions management is to use two different users for accessing files. In this case, you’d have one user for console access and another user for web-server. It gives you ability to setup permissions in more flexible way.
• #11: The console user should be the owner of the files (in most cases) and should have full (or highest) permissions to the files. Use this user for running console commands and crons. The web user should have limited access: read-access to most of the files and write access to a limited number of files that the Magento web application really needs. Both users should be in the same primary group. So, basically, the web user gets access to the files basing on the group.
• #12: Let’s see which permissions are necessary for the web user in case of two-users setup. This is a setup for the group permissions. Follow the rule “deny-allow”, or in our case “read-write”. First, give read-only permissions to, basically, everything.
• #13: And then give write permissions only to the files and folders that are really necessary in your case. For production, here is a list of paths that must be writable.
• #15: All Magento packages, including extensions and Magento core packages, are distributed via Marketplace. In order to download the packages for your installation, it is necessary to have Marketplace keys. The keys are not the same as Marketplace username and password. The reason for this differentiation is that username and password allow you manage your entire Marketplace account (make purchases, change your information, etc.). And Marketplace keys allow only one thing – download already purchased packages. Also, you can revoke the keys at any time and replace them with the new ones, if the old ones are compromised.
• #16: When you deploy Magento, the keys are usually stored for future usages (e.g., in case of upgrade or installation of new extensions). The keys, actually, can be stored in two different places depending on the actions you perform: If you run Composer commands (e.g., “composer create-project”), the keys you specify are stored in the Composer’s home folder, which is usually under your OS user’s folder If you run Magento commands or specify the keys in the Setup Wizard, the keys are stored in a special Composer folder under Magento project. The reason for existence and usage of these two places is a potentially different version of Composers used in different processes. Magento actions (console commands and Setup Wizard) use a specific version, embedded in the Magento project. Composer used as a separate tool and installed in your system may be of a totally different versions. And if these versions store data in different format, the results can be unexpected in case auth.json file is stored by one version of Composer and then loaded by another.
• #17: Upgrade is another big topic. There are different ways to upgrade Magento store. And it highly depends on how you deployed it. I’ll go through a few options, which should be general enough to be applied for other specific cases.
• #18: Composer-based upgrade via console. It is applicable in case you have a Composer-based or an archive-based deployment AND you have access to console and want to use console for the upgrade process. It’s the preferable procedure. So… If it’s necessary to lock the store, enable maintenance mode Uprgade dependency on a new version of Magento product using “composer require” command with “--no-update” option composer upgrade. It’s necessary to run these two commands instead of only “require” command because in this case all dependencies will be updated vs. only specified one, which helps avoid compatibility issues Upgrade Magento DB: magento setup:upgrade By default the upgrade command cleans generated files. If you’re in production mode or want to speed-up the store, generate new files Disable maintenance mode
• #19: Another console-based approach is for Git-based installation. It is the way contributing developers would go with. Assuming you have a remote named “magento” for the original Magento repository Fetch magento Merge magento/develop into your branch Run “composer install” to update dependencies. It’s important to run “composer install”, not “composer update” – it guarantees that all developers will work with the same set of dependencies, even if some of them are not strictly defined Upgrade DB: magento setup:upgrade Now your Magento installation should be on the new version.
• #20: And one more way to upgrade is to use Setup Wizard. It is applicable in case you don’t have access to the command line. Go to Admin Panel > System > Tools > Web Setup Wizard. And there select System Upgrade. Select version Optionally select other components to upgrade. It is recommended to do this though, as some installed components may be incompatible with new version of Magento, so it’s better to upgrade them to the new, compatible versions. Then just go through the steps
• #21: Now, I’d like to focus for a moment on how you’d upgrade real production. Though, in general the result should be the same, you’d like to reduce downtime as much as possible. In this case it’s recommended to have a staging server additionally to the production one. The staging server should have a copy of your production - same code base. DB and user data may differ, but the application must be installed. Then, to perform upgrade: Do the upgrade on the staging server: Upgrade the code (Composer-based precedure via console) Clean generated files Upgrade DB Generate DI and static view files Then you need to move the updated code to production: Put production in maintenance code Replace the code with the one from staging. Here you may use Git and make ”git pull” or you may copy a zip archive to the production and now unzip it. But, independenctly on the approach, ensuew that you REPLACE the files, not COPYING them ON TOP. You must be sure that files that are not present in the new version, are removed. Just be sure that you don’t replace user data and environment configuration. Upgrade DB with --keep-generated option (as you already have files generated for the new version, you do not want them to be removed) Disable maintenance mode With this approach, the only downtime is caused by two actions: replacing the code (with can be very quick depending on your approach to doing this; e.g., you could pre-populate the new Magento folder and then just rename it to the “current” on in an instant) DB upgrade. This highly depends on the size of your DB and on efficiency of upgrade scripts of the extensions. If there are extension developers here, a request to you would be to test upgrades of your extensions on big DB, and optimize the scripts if possible.
• #22: About files generation. During deployment and upgrade it is necessary to generate DI and static view files for your production. As I just described, in order to reduce downtime on production, it is recommended to generate these files on staging. Here are few insights about this process: Use “setup:di:compile” command for generating DI files Use “setup:static-content:deploy” command for generating static view files. A few notes about this command: It requires installed Magento instance to run. But it’s not necessary to have an identical DB, as on production. What must be identical is to have same themes on staging (Magento will register them in DB automatically, so the DB will have same set of themes and so will perform files generation compatible with your production). What may be identical, but may be bypassed: same stores with same locales selected. The command automatically identifies set of locales used in the application to deploy files for these locales. But you can bypass this requirement by manually specifying necessary languages as command’s arguments. Just ensure that you don’t forget any language (locale) used on production. In most cases you don’t use default Magento frontend theme on production. To speedup the generation process you can specify only themes you use on production, so files will be generated for these themes only. Use --theme option. An important moment for both commands is to have same set of enabled modules on both staging and production. Both commands may generate different result, if the sets are different.