This post will explore Ambaum’s thoughts, best practices and lessons learned from multiple Magento to Shopify Plus migrations. Going through a re-platform event is challenging and to do it right you must be skilled at: project management, programming, design, data migrations, integrations, theme development and much more. This article will focus specifically on our lessons learned during the data migration portion of Magento to Shopify Plus migrations.
Before we walk through the details of our data migration experiences let’s explore the macro trends that are driving merchants to adopt Shopify Plus.
The Old Paradigm
Since the beginning of the 20th century manufacturers and brands have sold their products to customers through brick and mortar stores. Only in the late 1990’s was there a shift in how customers bought good and online ordering through the Internet became a real way to move your products. Initially the greatest new channel for manufacturers and brands came through Amazon as these companies leveraged Amazon’s established customer base. In 2020, Amazon now accounts for almost 38% of all online sales in the US so it continues to make sense for manufacturers and brands to sell on Amazon.
The old paradigm is a multi-step process for manufacturers to get their products in the hands of customers. A manufacturer sells to a wholesaler then a distributor, then to a retailer and finally into the hands of a customer. Amazon came along and became the online distributor for many manufacturers, but Amazon extracted a heavy toll of approximately 15% (or more) on products sold through their site.
Even with Amazon’s high take rate of ~15%, manufacturers could still make money, but Amazon was extracting a heavy “tax” on everything sold through their site. In addition, brands that sold through Amazon are unable to fully own their customer data and market to customers because Amazon really owns the customer. Brands on Amazon get access to millions of customers, but the brands remain renters, Amazon is the owner in that relationship. Amazon’s high take rate wasn’t sustainable for manufacturers and brands so they went in search of a new model.
The New Paradigm
Manufacturers and brands rely on Amazon to sell a lot of their products, but the trade-off is a heavy reduction in margins. The answer for most brands and manufacturers is to sell their own products direct to customers through the channel they can fully own – their website. This isn’t an earth-shattering revelation, but it’s an important distinction that has fueled the rise of ecommerce platforms in the last 15 years.
Ecommerce platforms end up in two distinct camps: on premise (Magento, WooCommerce, etc) or cloud-based platforms (Shopify, etc). If you want to learn more about on premise vs cloud based platforms, check out this post from our blog. On premise solutions like Magento, were the very first ecommerce software choices for brands, but over the last 3-5 years cloud-based solutions like Shopify Plus have added enough features that have now surpassed many of the incumbent platforms in the ecommerce space.
Magento to Shopify Plus
With so many great new features being added to Shopify Plus each year and Shopify Unite, it’s driving more merchants to consider migrating off their old ecommerce software. The most popular platform we see moving Shopify Plus is companies moving off of Magento. Businesses are moving off Magento for the following reasons:
- Tired of paying high hosting costs
- Frustrated with the large cost of general maintenance support fees/time spent by developers
- Unable to easily implement and rapidly iterate on digital marketing ideas
- Loss of control over day to day site updates, general pain with making even small changes on Magento
Once the decision has been made to move to Shopify Plus, merchants want to know how the following data can get safely, securely and accurately moved over:
- Customer records
- Products
- Historical orders
- Product reviews
- Blogs posts
- 301 redirects
The rest of this post if going to focus on our lessons learned form moving over customer, product and order data from the merchant’s systems into Shopify Plus.
Magento to Shopify Plus Data Migration Example
Data migrations can be messy business. The migration must be planned out correctly, the data needs to be mapped from the old system to Shopify Plus and you need a way to test and handle exceptions in the process. After performing more than 10 migrations in the last 2 years, we have a pretty good idea of best practices as well as the things to watch out for during a data migration process.
We recently moved data into Shopify Plus from Magento and multiple legacy systems and data sources which presented data in vastly different formats. This client had three systems we needed to import data from:
- Magento: ~125,000 customer records
- Retail Point of Sale System: ~80,000 customer records
- Mail Order Software: ~70,000 customer records
Each of the customer records was supposed to contain the following fields: customer name, phone number, email, however many of the records were duplicate or missing data. Many of the records only had a name and in order to get them into Shopify Plus we had to create dummy email addresses and if there was no phone number, we left that blank.
In the case where we had no email record, we would take the first and last name of the customer and combine with @noemail.com to create a dummy record. Because there were so many generic names such as “John Smith” we would encounter a problem with non-unique records. If there were multiple “John Smith” records, we ended up having to use their phone number + @noemail.com for their email record.
Things to Consider Prior to your Shopify Plus Data Migration
- Practice proper data hygiene – there is a direct correlation with the hygiene and quality of data going in with the data integrity post-import. The GIGO acronym applies here: garbage in, garbage out
- Phone numbers can live in different places in Shopify – when importing phone number into Shopify, they can place those records at different levels, depending on how you import the data.
- Import via Excelify or CSV – the customer record will end up in the “Customer overview” field
- Import via API – the customer record will end up in the “Default address” section
- Pre-clean your email addresses – make sure they are correct format and email domains are real
- Pre-clean your phone numbers – ensure there is a proper phone format, pick a phone format such as: 12063456789, +12063456789, or 2063456789 and use this format for all phone numbers
- Determine the source of truth – If you are importing from multiple data sources, first and foremost determine which data set is the master and import that last (since Shopify will overwrite previous records). If you are only doing a Magento to Shopify data import with no other systems, this won’t be an issue.
- Shopify will use different customer data fields to create a unique customer ID – If you have incomplete customer records Shopify checks the customer data fields (in this order) to find the:
- Shopify ID
- Email Address
- Phone Number
For example, if your new data contains a Shopify ID that already exists in your Shopify dataset, Shopify will overwrite the new customer data on the old record. If your data is missing a Shopify ID, your unique customer email address will trigger Shopify to create a new unique customer ID. If your data is missing a Shopify ID and missing an email address, your customer phone number will trigger Shopify to create your unique Shopify ID.
- Customers should be imported before orders – If you import order data before customer data, Shopify will look at that order data and create a Shopify ID so that the order data is associated with a customer. If you then go in and import new customer data you could end up with data integrity issues because you didn’t follow the proper order of operations.
- Blank fields in a CSV import will overwrite existing data in Shopify – When you are importing new customer records, if your customer ID matches an ID already in Shopify and your import is missing the address field, your customer record in Shopify will no longer contain address information, it will be blank.
- Shopify checks email address formatting – Customer and order records will not import if the email address is either:
- An improperly formatted email address such as: danny.companyx.com
- Not a registered domain name. Shopify is most likely taking each domain name and comparing it to a master list of domain names that have been officially registered.
General Data Migration Best Practices
- Data formats – the data you plan to import should be in one of these formats: 1) JSON, (2) CSV, (3) XLSX, (4) XML (but this is a last resort- XML can be hard to work with sometimes)
- If you anticipate that scripting will be necessary, then JSON is probably the best option for exporting data
- There are online tools available for converting file formats, here is one we like to use: CSV to JSON file conversions
- CSV imports are faster than the Shopify API – the Shopify API imposes a rate limit of roughly 2 records per second, so large datasets can take days or even weeks to import into Shopify via the API
- Utilize Shopify metafields and tags to deal with custom data fields – For example, a merchant might have a custom field for BOGO sales, so in Shopify a tag can be created and then with some code you can look for the tag and then display a sale message on Collection and Product pages to the customer.
- Leverage migration apps – Excelify is a good one that we have used and had success with, but keep in mind that apps can’t do everything. For example, Excelify is good for importing WordPress Blog Posts into Shopify but it’s not good for importing WooCommerce Orders. The Excelify developers have built the app to automatically map the blog post fields to Shopify but the app will not automatically map the order fields. Store owners can use migration tools to Migrate from Magento to Shopify Plus (including Magento 1, 2, CE, EE, Shopify, Shopify Plus) from Litextension – a company specialized in shopping cart data migration.
- Migrate a test batch of data and then start analyzing
- Compare Shopify data to source data and start figuring out what needs to be fixed. For example, maybe the “cost per item” showed up in the “compare at price” field or maybe all product reviews are missing their images. You’ll need to troubleshoot those types of things. Issues might need to be addressed by a custom script, an app, or some configuration in the export or import.
- Keep in mind that every e-commerce platform has different fields available to shop owners and Shopify might not have a good solution for migrating all of the data. There will be some situations where you need to get creative and find a way to not only migrate the data but how to use it on the frontend. For example, one client had some data that was being used to link customers to specific magazines. We dumped all of the data into a Shopify metafield as a JSON object and then wrote some code to find the link that was needed on the frontend. It turned out to be a good solution because it provided a single source for the client to maintain the data and it was fairly easy to access the data from the frontend.
Project Management and Data Dependency Considerations
Think about dependencies. Which data is dependent on other data? For example, Orders need to be linked to Customers and Products and Product Reviews need to be linked to Products. So with that in mind, I suggest migrating Products first and then Customers. After that the order doesn’t matter but I think this would be a good order: Products, Customers, Orders, Product Reviews, Blog Posts, Pages, Discounts, and then any custom data such as QuickBooks for example.
After all of the data has been migrated then you need to take care of the redirects because Shopify has different folders structures. For example, WordPress blog URLs look like this: “domain.com/blog-post-name” and Shopify blog URLs look like this: “domain.com/blogs/blog-name/blog-post-name”. The redirects prevent 404s.
Also, keep in mind that while you are migrating data there are still customers shopping and shop owners making updates. So just before launch you will need to re-migrate anything that has been updated, added, or deleted. Such as Products, Customers, Orders, Product Reviews, and Blog Posts. Check with the client to get an idea about what has changed and be sure to work that into the timeline and budget.
Communicate Often to Keep Everyone in Alignment
It’s important to update the client regularly if you are helping perform a data migration. Merchants want to know that progress is being made and they also want to know when something isn’t working as expected. If there is an issue that you need to tell the client about, be tactful and try to present a solution at the same time.
Data can be complicated, and it pays to plan way ahead when doing a data migration. Hopefully these tips we presented in this post can help make your next Shopify Plus data migration just a little smoother!