How to efficiently test a major website upgrade and platform change of their public facing website with 50,000+ pages in 34 localized regions with a small team.
ESAB leveraged a new cloud-based visual testing tool designed specifically for website upgrades to automate and streamline their testing efforts. Using the tool saved an estimated three weeks of manual testing time, increased accuracy significantly, and gave the team peace of mind that there were no lurking surprises.
ESAB is a world leader in the production of welding and cutting equipment. Their innovative, world-renowned equipment and solutions are built with the expertise and heritage of a global manufacturing leader.
The ESAB website reflects its global presence and strong corporate values. It is a modern, responsively designed site that showcases their broad product lines, and provides a wealth of information to their loyal customers and prospects. The ESAB website is a large site consisting of over 10,000+ non-product catalog pages with another 40,000+ pages displaying detailed product data, technical specs and purchasing information about their products.
What also makes the site impressive is that it is localized to over 34 different languages and/or regions.
To pull this off, ESAB has a global team of dedicated content editors and translators that manage the web pages and product catalog information daily.
The major components of their website include:
As a multi-pronged initiative this project’s main tasks included:
All these tasks were to be executed using a subset of their global team. All while keeping up with the day-to-day responsibilities of a large multi-lingual site.
Following good development practices, all of the development was done on Virtualized Machines in a staging environment, which allowed for several iterations of development and testing.
The plan was a 6-step approach:
The WebsiteUpgradeTester.com tool, which is a new cloud-based, automated visual testing tool, was first introduced to ESAB early in the first phases of the project but was not fully utilized until a beta version was made available during the latter part of the second manual testing phase after the Dry Run Upgrade was performed.
ESAB had diligently performed manual testing over a several month period after the initial and dry-run upgrades using a relatively small team from both IT and Marketing who were intimate with the site.
With the second migration and CMS re-upgrade complete, ESAB’s team was in the middle of the repetitive and iterative testing/issue resolution cycle. ESAB was anxious to get the site deployed into production and very excited to utilize a beta version of the cloud-based WebsiteUpgradeTester.com tool, to help expedite the launch and improve their testing accuracy.
Darrin Elkins, Global Web & Digital Development Manager, said “From our initial conversations and demonstration, we felt that the tool would help us significantly by shortening our testing effort and increasing our testing coverage in ways we could not do with manual testing. With a site our size, there is only so much we could test manually.”
So, the first step was two-fold.
With this work done, the site was ready to be run in the tool. Next, several regions covering thousands of pages across several different languages were identified and run through the tool overnight, capturing a screenshot of each page at three (3) screen sizes representative of a mobile, tablet and desktop devices.
Results were positive on several fronts.
The WebsiteUpgradeTester.com tool works in a very straightforward manner. It takes digital snapshots of all your website pages, at various device sizes that you designate, before you upgrade, and then again right after you upgrade. It then digitally compares each screen capture pixel by pixel to see if anything changed.
Any screenshots that digitally compare to be identical, are marked “Same”. These are the pages that you then don’t need to worry about. Their rendering was unaffected by the upgrade.
If the tool detects even the smallest change it will mark it as “Different” and report it to you. You can then quickly browse through all differences, viewing them in one of several modes (side-by-side, splitter, redlined, flicker, individual) and tag them into buckets for resolution.
You can also quickly select and rerun certain individual pages, or pages grouped by tag template or subsite. So, as you resolve issues you can retest them on-the-spot.
The following are just a few samples of some of the issues that were found during ESAB’s use of the Website Upgrade Tester tool.
Here’s a Swedish page where a submenu item beneath the ESAB yellow banner was not rendered correctly post-upgrade, as detailed in the red-lined version.
Here’s a Bulgarian version of a page, rendered for an iPhone 6, where the main hero carousel “Robotic Retrofits” was missing due to a CSS media query conflict.
Here are a couple of formatting/style issues that were caught in various sections of the site.
Change in margin on a heading.
And a CSS issue that affected the color of the active submenu in the gray menu. The pre-upgrade color was yellow while the post-upgrade color was white.
Here’s another Bulgarian page where the light blue Call to Action callouts in the right-side column were missing entirely from the post-upgrade page.
And a case on a Chinese page where a server-side error caused the embedded form in the left-side column to not render, again missing from the post upgrade page.
The tool also detected an ordering problem in one of the sub-menus on Russian pages like this one.
Although the WebsiteUpgradeTester.com tool was not available to use until the latter half of the Dry Run stage, it saved ESAB an estimated three weeks of manual testing time, in just the time that it was used. Additionally it found issues that possibly would not have been found otherwise and provided confidence that all the issues were uncovered and corrected before go-live.
presented the differences found in an easy to view manner allowing for efficient resolution of any problems. The tool revolutionizes how proactive automated testing simplifies and adds significant value to the upgrade process.
Ann Sparén, webmaster and global team lead in Sweden, said of the tool “I thought the tool was great. It was definitely worth running as it helped significantly reduce much of our manual testing load. Looking at the results was fast and easy, allowing us to quickly review differences that were found across major portions of the site in a short period of time. I wish we had been able to use the tool from the start.”
WebsiteUpgradeTester.com is the only cloud-based, automated visual testing tool designed to help you efficiently and affordably test your website upgrade. Whether it’s a major version upgrade, service pack or security patch, you can quickly test your entire site, no matter how large, to uncover any lurking issues due to the upgrade.
Don't waste time manually testing your next Website upgrade. Visit WebsiteUpgradeTester.com and sign up for a free trial.
In this Ultimate Guide to Upgrading and Patching Your Website, we’ll walk you through all of the terms, concepts, ideas, and best practices around upgrading a medium to large scale website. We’ll impart our experiences of helping customers perform and test their website upgrades over the past 20 years, so that you can upgrade faster, more efficiently, more often and with better results. Read More