Blog

 

Overview

ESAB is a world leader in the production of welding and cutting equipment. Their innovative, world-renowned equipment and solutions are built with the expertise and heritage of a global manufacturing leader.

The ESAB website reflects its global presence and strong corporate values. It is a modern, responsively designed site that showcases their broad product lines, and provides a wealth of information to their loyal customers and prospects. The ESAB website is a large site consisting of over 10,000+ non-product catalog pages with another 40,000+ pages displaying detailed product data, technical specs and purchasing information about their products.

What also makes the site impressive is that it is localized to over 34 different languages and/or regions.

To pull this off, ESAB has a global team of dedicated content editors and translators that manage the web pages and product catalog information daily.

The major components of their website include:

• an enterprise level CMS, that manages all of the non-product information and renders all pages of their website (including all of the product pages)
• a multilingual product information management system that tracks the entire product catalog
• an extensive amount of bespoke development to help manage translation tokens
• a multi-server, redundant web server and database infrastructure for both the Production and Development environments

The Project

As a multi-pronged initiative this project’s main tasks included:

• Development and integration of a caching layer that would improve product catalog performance
• A major version upgrade of the CommonSpot CMS from version 9.0 to 10.5
• Migration from a replicated database environment to a shared & clustered database environment
• Upgrade of underlying back-end stack including OS (Windows), Database (SQL Server), and Application Server (ColdFusion) versions
• Replacement of the de-supported Google Search Appliance

All these tasks were to be executed using a subset of their global team. All while keeping up with the day-to-day responsibilities of a large multi-lingual site.

The Approach

Following good development practices, all of the development was done on Virtualized Machines in a staging environment, which allowed for several iterations of development and testing.

The plan was a 6-step approach:

1. Initial Setup & Upgrade - New virtualized servers would be stood up, and updated versions of the Web Server (IIS), Web Application Server (ColdFusion) and Database servers (SQL Server) were to be installed on these servers. Backups of the CMS file system and databases would then be taken from Production and deployed to the new virtualized servers. Lastly, an upgrade to the latest version of the CMS would be performed, and the custom code for the caching layer would be deployed.
    
2. Manual Testing Phase 1 - This initial migration would then be followed by a period of manual testing of the site and product catalog by a small group of people within the global team. As issues are identified, they would be passed to development for resolution. This iterative cycle of testing and issue resolution would continue until all priority issues were resolved and tested in the staging environment. Along the way, any code or configuration changes would be tracked, so that they could be applied in subsequent migrations.
    
3. Dry Run Upgrade - A second “dry run” migration and upgrade, would then be planned and executed, re-applying the then-current website data into staging along with any code level updates. Because the CMS supported both the current and new versions of the Application server and Database server, only re-upgrading the CMS in this stage would be needed.
    
4. Manual Testing Phase 2 - Again an iterative period of manual testing and issue resolution for the site and product catalog would be performed, until all priority issues were resolved and the team was comfortable with the results.
    
5. Final Upgrade - Lastly the final migration was to be scheduled, and a global content freeze instituted. During this period only critical updates would be made to the website, by updating the content on both Production and Staging. After performing the freeze, a final migration and re-upgrade of the CMS would be performed.
    
6. Final Manual Testing Phase – For the final phase, before making the upgrade live[SM1] , another shorter round of manual testing was planned. Then once complete and satisfied that all things were working correctly, the DNS would be repointed and the staging environment would become the new production environment, and the content freeze lifted.

How the WebsiteUpgradeTester.com Visual Testing Tool was Utilized

The WebsiteUpgradeTester.com tool, which is a new cloud-based, automated visual testing tool, was first introduced to ESAB early in the first phases of the project but was not fully utilized until a beta version was made available during the latter part of the second manual testing phase after the Dry Run Upgrade was performed.

ESAB had diligently performed manual testing over a several month period after the initial and dry-run upgrades using a relatively small team from both IT and Marketing who were intimate with the site.

With the second migration and CMS re-upgrade complete, ESAB’s team was in the middle of the repetitive and iterative testing/issue resolution cycle. ESAB was anxious to get the site deployed into production and very excited to utilize a beta version of the cloud-based WebsiteUpgradeTester.com tool, to help expedite the launch and improve their testing accuracy.

Darrin Elkins, Global Web & Digital Development Manager, said “From our initial conversations and demonstration, we felt that the tool would help us significantly by shortening our testing effort and increasing our testing coverage in ways we could not do with manual testing. With a site our size, there is only so much we could test manually.”

So, the first step was two-fold.

1. The ESAB team made the staging servers available to the cloud-based tool, and
    
2. The WebsiteUpgradeTester.com team imported a CSV file of all the pages to test and then ran a preliminary run against the public facing production site (www.esab.com) detecting all dynamic, interactive and randomized content in the site. Next, this content was mitigated through simple features provided in the tool, so that it would not cause false positives when later visually comparing the pre and post upgraded versions of the site.

With this work done, the site was ready to be run in the tool. Next, several regions covering thousands of pages across several different languages were identified and run through the tool overnight, capturing a screenshot of each page at three (3) screen sizes representative of a mobile, tablet and desktop devices.

Results were positive on several fronts.

1. First, literally overnight the tool was able to provide results across 15,000+ screen captures that over 92% of the pages in the newly upgraded website were not affected by the upgrade. The tool digitally showed that they rendered exactly the same as they did before the upgrade, down to each and every pixel. This reinforced the previous manual testing effort that was done before the tool was introduced, and gave assurance to the ESAB team that the months of testing and issue resolution were successful, and that they were ready to move forward with the final step. 
   
   Robert Sneed, one of the web development team members responsible for helping in the manual testing effort commented “After the first run of the tool, it was very validating to see that the vast majority of the pages rendered identical to the pre-upgrade version. This gave us great peace of mind that there would be no uncaught or lurking issues, something that we were very worried about given the size of our site. The tool gave us incredibly accurate results, catching even the smallest of rendering differences, and allowed us to test areas of the site we couldn’t do manually given our resources and launch time frame.”
    
2. Next, ESAB was able to focus on the 8% of the screen captures where a difference was detected. The tool allowed ESAB’s content team to quickly scroll through side-by-side comparisons of pre and post upgrade screenshots and categorize the types of differences by simply applying a tag to the screenshot. In an hour or so, the content team categorized the issues and were able to pass these off to development for issue resolution, or mitigation via the tool.
   
   In the Dry Run phase where the tool was first used, it exposed several issues that were not found by the previous manual testing efforts, including partial language translation and token resolution issues, visual display errors due to style conflicts, several cases of missing or incorrect content, and even performance and JavaScript issues.
   
   Darrin Elkins said, “The tool identified many issues that would not have not been uncovered with our manual testing efforts or would have taken us a lot more effort to find.”
   
   See the section Examples of What the Tool Found for details and screenshots of some of the actual errors that were caught.
    
3. Because the tool could retest the entire suite of test pages overnight, the iterative cycle of issue resolution could be significantly shortened. An issue that affected dozens of pages could be resolved during the day, and then confirmed resolved the next morning by the content team.
    
4. Now ready to start the Final Upgrade step, with the tool in hand and proven in the previous step, the final step could be performed much more quickly, minimizing the time of the content freeze and reducing the manual testing load on the ESAB team. As soon as the final migration was performed, the CMS re-upgraded and code updates merged, the tool was kicked off again to check thousands of pages across multiple device sizes. Great news. Nothing new was discovered that needed to be fixed. Digitally everything compared pixel by pixel, and the site was ready to go live.

How the Tool Works

The WebsiteUpgradeTester.com tool works in a very straightforward manner. It takes digital snapshots of all your website pages, at various device sizes that you designate, before you upgrade, and then again right after you upgrade. It then digitally compares each screen capture pixel by pixel to see if anything changed.

Any screenshots that digitally compare to be identical, are marked “Same”. These are the pages that you then don’t need to worry about. Their rendering was unaffected by the upgrade.

If the tool detects even the smallest change it will mark it as “Different” and report it to you. You can then quickly browse through all differences, viewing them in one of several modes (side-by-side, splitter, redlined, flicker, individual) and tag them into buckets for resolution.

You can also quickly select and rerun certain individual pages, or pages grouped by tag template or subsite. So, as you resolve issues you can retest them on-the-spot.

In addition to performing visual comparison testing, the tool also detects and records any resource loading or JavaScript errors. These are errors that may cause problems for users interacting with your pages. Similar to the reporting for visual errors, these hidden errors are also tracked and reported to you.

Examples of What the Tool Found

The following are just a few samples of some of the issues that were found during ESAB’s use of the Website Upgrade Tester tool.

• Several of the errors were related to the complex language translation and tokenization features in the site. For example, here it was determined that under load if multiple simultaneous requests were made for different languages, some of the translations on the page may not render correctly
  
  
  And here’s an example where the tokenized content was not replaced properly:
   
• Here’s an example of a product listing page, displayed in the tool’s side-by-side viewer showing the pre-upgrade (‘before’), post-upgrade (‘after’) and ‘red-lined’ screen captures. As you can see the post-upgrade version incorrectly displayed an additional product in its listing, showing three (3) products vs two (2).
  
   
• And an Italian version of a product page where the post-upgrade version is missing a product listing, including its name, picture and description.
  
   

• Here’s a Swedish page where a submenu item beneath the ESAB yellow banner was not rendered correctly post-upgrade, as detailed in the red-lined version.
  
   

• Here’s a Bulgarian version of a page, rendered for an iPhone 6, where the main hero carousel “Robotic Retrofits” was missing due to a CSS media query conflict.
  
   

• Here are a couple of formatting/style issues that were caught in various sections of the site.
  
  Change in margin on a heading.
  
  
  And a CSS issue that affected the color of the active submenu in the gray menu. The pre-upgrade color was yellow while the post-upgrade color was white.
  
   

• Here’s another Bulgarian page where the light blue Call to Action callouts in the right-side column were missing entirely from the post-upgrade page.
  

• And a case on a Chinese page where a server-side error caused the embedded form in the left-side column to not render, again missing from the post upgrade page.
  
   

• The tool also detected an ordering problem in one of the sub-menus on Russian pages like this one.
  

• In addition to the above listed visual problems, the tool also identified several JavaScript and Resource loading issues reported below, as well as one performance related issue.
   

Conclusion

Although the WebsiteUpgradeTester.com tool was not available to use until the latter half of the Dry Run stage, it saved ESAB an estimated three weeks of manual testing time, in just the time that it was used. Additionally it found issues that possibly would not have been found otherwise and provided confidence that all the issues were uncovered and corrected before go-live.

presented the differences found in an easy to view manner allowing for efficient resolution of any problems. The tool revolutionizes how proactive automated testing simplifies and adds significant value to the upgrade process.

Ann Sparén, webmaster and global team lead in Sweden, said of the tool “I thought the tool was great. It was definitely worth running as it helped significantly reduce much of our manual testing load. Looking at the results was fast and easy, allowing us to quickly review differences that were found across major portions of the site in a short period of time. I wish we had been able to use the tool from the start.”

---

WebsiteUpgradeTester.com is the only cloud-based, automated visual testing tool designed to help you efficiently and affordably test your website upgrade. Whether it’s a major version upgrade, service pack or security patch, you can quickly test your entire site, no matter how large, to uncover any lurking issues due to the upgrade.

Don't waste time manually testing your next Website upgrade. Visit WebsiteUpgradeTester.com and sign up for a free trial.