Key Concepts

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.

For simplicity's sake, the WebsiteUpgradeTester.com tool classifies the environment in which you upgrade into two buckets; In-place and Separate Server.

1. Environment 1 is an “In-place Upgrade Environment” where the upgrade is performed on the live Production server.  This environment is not recommended, but is supported by the Website UpgradeTester.com tool.

2. Environment 2 is what we call a “Separate Server Upgrade Environment", where you clone the Production Server to one or more Development Servers and perform the upgrade in Development.  If possible we recommend cloning the Production server to two Development servers, that way there is a static copy of the pre-upgrade version that does not change if you need to make updates to your live site during the upgrade process.  If you only have one Development server, your Production server can be the 'Before' server. Just try to limit changes to this server while you are running the tool.

As previously stated, the In-Place environment is not recommended, but you can still use the tool effectively  in this scenario. However, additional CPU time will be needed, and you need to make sure that you have done a thorough job of handling any dynamic, interactive, randomized or time-sensitive content before upgrading.  More on this later.

One of the significant advantages of Separate Server Upgrade environments is that you can perform one or more dry-runs of your upgrade using the WebsiteUpgradeTester.com tool to quickly test the results.  An example of this is as follows.

• You perform the first dry-run upgrade and the tool catches several issues due to the upgrade that need to be fixed.

• You fix those issues,then try another dry-run upgrade and test that.  

You can iteratively repeat this process until you are comfortable that all errors are resolved.

If you are going to be upgrading 'in-place' on the same server, or on just 1 Development Server, additional care must be taken to perform certain tasks before you upgrade.
It is really important that you identify and mitigate any dynamic, randomized, or interactive content that will cause screenshot comparisons to show differences.

You can specify some or all of the pages on your site to test by creating one or more Page Sets.

A Page Set is simply a listing of URLs. You can have one list of all pages or multiple lists of various pages of your site. What works best is up to you.

However, note that when defining and running a Baseline Run or Comparison Run only one Page Set can be specified.

You can create Page Sets in any of four ways:

1. Manually
2. Import CSV spreadsheet file
3. Specify SiteMap XML file
4. Spider your site

For more information on Page Sets and the Page Set Members page, please watch the following video:

• How to create and manage Page Sets

There are three (3) types of runs you can perform:

• Baseline Run ('before'),
• Comparison Run ('after)
• or Combined Run (both 'before' and 'after').

When you create a Baseline or Combined Run you specify the Page Set to run against, the screen sizes to capture, and optionally a login to use.

For Comparison Runs you simply specify which Baseline Run to run against and an optional login, and it will pick up the Page Set and screen sizes to process.

When the run is started, this tool will navigate to each page specified in the Page Set, for each screen size, and will capture a screenshot image. For Comparison Runs, this tool will also perform a pixel by pixel comparison of the 'before' image against the 'after' image.

For more information on each run type, please watch the following video:

• Baseline, Comparison and Combined Runs

A Baseline Run is the first run in the process of comparing your existing site with the upgraded version.

When you specify a Baseline Run, you specify the Page Set, one or more device sizes, the root URL and optionally a Login to use.  Based on these parameters, the baseline run will navigate to each page in the specified Page Set, for each device size specified, capturing a screenshot.  This is the baseline, or 'before', screen shot that will be used when comparing against the 'after' screenshot to determine what if anything has changed on your site.

It is critical that your Baseline Run has the proper 'before' screenshots, prior to upgrading, as you cannot go back and regenerate any screen shots once the upgrade has run.  As such you need to make sure that your Baseline Run accounts for and mitigates any known dynamic, randomize or interactive content that might falsely show the page as being different.

After the initial Baseline Run, you should perform a Comparison Run against the same version of the site, to determine any dynamic, randomized, or interactive content,  And then apply appropriate Dynamic Regions as necessary to mitigate any of the know dynamic content, and then rerun those pages.  You should also resolve any errors, and rerun those pages, prior to upgrading.

Only after you are confident in your Baseline Run snapshots should you perform the upgrade.

For more information on Baseline Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

To test what's changed on your pages after you perform your upgrade, you will kick off one or more 'Comparison Runs'.  A Comparison Run will create a comparison, or 'after', snapshot for each page/screen size, and will perform a pixel-by-pixel comparison of the 'before' and 'after' screenshot image captures to determine if any changes have occurred.

When creating a Comparison Run, you choose the Baseline Run to compare against, the root URL, and optionally a Login to use.  The same Page Set and device sizes defined for the Baseline Run are used for the Comparison.

As the run proceeds, a comparison screenshot is created for each page in the Page Set and each device size. Then immediately after the screenshot capture, a pixel by pixel comparison is made between the baseline screenshot and the new comparison screenshot. If any differences are found, a 'Difference Image' is created, which highlights in red the changed pixels.

You can create as many Comparison Runs as you need against a Baseline Run, or you can rerun selected pages from an existing Comparison Run.

For more information on Comparison Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

A Combined Run, is a special type of run that runs both the Baseline and Comparison Runs together. It processes the 'before' image capture, 'after' image capture, and image comparison, back-to-back for each page/screen size.

This type of run can be useful to more quickly see how many differences you may have due to dynamic, randomized or interactive content — instead of having to run a full Baseline Run, then Comparison Run to start to see results.

A Combined Run can be run against a 'before' and 'after' server, however, in the case where you are upgrading in-place, it might be useful to run against the same 'before' server to identify dynamic, randomized or interactive content.

For more information on Combined Runs, please watch the following video:

• Baseline, Comparison and Combined Runs

If, after a Baseline or Comparison run, you see issues with any pages, it's easy to rerun them and generate new screenshots.

Here are some of the ways to rerun pages:

1. Rerun selected pages
2. Rerun pages with errors
3. Rerun modified pages
4. Rerun pages by tag
5. Rerun pages by template
6. Rerun all Differences

Most of these options are available by checking the checkbox for one or more items on the results page and then opening the More Actions menu.

Once you have run a Comparison or Combined run you can see a count of differences, and drill into a listing page that lists those pages and screen sizes on your site that are different.

You can then drill into each page individually to see the specific differences for that page. 

There are four (4) modes that you can switch between to see the differences:

• Single Image - Show one of the images (before, after or comparison)
• Side By Side (DEFAULT) - Show all three images (before, after or comparison) side-by-side
• Flicker - Continually switches between the before and after images, allowing you to see what's changed
• Slider - Shows two of the images (before, after or comparison) overlaid on top of each other with a splitter handle that you can drag to expose either image.

As you view the differences, it might be handy to tag (or classify) why the images are different. 

If your site has pages that have dynamic, randomized, interactive or time-based content, you should identify and hide (or 'white out') this content to reduce the number of differences in your comparison results.

If you have two (2) cloned Development servers you can analyze and resolve any differences after you upgrade and perform a Comparison Run.

However, if you are running the tool against one (1) Development server, or a single Production server that will be upgraded in-place, we recommend that you initially perform a Baseline and Comparison Run against the same 'before' server to help identify any dynamic, randomized or interactive content.

This is especially important if you are upgrading in-place, as you are not able to go back and retake any 'before' screenshots once you have upgraded.

The tool provides two means for handling this type of content: Dynamic Regions or Action Sets.

For the purposes of this tool, we are defining interactive content as any content that is presented or exposed on the page based on a user action, such as a mouse click or hover.

There are many examples of this in a modern site, such as

• Mega Menus that show their content on hover or click,
• Accordions
• Tabs
• Sliders / Carousels
• etc.

If you want to test interactive content on a page you can do so by creating specific Action Sets, which in turn invoke small pieces of JavaScript code (Page Actions) that you choose from or write.

If a page has an Action Set assigned to it, it will run the appropriate Page Action based on the current device size, passing custom parameters to the Page Action code before the screenshot is taken for the page. 

For example, say you want to test that a Mega Menu with 4 sub-menus renders properly, you will need to do the following:

• Create and register a Page Action that simulates the user clicking a menu item, or better yet use the out-of-the-box 'Click' Page Action and pass it the CSS selector of the menu item to open.
• Create 4 Page Actions, one for each sub-menu item.
  • For example SubMenu-1, SubMenu-2, SubMenu-3, SubMenu-4
  • Each will reference the 'Click' Page Action, passing a unique CSS selector for the menu item to open.
• Add the Home Page (or any other page) to your Page Set 4 times.  Each will be assigned one of the 4 unique Action Sets.

When the Home Page is rendered each of the 4 times, JavaScript will be invoked that opens the Mega Menu and then takes the screenshot.  Since this same code will run on the baseline ('before') capture as well as on the comparison ('after') capture, the rendering of the Mega Menu can be compared.

Dynamic Regions are a mechanism to mitigate the dynamic, interactive, randomized or time-sensitive content on your site so that content of this type don't show incorrectly as differences.  For example, you may have a weather widget or a list of recent news articles or social media posts.  

If the content is known to possibly change between the time you run the Baseline Run and the Comparison Run, you should create a Dynamic Region to ignore this content, otherwise, pages with this content on it will be flagged by the tool as having differences, when they might not.

There are two types of Dynamic Regions you can create:

1. Dynamic Region based on CSS Selector
2. Dynamic Region based on Coordinates

For more detailed information on how to create and use Dynamic Regions, please watch the following video:

• Using Dynamic Regions to Mitigate Dynamic, Interactive or Randomized Content

An Action Set is a named set of Page Actions which are assigned to one or more screen sizes.

They are useful to either:

• Mitigate Dynamic Content - Similar to a Dynamic Region, or
   
  • For elements that can be easily referenced through a CSS selector, we recommend using a Dynamic Region. However, if you need more complicated JavaScript to find the dynamic element(s), you can write a custom Page Action (JavaScript Snippet) that either removes the element or covers it with a white div.
     
• Handle Interactive Content - So that the content can be captured in the page's snapshot image and compared.
   
  • Examples of this might be a Mega Menu, accordion or tab that exposes content when the user hovers or clicks or a type-ahead widget that hides items when the user types in an input control.  To capture and compare this content you can write a JavaScript Snippet that simulates the user action.
  • Note only one screenshot can be taken per page load. If, for example, you had a tab bar with 4 tabs, you would need to create 4 references to the same page in your Page Set, each applying a different Action Set (which invokes the same Page Action but with a different parameter identifying which menu to open).

For more information on Page Actions and Action Sets, please watch the following videos:

• Using Page Actions to Test Interactive Content (Part 1 of 2)
• Using Page Actions to Test Interactive Content (Part 2 of 2)

Tags allow you to easily classify page results, either for a Baseline Run or a Comparison Run.  By tagging one or more results, you can then filter the results to see results that are similar and apply actions against those results.

For example, from a Comparison Run you might see that certain pages render the menu incorrectly after the upgrade.  You could create a 'Menu Rendering Issue' tag, and classify any of these pages with that tag.

The following tags are predefined: 

• Content Changed
• Dynamic Content
• Insignificant Difference
• Rendering Error
• To Be Reviewed

You can add additional tags, or make any of the predefined tag inactive.

The Tags page, provides a listing of all registered tags for this site, along with a count of how many results associated with that tag.  You can add new Tags by clicking the 'New Tag...' button, or edit an existing tag.