Test Automation Framework: How we built it for Document360

Document360’s test automation framework

We developed a hybrid test framework using Selenium C# with NUnit. This improved the testing efficiency and helped to optimize total testing time. The main advantage of a Hybrid framework is that it allows us to combine any two or three other frameworks’ approaches to overcome the limitations of one framework. This brings more flexibility while doing end-to-end automation testing.

Below is the Architecture Diagram of our Document360 test automation framework.

Our Automation Framework has two parts,

• Core
• Document360

Core Library

Our core library has all the standard utilities which are required to write and execute tests right away, irrespective of the application like

• Creating various browser driver instances
• Selenium wrapper methods and custom methods
• Excel Class to read the test data from the Excel file.
• Mailer to read Outlook mail and validate its controls.
• A web client for REST API methods (Get/Post/Put/Del)
• Snippet methods to validate the API response.

Document360 library

Document360’s automation framework combines modular-driven and data-driven approaches, in which we split modules and organize them in the same structure as in the application. All the test data are kept separately from the script for easy maintenance.

Our framework also follows the Page Object Model (POM) design pattern. Every page, including the blade and popups in Document360, is considered an object, and separate classes are created for each page to have its web elements and methods to interact with it. All the web elements on the page are initialized using a page factory with polling intervals to avoid unnecessary wait time and “NoSuchElementException” due to external factors.

Document360 automation test framework has the following components.

Base Layer:

This Layer is a test bed for all other classes. It has pre- and post-scripts to run each test case in our automation suite.

Configuration Layer:

This Layer contains the configuration-related data and its classes. It has data like threshold limits, URLs, and credentials details for different environments (like DEV/QA/PROD) that need to be used in our application.

Test Layer:

The Test Layer comprises test classes containing the automated steps to validate the available cases and their corresponding page object classes.

Data Layer:

This Layer has all the input data needed to pass it to test scripts stored in a separate file and maintained globally to reuse it everywhere required.

API Layer:

We allow customers to integrate Document360 as an extension with helpdesk and team collaboration applications like Freshdesk, Slack, Microsoft Team, and many more.

Not all the cases from these modules can be automated through UI alone to ensure functionality, and populating the high volume of test data for modules like analytics and bulk operation is a tedious one. We addressed these kinds of challenges at the API level.

Document360 APIs are well secured, and we need a valid bearer token and project id to access Document360 portal APIs which we extracted from headers using Selenium Chrome DevTools Protocol (CDP)

This Layer has logic to read the API details to maintain in Excel and generate a request to send it to the web client and receive server responses.

Reporting

We have robust reporting mechanisms, implemented using “Extent Reports” to generate a clear report with all relevant information about the tests. We logged exceptions and screenshots in case of failure for further analysis. This Layer holds all the screenshots and reports generated for each test run.

The sample extent report dashboard with failures

Test execution using Azure pipeline

We integrated CI/CD pipeline (Azure DevOps) with our automation project to execute the automated cases and give feedback about the build quality faster.

We have set up two different pipelines, one to execute all the automated cases on the scheduler configured and another to run cases from the test suite we have in Azure Test Plan on a demand basis.

A successful outcome of our test automation

At first, we could not measure the outcome of our automation framework accurately like other automation teams, we expected test cases to fail or be unstable due to other factors.

To overcome this, we identified and ignored the flaky tests in execution to avoid misleading results. We ran the test suite constantly to fix the flakiness in tests and included it in the test run.

Once a robust framework was in place, the test cases in the regression suite were automated and stabilized. We have seen huge progress and reaped the following benefits of automation:

Saved time and effort 

Automation in the regression cycle, automated cases that required checking for different data in the suite. The execution was fast and gave us the results in a few minutes. It reduced our testing time for those cases from a day to a few hours.

Identifying bugs in the earlier stages 

We have configured our automation script to run daily This helps us to identify bugs in the early testing stage rather than the regression phase.