Through a declarative development approach, Salesforce has managed to make the application creation process easier, faster, and more accessible. However, its out-of-the-box tools for DevOps, particularly for automated testing, have been lacking. One way to compensate for this is by employing third-party DevOps tools for automated testing and deployment.
In this guide, you'll look at comparisons between two of the most prominent solutions in this arena, Provar and Selenium. You'll explore the features, ease of use, speed, maintenance requirements, accessibility, and coverage of each product.
Salesforce’s slew of impressive low-code builders and tools have made the application development process incredibly easy and fast. However, this ease of use has come at a cost: it has seemingly lulled organizations into a false sense of security.
Consequently, many of them fail to establish well-defined development cycles. This often forces them to build, test, and run in production before deployment. While this may not be an issue for small-to-medium enterprises (SMEs) creating simple in-house apps, it can lead to an increased risk of failure, especially for development teams constructing complex modular programs.
While Salesforce provides sufficient tools for sandbox testing, it can make configuration and deployment more challenging. This is most commonly a problem when you’re working with complex workloads that involve extensive amounts of metadata.
Deployments are likely to become more infrequent as they increase in complexity. In many cases, this forces companies to rely on large deployments that hinge on careful coordination between members of your team. These large deployments still carry high risks of failure, despite how well your team cooperates, clearly illustrating the need for DevOps tools to help you orchestrate and automate the testing and deployment processes.
Read more: Our detailed comparison of Salesforce DevOps tools: Opsera vs Copado
While these tools are often discussed in similar contexts, they are very different—especially in origin. As such, it’s important to understand their backgrounds before you commit to deciding which one will ultimately suit your Salesforce DevOps testing requirements.
The key similarity that both solutions share is that neither of their primary products require any scripting for test building, allowing both developers and non-developers to utilize them. However, this does not mean they are both equal in ease of use.
Provar is a low-code, end-to-end test orchestration solution designed specifically for Salesforce DevOps. It uses a similar user interface style to Salesforce, making it easier for experienced Salesforce users to acclimatize. Provar Testing was established in 2011.
As a paid product, it doesn’t have the community support of Selenium. Furthermore, it’s made specifically for Salesforce, so much of its API support is reserved for Salesforce APIs, such as SOQL and SOSL.
Selenium is a more generalized automation tool for web applications. It offers an advanced, low-code Integrated Development Environment (Selenium IDE) that allows you to create functional tests that are compatible with a large variety of browsers. Selenium has been around for longer than Provar, with the first version released in 2004. Since then, it has been embraced by the open source community. Consequently, many different versions of Selenium have emerged, primarily supported by donations from community and corporate sponsors.
Many of these sponsors use parts of the Selenium project in their own products. Selenium has always supported a diverse range of frameworks and programming languages. Even as the latest version of the IDE uses a no-code record and playback methodology to test building and automation, the WebDriver, Selenium Remote Control (RC), and the Selenium API can be plugged into custom applications and code.
Selenium supports a number of languages, including Java, C#, Ruby, JavaScript, and Python.
Selenium has been around longer than Provar, has a thriving community, and is compatible with a long list of programming languages and frameworks. It has been tried and tested longer than Provar has.
With every passing year, there's an increasing corporate emphasis on the use of tools facilitating no-code and low-code DevOps orchestrations. As more organizations apply fusion team models to their CI/CD pipelines, they need tools that can be used and understood by non-coders. While both Selenium and Provar are no-code solutions, not all no-code tools are built the same, and some are much easier to use than others.
Provar caters to users with very little automation experience. It consists of two main parts, Provar Desktop and the Test Builder. Installation and configuration of both are expectedly straightforward. However, there are some prerequisites that you should be mindful of.
For instance, you must enable Salesforce system administrator permissions to integrate Provar into it. Furthermore, to allow Provar to access your environment’s metadata, you must ensure that your profile’s "API Enabled" and "Modify All Data" fields are ticked.
Provar Desktop uses a standard installer containing a helpful wizard to ensure the setup goes as smoothly as possible. Once you’ve completed the installation, you’ll have access to Provar’s desktop application, as well as its Electron-powered, web-based Test Builder. The Test Builder allows you to author UI tests for websites and web applications, including Salesforce and non-Salesforce applications. It is the most accessible tool of Provar’s offerings.
Provar Desktop is intended for more robust test building, such as API testing, and thus offers a classic low-code IDE look and feel. It’s very similar to how Salesforce organizes its user interface. However, some users may find themselves overwhelmed by all the UI controls. To combat this, Provar provides you with an Assistant view, which displays helpful tips and information about the currently selected tool to make the test-building process go more smoothly.
The Assistant view also includes a search function that can be used to find additional information. Both Provar Desktop and the Test Builder are code free, and feature drag-and-droppable elements.
Because Provar uses a style similar to Salesforce’s suite of applications and schema builders, knowledge from Salesforce transfers well to Provar, allowing users to adapt to it quickly.
An example of the Provar Desktop user interface, including the Assistant, Navigator, Objects, and Test Settings views
Much like Provar, Selenium is a multifaceted testing solution that has evolved over the years. Initially, the Selenium IDE was a desktop application that required scripting for test building. However, the latest version comes packaged as an extension that’s available for most browsers.
While it’s slightly similar in function to Provar’s Test Builder, you can install it on your browser without committing to the entire Selenium Test Suite. Additionally, Selenium IDE’s interface, controls, and features are far more minimalistic than any of Provar’s offerings.
Selenium IDE is essentially a record and playback test automation tool. There are no prerequisites for installation. Selenium also offers users a desktop version of the IDE, as well as its source code. The entire Selenium project is open source, which allows you to modify and expand its features.
Both versions of the Selenium IDE have clean, well-organized interfaces.
While Provar is specifically designed for Salesforce test automation, Selenium’s IDE is easier to set up and use, requiring minimal setup and configuration.
This is far removed from what the Selenium IDE was in the past. Previous versions required prerequisite software components such as the Java SDK and Eclipse IDE. While you can still access these older versions, it’s unnecessary, as Selenium’s libraries and WebDriver offer the same functionality.
The Selenium IDE is more than enough for most test cases, and you can also use it to orchestrate other aspects of your CI/CD pipeline. While it’s true that Provar’s products are easy to get started with, they cannot contend with the Selenium IDE's simplicity.
It’s important to consider how well your test cases can be modified and adapted to evolving situations. Your Salesforce-built applications will go through many changes as you refine them, and you don’t want to build new tests each time you change a field or UI element. It’s also important that your candidate test automation solution features ample customer/user support.
Provar’s testing solutions were made for Salesforce, and any updates Salesforce makes to its app-building products will be reflected in Provar. This effectively guarantees that you won’t ever have to work hard to apply Provar’s tests to Salesforce’s future features.
Most complex Salesforce applications use front ends with dynamic frames and containers, which many generalized test orchestration tools struggle to handle. Provar allows you to generate tests that provision for these cases.
Salesforce web applications tend to have dynamic fields and elements, which means you cannot hard code the locators for these values into your test projects. The latest version of Provar allows you to build tests that can flexibly access and interrelate with dynamic forms. These adaptable test projects are easier to tweak, maintain, and replicate.
Selenium is made for general web browser automation, not just test automation. The Selenium IDE isn’t made to create tests that accommodate dynamic content—at least not on its own. You will need to use Selenium’s developer tools and WebDriver to access the dynamic content’s attributes.
Additionally, the Selenium IDE isn’t great at testing complex UI controls such as Shadow DOM, dynamically populated tables, and pop-up windows. It takes a substantial amount of effort to author automated Selenium tests for these scenarios. If you’re building complicated Salesforce apps and don’t have any intimate knowledge of how Salesforce generates its code, building effective tests will be nearly impossible. These tests won't just be difficult to author, either; they’ll be hard to maintain as you add more dynamic UI controls, elements, and hidden fields.
Moreover, the effort involved in this undermines the purpose of these tools, which you're most likely using because you want to increase your efficiency and avoid spending valuable time writing test code.
Provar is unmatched in this aspect, as it’s made for Salesforce. It has a comprehensive list of functions and features for dynamic tests. Additionally, its debugger is far more powerful than Selenium’s. You can use it to record variables that may not be visible during the test’s execution.
Provar also allows you to draft test steps, offering you more control over every detail of your test case. Provar Test Builder infers some of the fields using your previous information. It will actively examine your Salesforce environment, and predict what actions you would most likely take.
It enables you to create more intricate test cases quickly. If you have any issues with Provar Desktop or Test Builder, you can reach Provar’s customer support using their social media channels (such as Twitter and Facebook), email, or phone. Ultimately, tests built using Provar are more resilient and require less maintenance than those created with Selenium IDE.
Modern web applications are multifaceted and offer a range of functionality. Consequently, you need a tool that’s capable of testing all the components and modules of your Salesforce app. This category will look at which tool has the best coverage.
Provar is touted as an end-to-end testing solution for Salesforce, and it delivers. It allows you to orchestrate tests for your entire business processes. Again, the Test Builder is suitable for front-end user interface tests, while the Provar Desktop test creator is designed for backends, API testing, etc. The Test Builder is capable of handling Visual Force and Lightning components.
It’s important to remember that Provar was designed for Salesforce testing, and is built to be capable of handling most Salesforce DevOps test case scenarios. Provar is available for the latest Mac, Windows, and Linux operating systems, requires 8 GB of RAM (although systems with 16 GB are suggested), and only supports Java 8.
Selenium doesn’t have the same intrinsic coverage as Salesforce, at least not out of the box. It takes considerable effort and customization to adapt Selenium for end-to-end Salesforce testing. However, Selenium is more sparing with system resources. It requires a minimum of 4GB of RAM and will work on any operating system with the latest browser.
The Selenium IDE is more suited to general front-end tests. Provar was made for each step and phase of the Salesforce development cycle. While it does have higher system requirements, your enterprise should be able to handle them, especially if it builds complex web applications.
Both options have their strengths and weaknesses. Here’s an illustration of how they compare.
| **Category** | **Provar** | **Selenium** |
| ------------ | ---------- | ------------ |
| Background | | **✓** |
| Ease of use | | **✓** |
| Maintenance | **✓** | |
| Coverage | **✓** | |
| Support | **✓** | |
| Overall features | **✓** | |
There are a lot of good things about Selenium. However, Provar offers more intensive Salesforce testing. While it’s made to be accessible to non-developers, it features a host of advanced configurations to help you test against complex test scenarios. Selenium simply can’t compete with its array of features.
Selenium is likely a good choice for startups and SMEs who build simple applications. It also may be the better option if your company hasn’t committed itself to Salesforce, and might move away in the future.
Provar, on the other hand, is made for Salesforce DevOps test orchestration. Selenium simply can’t compete with Provar’s range of tools and features. From its integration of AI for quicker test case building to its superb handling of metadata and dynamic visual content, Provar fully integrates into your Salesforce ecosystem.
Test automation and orchestration should only be a part of your DevOps and CI/CD toolchain. It’s more than likely that you’ll use different solutions for monitoring, security, and deployment, and these solutions must work together within your pipeline. Opsera is a solution to help you orchestrate these processes.
Opsera offers complete end-to-end visibility, and provides unified insights through analytics and diagnostic reports, which makes it an ideal fit for enterprises utilizing DevOps fusion teams. Much like Provar and Selenium, it uses a declarative, no-code approach to automation and CI/CD pipeline creation, allowing all members of the team, regardless of their technical background, to contribute to the pipeline creation process, accelerating development. Opsera also integrates seamlessly with Provar, Selenium, and many other tools, allowing you to create a toolchain that works exactly the way you want it to, using the tools you want to work with.
