Testeum is a crowdtesting platform that offers several direct advantages :
- Real-world testing in realistic environments
- Portability testing
- UX testing
- Internationalisation testing
- Geolocation testing
But it also offers some indirect benefits, such as the possibility of acquiring new users who know your services perfectly !
However, questions remain that prevent some organizations from taking the plunge : how does it work in practice ? How can crowdtesting be integrated into one’s organization and quality process ?
How does crowdtesting work ?
Simply put, on one side, we have an organization and its’ digital service (website, mobile app etc), and on the other side, tech lovers looking for extra income. Crowdtesting consists of connecting these two groups in a relevant and organized way, using a crowdtesting platform. The platform allows organizations to broadcast their tests and allows testers to provide their feedback : successful tests, failed tests, bugs found, and so on.
For example, on Testeum platform, dashboards are available and updated in real-time as tests are run by crowdtesters.
What crowdtesting platform to choose ?
This matter will obviously have a considerable impact on your crowdtesting journey ! There are currently many platforms, each offering its own specificities and added value. Testeum cannot be the judge of course. But we promise this is the only question in this article we won’t attempt to answer.
What can crowdtesting actually test ?
The offers may vary, but in most cases, functional tests, usability tests and portability tests are the most frequently offered.
When to deploy crowdtesting ?
When you already have relatively good confidence in the quality of your version, crowdtesting allows you to find the bugs that may have escaped you, and/or that you would not have been able to find internally. It also allows you to improve your internal testing strategy, as what is discovered during the crowdtesting phase can help you level up your regression tests.
Crowdtesting can also be deployed earlier, but be careful to still ensure sufficient quality: it would be unfortunate to receive bug reports that you could have easily spotted yourself !
How to choose scenarios to be tested ?
Obviously, the first scenarios you should choose are critical ones : “If this feature doesn’t work, we’re screwed !”
End-to-end tests are excellent candidates for crowdtesting. Some general test goals examples :
- Sign up on the website,
- Complete your profile,
- Browse the items,
- Use the search engine of the website,
- Make a payment,
- Check the invoice.
Also think about tests that are not automated, or difficult to automate.
How to write tests for crowdtesting ?
First guideline : keep it easy to understand.
First and foremost, it’s important to keep it simple. Crowdtesters who will access your product may not necessarily be familiar with your field of expertise. Therefore, it’s important to explain any specific vocabulary.
If your website is intended for the general public, such as an online store, the risk of misunderstanding is relatively low, but be careful with web jargon ! For example, the term “modal” is not well-known by the average person, and it might be better to use the term “popup” (even if it’s not exactly the same thing) to ensure everyone understands. Similarly, words like “breadcrumb,” “footer,” “widget,” should be expressed with simple terms.
Second guideline : keep your tests independent
This good practice is valid for both crowdtesting and other testing contexts.
Each test case must be able to be played in any order, without affecting the other tests. If you have a test case, for example, “Creation of a user named Yummy,” do not have another test case “Deletion of the user Yummy.” Otherwise, beware of the snowball effect if the first test in the series fails!
Playing tests in any order is a worthwhile exercise. It is also interacting with the application in a way that is closer to reality; in production, users follow their own path and take steps in the order they want, which is not necessarily the one we had imagined.
Also, keep in mind that the same test case could be played by several different testers. Pay attention to the related constraints :
- duplicates (multiple people may try to create the user “Yummy” and receive an error message “This user already exists”)
- concurrent access (on some websites, it is only possible to have one active session per user, with an automatic logout system).
Is crowdtesting relevant for testing B2B services ?
Of course ! Crowdtesters are there to carry out test scenarios, and if they are specific enough, there is nothing stopping you from having your B2B services tested by them.
In a wider perspective, getting feedback from users outside of your marketing target can be very interesting; for example, it could make you aware of certain problems that you have not yet encountered in production, but which could arise later if you open up to other audiences or geographic areas.
We can't find any more bugs internally, what can crowdtesting bring us ?
You have tested to the best of your abilities, within a limited time frame, with limited resources, and undoubtedly some biases -as we all have! Crowdtesting allows you to break free from these material and intellectual limitations: your service will be tested on a range of machines of all brands, sizes, and years, by people from all walks of life.
The bugs found during crowdtesting campaigns are therefore as varied as they are unpredictable.
Here’s a juicy example. One of our crowdtesting campaigns revealed that on a website, a certain button was clickable using a mouse, a trackpad button… but not with “tap-to-click” !* No one had found this bug internally, yet the feature had been in production for a few weeks and seemed remarkably stable. Was that infamous button the “Pay My Order” button? The story doesn’t say… but just imagine !
To get down to business we recommend reading this feedback, which recounts a crowdtesting experience with Testeum. Happy reading !
* “Tap-to-click” is a feature of laptop trackpads or touchpads that allows you to click on an element by simply tapping on the pad, rather than pressing a physical button.
