How to keep the advantages of manual testing with the speed and quality of automated testing
One of the advantages of manual testing that often gets brought up during the manual vs automated testing debate is the fact that people execute manual tests. This ensures that people will find the web app easy to navigate; if a site is too frustrating, a user will usually leave and go to a competitor. However, with all of the latest advances in acronym technology like Artificial Intelligence (AI), Machine Learning (ML), Natural Language Programming (NLP), and Robotic Process Automation (RPA), automated testing software can truly test like an end-user would.
When authoring tests with software that uses NLP where you have to write out the tests yourself, there is already that hint of the human element. You have to understand the website in order to write the tests, so if the tester doesn’t understand it, there’s a chance that the user won’t either (as opposed to if the dev team were testing a website that they knew inside and out). The last thing you want is to end up with a site like this. It doesn’t fit on the screen, one of the headers has code remnants in it, and one of the hyperlinks dropped a letter. It’s a disaster, but it’s supposed to be a disaster, so we can badmouth it freely. And though it’s an exaggeration, these are real examples of mistakes that can annoy users into looking elsewhere.
Another important aspect of UX that needs to be tested is accessibility. Text needs to be easily readable on its background color, and voice-to-text functions need to be able to read all website text aloud. Other examples of accessibility aren’t completely obvious at first thought. For instance, hit size area. Text fields and buttons need to be large enough for users to interact with. Fonts also need to be correctly sized to differentiate between titles, subtitles, and main text.
Testing a site like a user can also help unearth bugs. Sometimes when there are glitches in communication between the frontend and backend, site users that should only have surface-layer privileges can slip into an admin role. Web app code isn’t airtight, and sometimes things can slip through the cracks. There are some crazy bugs out there.
For example, according to a thread from 2008, Open Office when through a period where it wouldn’t print on Tuesdays. There’s a bug that only Continuous Testing could have found and not an obvious bug that devs could have discovered. A more recent example is a Tweet from 2017 in which a white person successfully activates an automatic smoke detector but a black hand failed to activate it. This could have been avoided by having a testing team testing from the perspective of the userbase, which would have included diversity. If you’re interested in more awesome bugs, there’s a great list on GitHub that has some great stories.
After establishing the importance of testing like a user would, we have to look at how technology can simulate that human gaze. Simulate, not replace. When running visual regressions tests, the tester needs to set the baseline to tell the testing software what the “normal” is. The software also can’t know whether changes are accidental or planned - for example when we put a Christmas hat on our logo, all of our tests for our site went berserk because every single page had been updated (yeah we drink our own champagne). Luckily with Virtuoso, it’s easy to accept all visual regression changes with one click, but it’s also reassuring to know that smart automation software can detect any and all changes.
Some automation software (like Virtuoso) can explore sites on their own, starting from the home page and pathing the web app out on its own. Since Virtuoso tests web apps on real browsers, all of the tests are conducted on up-to-date sites (no need to wait for Google to crawl it). All of Virtuoso’s busy-bee bots are working on testing your site to perfection, leaving more time for you to do exploratory testing. Plus, Virtuoso uses the Axe library to conduct accessibility tests and find violations. Sound right up your alley? Book a demo with us and take a tour of our platform to get automated testing that tests like your users.
