Best Practices in Agile Remote Testing

Anyone that has carried out a software test knows just how much work is involved. In the following article, we show you a concise summary of best practices for agile remote software testing.

Today is the day: time to test a software prototype.

Anyone that has carried out a software test knows this scenario well, and all the work that goes along with it. Participants have to be recruited and invited, and you spend the whole day carrying out three to four sessions. But there is an alternative that lets you reduce your workload and that is quick and simple to implement:

agile remote testing.

In agile remote testing, your participants take place in the test virtually, from wherever they are. If you master the following tips and tricks, you will get good results even more efficiently.

Saving Time: Participant Recruitment

Comission a Recruitment Agency

Recruiting for remote usability testing can be carried out by the customer or by an external service provider. Especially when dealing with consumer products, it makes sense to commission an external partner. You won’t just save time on organization but will also get ideal test participants from a huge pool of age groups within hours, complete with CVs and interests. Participant criteria and dates can be defined in advance. Most participants will have already carried out usability tests, so will be familiar with the process. This will save time on briefing.

A usability test should not exceed 5-8 participants. According to Nielsen, 85% of all usability problems are identified with just five participants.

Have Replacements on Hand

It can always happen that a participant drops out. So you should always look for two to three people more. When you book participants via an agency, their factsheets will help you make your selection and you will have access to their email address or, better yet, their phone number. This ensures you can find a replacement quickly. As each drop-out will take up time, plan enough time between slots to avoid stress.

Well Prepared: Set-Up

Set Up and Check

Always use a LAN connection rather than WiFi to avoid any connection problems.

You call the participants using an online meeting tool (Skype, GoToMeeting, Teamviewer etc). The whole screen is recorded using a recording tool (such as Quicktime). Duplicate the screen so that all connected monitors see the same thing.

“Eye contact” between the participants and you as the moderator is provided via a webcam.

For the best sound quality, it is best to use additional speakers and an external microphone.

Don’t forget it’s important that you feel comfortable while you moderate the test. Before the session, set up the room so that you have everything you need within reach, and remove distractions out of the participants’ view.

Benefit of an Observation Room

Even though an observation room is not strictly necessary for remote testing, it still comes with some benefits.

Everyone involved in the project can follow the test without disruption and carry out a live evaluation of the findings in the form of a user journey map. Plan in enough time between test sessions to consolidate findings. This lets you save time later on by avoiding lengthy video evaluations.

Ready? Preparation and Technology

Send Participation Form in Advance

To ensure that remote testing can begin immediately at the start of each session, you should send the participation form to participants in advance. Especially if the participants have to take on a role, for instance, it is beneficial to provide this information in a timely manner.

For reasons of practicality, the non-disclosure agreement is recorded verbally on the soundtrack. In the future, you could request a digital signature at this point. As this process currently requires certain hardware for each participant, it is not yet standard practice.

Test documents - such as content for forms or specific behavioral requests - should NOT be sent in advance so that the test is not affected.

The actual tasks are read out by the moderator during the test, as it is hard for the participants to switch between prototypes and tasks.

Professionalism is Important

Make sure to be professional even before the testing begins. It makes a better impression among the participants if you call them from an official company account. As they can see you, clothing should also be appropriate.

Start each session with an official greeting in which you introduce yourself and your employer, and explain what the test will entail. Tell them that they can ask you questions at any time. It is also important that the participants know that the test can be stopped at any time. Nobody should feel forced.

Explain Participant Set-up in Advance

Always remember that the participants’ equipment is mostly not professional. They mostly carry out discussions and tests at home and mostly use low-quality microphones, cameras and/or computers. Dips in connectivity (e.g. buffering) must always be taken into account when scheduling.

If the participant is lit from behind, for example, tell them and ask them to find another place for the test where you can see their face. Distracting pop-ups (e.g. antivirus) often open automatically. You can prepare your participants with a test call before the session. Explain step by step how to start the prototype.

YOU’RE LIVE!

Providing the Prototype and Instructions

To ensure latency-free interaction with the prototype, you should make it available on a password-protected server. After the introductory questions, you should provide the necessary information via chat.

After the test, you can transfer your screen to address questions in detail. This has the benefit of allowing you to jump to certain content more quickly.

Place the Prototype in the foreground

In order to be able to completely understand the prototype, you must inform your participants that camera windows and other distracting windows should be closed.

As the participants have different hardware and screen resolutions, static prototypes may not be completely visible and may have to be cut horizontally. Important operating elements cannot be properly viewed.

Ask the participants to minimize the zoom in their browsers or to center the field of view on the prototype.

Last but not least

Watch your Body Language

As the participants can see the moderator and minute taker on the camera most of the time, you should watch your body language. Positive body language will motivate and encourage the participants to solve their tasks.

Be Friendly and Helpful

Always make sure to be friendly so that the participant feels at ease. With your positive presence, you will encourage participants to give open feedback. The situation becomes more open and communicative, which is essential for good testing.

Of course, you should keep calm during technical problems. After all, a solution is usually quick to find with a cool head.

Summary

Our “Best Practices” at a Glance:

• Commission a recruitment agency

• Have replacements on hand

• Set up and check

• Send participation form in advance

• Professionalism is important

• Explain participant set-up in advance

• Provide prototype via server

• Complete visibility of the prototype

• Watch your body language

• Be friendly and helpful