The continuous process of receiving and processing user feedback and timely responses to it is the key to the project’s success. Such a process is necessary for any scenario: developing software from scratch or improving an existing one. Usability testing is a method of evalusering an interface in terms of usability and the efficiency of its use. To get it, you need to attract representatives of the target audience of the software product.
Often, usability testing is carried out in two stages: the user passes N number of tasks (quantitative or qualitative tests), as well as a conversation, filling out questionnaires, or in-depth interviews with the user (qualitative research). As a rule, it should be carried out when an interface in the form of a paper or digital prototype or a ready-made software product is “done,” and you want to understand what problems arise and how the product meets users’ expectations.
The testing process takes, on average, from a month to three. It depends on the availability of different types of products, the complexity of the scenarios, their alternatives, the number of user roles. User testing is a process in which a group of people examines the effectiveness of a service and its functionality. Another name is beta testing. Testing is one of the building blocks of a project. The product designer must focus on each of the work steps.
What types of user testing there are, and which should you choose?
- Alpha/Beta testing: Instead of users, the product is tested by employees and other people close to the alpha stage project. A beta test is a next step when a group of potential customers gathers to test. For example, when game developers send out invitations to thematic resources to recruit people.
- Contract acceptance testing: They are used to check if the project meets all the requirements of the agreement between all participants. Most often, the process is necessary when working with a hired development team. The customer needs to make sure that the contractor has completed all the tasks.
- Legal acceptance testing: It ensures that the product does not violate laws and complies with all regulations within a particular industry. Most often, verification is needed for projects in the field of health and finance.
- Operational acceptance testing: It determines the effectiveness of processes outside the client’s visibility. Still, it is necessary to implement all product functions. This type helps to analyze data collection, defense systems, and so on.
- “Black Box” strategy testing: It is designed to study the causal relationship between user interaction with a product and the result. At this stage, people have explained what the product is for, but how exactly it works, they study on their own.
When the product is ready for user testing?
User testing shouldn’t be started out of nowhere. The product must be ready for it. The conditions for user testing are:
- Articulate business requirements.
- Requirements are outlined in user acceptance testing documents to:
- All parties have come to an agreement;
- Articulate how developers see the product;
- Collect information for the following stages of work;
- Describe how the product will solve users’ problems, meet the needs of businesses and customers.
User testing does not apply to functional tests. It is not suitable for finding crashes, bugs, and errors. Instead, user testing focuses on usability – whether everything functions the way it was intended. If at the moment, the project requires improvements, then it is still crude for user testing. Bugs need to be logged, corrected, and re-tested. To prepare for user testing, they need to be not only fixed but also fixed in a separate file:
- What was the problem;
- How it was eliminated;
- Confirmation that testing has been carried out;
- The result of the correction.
This method creates a transparent structure and visibility of the work that will satisfy all stakeholders. At this stage, the development team confirms its readiness for a beta launch among a limited number of users.
How to conduct user acceptance testing?
User acceptance testing requires plan, requirements, and deadlines, and a clearly defined goal. Every study must have a goal. You must understand precisely what information you want to receive. It is essential for yourself to answer the following questions:
- What will be vital for you in your research? What user roles or specific tasks are you looking to test?
- What are you hoping to capture in your notes? For example, you want to use video filming and mark interesting moments in video protocols to search and process feedback.
- Are you planning to explore the users’ workspace? There are times when it affects the organization of the interface.
- Is there a need to take photos or videos while observing the user? Perhaps some scenarios are easier to record on tape. For example, when designing a sound monitoring system, it is necessary to understand which communication devices the Service Supervisor interacts with and how many monitors it uses.
- What metrics will be used in testing? Quantitative or qualitative? It depends on what you are interested in in the research and what you are checking.
Suppose you are working on an application designed for a user’s workflow. In that case, you are unlikely to be interested in the degree of their involvement. You are very likely to be interested in scripting speed, user errors, scripts executed, and user satisfaction. You must understand precisely what information you want to receive.
Test plan creation
Every hero journey has a battle plan, every user testing project must have a test plan. To form a test plan, you must first of all determine:
- What product or products are you planning to explore? For example, a closed Product Experience Management web platform that manages all product information.
- What will be the subject of the testing? For example, the same web platform or a clickable, responsive prototype of its new version.
- What types of devices will be used? For example, a personal computer (PC), tablet, and mobile phone.
- What user roles and features will be tested? Let’s say the role is Administrator. The task in this hypothetical will include: creating a template for a product or product, checking a product template, approving a template.
If there are several products, such as a mobile web and desktop application, and you want to test each of them, it is better to do so. This way, get more accurate feedback and avoid mixed feelings about testing all products at once.
Every UX testing plan & script consists of the following elements:
- Introduction;
- Immersion of the user in the context of the tasks being solved;
- Description of tasks and a list of steps with the ability to add comments for each of them;
- A list of questions for each task;
- The final round of questions.
Don’t try to spot all of the usability issues in a large product at once. It is much more effective to split the scripts into separate small studies by roles with a specific purpose in each of them. To start on the right foot, try decomposition. What is it? We replace the solution of one problem with a series of small, simple, and interconnected subtasks with a specific goal in each.
Let’s take an example. The scenario of the task “As a user, we can book a hotel we like” actually consists of several small subtasks. As a user, we can find the hotel. We will find out how the user searches for hotels and check the appropriateness of the selected search criteria and its algorithm.
As a user, we can get information about hotels. We will find out if there is enough information in our data. Task scenarios should be specific and reflect the real needs of the target audience. Don’t ask the consumer to buy meat if they are vegetarian. Do not forget that the tasks for qualitative and quantitative methods are different. Let’s look at examples of tasks for each of them.
As a user, we can book a hotel. We check the chosen algorithm’s appropriateness, determine whether all the steps are clear to the user, whether it is possible to speed up the booking procedure, etc. We need to prepare a work plan and familiarize all parties and the development team with it. It is recommended in the letter to indicate the details, timing, and goals of testing, then to convene a conference with the participants to highlight the main points. All information for the test must be prepared in advance so that users do not have any problems. The work may require voluminous data tables, a description of the parameters. During the checks, you need to prepare the environment: install software, software, configure the program. During testing, it may be necessary to return the product to its original condition periodically. To avoid problems, users need to be given instructions.
Users should always have access to the system requirements, accompanying papers (even “help”). The initial information will allow the team to find inaccuracies and errors. Users need to give the contacts of the persons responsible for support. If they find a mistake, they should know who to contact. Test takers need to explain who is responsible for:
- Test coordination;
- Product requirements;
- Technical issues related to software;
- Testing questions;
- Rights, accesses, accounts;
- Installing software and setting up the tested environment.
The objectives for quantitative research
In qualitative research, we will understand the user’s preference for user flow. In quantitative analysis, we will clarify whether all user flow steps are clear to the user. Qualitative and quantitative research should be complementary, not opposed. Example scenario: A customer has asked you to prepare a report. You want to start by searching for information on the site www.sample.com. The task is “enter the system using your username and password”. The step would be as follows:
- Open a browser.
- Enter www.sample.com in the address bar.
- Enter your email in the Email field.
- Enter characters in the Password field.
- Press the “Login” button.
Alternative steps:
- Open a browser.
- Enter www.sample.com in the address bar.
- Click on the “forgot your password” link.
- Enter your email.
- Open mail client.
- Find a letter from the server about authorization.
- Follow the link.
- Enter a new password.
- Repeat password.
- Press the “Change password” button.
If you conduct quantitative research and measure, for example, the script’s transit time, the number of errors in one script, passed/failed the script. The level of user satisfaction – make all measurements exclusively on the production server, with actual user load, with existing data and working services, with their validation and requests to accurate databases. It is worth running the scripts on a test audience in advance to ensure the session timing is up to date and the user understands all the instructions. I usually budget for running scripts with three trial users.
Determining the number of researchers
For quality surveillance, you must have the following roles:
- Interviewer – conducts usability testing using a UX testing plan & script. Often this role belongs to the interface designer himself.
- Observer – refines the details and delves into them as testing progresses. This role can be played by a product owner, domain experts, or interface designers.
- Moderator – Must adhere to timing and ensure that the user plays all scenarios. Project and product managers, scrum masters do an excellent job with this. They also invite for this role as an interface designer.
Often, interface designers are interviewers and unknowingly push their solutions during testing. This is since it is difficult to remain impartial if you are very close to the software product. What to do in such situserions?
First of all, you should love the problem you are working on, not the solution. Only in this way will you be able to evolve in designing the interface of a specific task. Next, attract three independent opinions in the interviewer, observer, and moderator, who will help explain the situserion correctly.
However, it is not always possible to involve three researchers in each testing session. Often the functions listed above are assigned to one interviewer, making the research process an ordeal. If you are testing alone, it is best to run no more than three tests per day with a maximum session duration of 1.5 hours. If testing goes beyond the specified time, break it into two parts.
Determining the target audience
When working on software from a narrow subject area, demographics are usually insufficient. It is essential to understand the roles and their functions. For example, you are working on a financial product. You plan to study a broker as part of a collateralized loan obligation (CLO) business in the US market. In this case, you need to understand the algorithms for the investment banker’s interaction with other financial roles.
Preliminary questionnaires will help to determine the target audience. They are created with free services, such as Google Forms. Ask candidates about their occupation, territorial affiliation? What functions of the application are they subscribed to, if there is a subscription at all? Ask how often and why they use the product. If their goals for using the service coincide with the research purposes, these are your people. By comparing the test plan and the preliminary questionnaire, you will select potential candidates for research correctly.
The rules for creating such questionnaires are straightforward:
- Formulate questions accurately. As a rule, people understand the exact phrases in different ways. By using insufficiently precise wording, we cannot be sure that the user understood the question correctly.
- Don’t ask respondents what you know in advance — for example, his name, age, or title.
- Use closed-ended questions wherever possible. It is much easier for users to choose an answer than to write their own.
- Make sure you include all the appropriate answer choices for closed questions, including “Other” or “Custom,” if none of the suggested answers are suitable for the user.
- Always number questions.
- Show the total number of questions and the progress bar, orienting the person in advance where he is and how many answers he has to give.
- Yes/No questions can be used to determine which next question the respondent will answer. Use Google Forms, for example.
Status tracking
It is necessary to provide users with information about the status of testing: what work was done, where there will be delays, what errors have been identified. This way, they will be able to assess the big picture and understand their task. Users need to submit a final report. He must show what the work has influenced. The report indicates:
- What problems were identified, their assessment;
- Plans to fix bugs;
- Stages of planned optimization and future tests;
The result of the work’s acceptance and the following steps: whether the version for release will be approved, whether a revision is planned.
Helping hand in taming the beast
Before selling a product to target customers, you need to make sure that users can work with it the way they want. User acceptance testing is an extensive and vital process for preparing a project for release. By following the rules, you can provide users and customers with a quality, well-tested, and debugged product. If testing is significant, professional testers can be involved. The continuous process of receiving and processing user feedback and timely responses to it is the key to the success of the project. Such a process is necessary for any scenario: developing software from scratch or improving an existing one.
Any development or revision of software passes the final stage of user testing. In this case, testing is carried out by business users of the adopted system. Ordinary business users are not professional testers and cannot fully test improvements. But we can help you in taming this exceptional best. Well-crafted user test scripts will make the work of business users easier and will significantly improve the quality of testing. Our company has extensive experience in conducting user acceptance testing. For our clients, as part of ensuring the quality of the software product, we carry out the following work:
- Development of a user acceptance testing methodology;
- Development of a detailed schedule of interactions between business units during testing;
- Preparation of acceptance tests;
- Organizing and maintaining users as a testing coordinator.
Our experience will allow you to competently plan the acceptance process and carry it out on time with a high level of quality.