Measure Twice, Accept Once (Part 4)

What Is to Avoid When the Acceptance Testing is Performed?

The execution is to come – it should not be the last step and should be done frequently and also manual when needed. Do not forget to record and evaluate the data, so that you are sure that nothing is missed out. What is to avoid when the Acceptance Testing is performed – you will face numerous pitfalls, the one you should definitely avoid are:

Insufficient planning – we are always ready until we are not;
Lack of system availability and stability;
Lack of resource availability and stability – imagine a massive turnover rate and everybody leaves when the acceptance starts;
Poor communications channels – it is no different than any other aspect of the development and the testing process: if you lack the proper communication and the right communication channels, then you are definitely going to fail as there will always be misunderstanding among the team members;
Limited record keeping – keep your eyes on the essentials;

A very important aspect that is a real pain is the user involvement – this is highly recommendable factor to make your acceptance successful.

Involving the End – Users: Challenges

Involving the end – users in the acceptance testing phase is great, but it also sets a few quite important challenges:

Users do the acceptance testing in addition to the their busy schedules and to avoid that you should have them testing not at the last moment, but as early as possible;
As it is the last phase, acceptance testing may be turned to be a “formality” and to avoid that, the users should write their own test cases and let test alone as they have less or no devotion to the team/ project/ software;
Then it comes the challenge to motivate the users to do thorough testing even when they have busy schedules. Well, get to know your users and use different motivational techniques;
The worst challenge you may face is the lack of understanding how the system works – here, the so much mentioned test cases created by the users will help you deal;

And then, having in mind all the troubles that can be met with the involvement of the end user, you decide to follow the scenario where you entirely rely on the QAs you have hired in your company and who have worked on the project. The fact is that the QAs that have tested the software influence the Acceptance testing phase too much. That is definitely not good if you want to prove that the system does the things it is supposed to do in the way it is supposed to do.

Acceptance Testing Performed by the Internal QA Team

The result of doing the Acceptance testing with the QA hired in your company and that were doing the system testing are as follow:

It only proves that the application works as shown by the previous test stages;
The coverage is quite small and mainly UI – some even argue that this is what most of the users see – the Pareto principle;
They do what they do day to day as the corner cases are covered by the Functional/ System Testing and is proven correct;
Acceptance testing is not supposed to find any defects – the things that do not work shall come as change requests, i.e. the primary requirement was at fault;
Testers should not be involved in Acceptance testing as it is a milestone for requirements and their correctness;

What Are the Main Reasons Why the Acceptance Testing Phase Often Fails?

Thus, we come to the point to find out what are the main reasons that the Acceptance testing phase often fails. It is because there is no collaboration and no management buy – in. The other reason is the wrong focus – mostly testers focus in how and not what. Besides that Acceptance testing is often neglected it is fully performed by different and sometimes not suitable tools. And then, when the objectives of the team are not aligned and the skill set required is underestimated, there is no possible to have a successful end of the Acceptance testing phase.

So, it is quite understandable to see the fear of acceptance testing when it comes to that point in the testing process. It requires a lot of efforts, a lot of good planning and the ability to be able to adjust quite fast to the new realities. It is also not good to underestimate the requirements of the end users involved in that phase – as most of them will not be very good at IT literacy, you should be careful and patient while explaining the system and the steps for writing their test cases. Yes, that is difficult – you use a set of terms with your team and then the end user comes and is not aware of those terms – you will have to change the way you express yourself and find words for the terms that are common to the ordinary world. But, never should be also neglected the result of that involvement – the ideas the end users can provide and the bugs they can find – usually the one you and your team will definitely have missed out.

What Is the Best Option to Do the Acceptance Testing Phase Successful and, at the Same Time Avoid All the Obstacles Set Before You?

To outsource the acceptance testing to an independent organization that will organize and carry it for you both with the QAs it has and with end users that they will find and train.

Yes, despite the model and the strong belief that outsourcing the acceptance testing phase is not a good decision, many nowadays may say quite the opposite - outsourcing the acceptance testing is one of the great options as you set the requirements and you can guarantee to your client that the results are really independent and show what the opinion of the others for the product is. The outsourcing model works the proper way when both parties are sharing information timely and that information is correct and not misleading in any way.

To Outsource Acceptance Testing or Not?

It will not be misleading to say that the acceptance testing phase is not the preferred one to be outsourced as the managers are quite reluctant to lose control and more to that, they strongly believe that the internal team has a better understanding both of the system and of the requirements of the end user. That is, as other research show not quite true and may even lead to a failure.
A few reasons in support of the outsourcing of the acceptance testing:

External team definitely adds value in terms of completing the test coverage;
They have a more objective view of the business scenarios that may occur in that industry;
External consultant can help to test the performance of the application during peak periods;

Before Outsourcing the Acceptance Testing Phase, As Always There Are Things to Consider

The main six are listed below:

Establish goals for engaging with the acceptance testing consultants – the vendor should have expertise in the area and also should be engaged quite early in the process;
Innovation and customization are key qualities for the vendors – look for vendors with a creative approach to testing;
Analyze trends and metrics
Encourage cross functional coordination and inter – organizations communication – proper communication and the cooperation between internal and external teams is the key to a successful outsourcing;
Select the right people for the right job;
Develop effective tracking and controlling mechanism – at least at the beginning. After you have established a good relationship with the vendor, you may skip those well – defined and measurable parameters for monitoring and control.

Thus, after being careful and planning everything and consider the option of outsourcing, you may have a really good release in the end of the acceptance testing phase. It is not an easy task and do not neglect it. Neglecting it will definitely cost more than executing the acceptance testing on your own with all the struggles and a good option is the outsourcing.

Quality House Team