This is the last part of the Mystery Shopping 101 series. If you missed any of the postings, you can find them on our blog. In the last posting, I talked about how the characteristics of the mystery shopper are built by the testing company. In this posting, I’ll write about the testing protocols and how the company performing the testing looks at the results.
Protocols for Testing
Consistency is key for any testing protocol. This means that a lot of time has to be spent with those who will be doing the testing so that they are equipped to handle any situation. Here are the five basic steps that are followed, once the mystery shopper has completed their training and understands the objective of the assignment.
Testers are to follow the exact same protocol which generally includes:
- Making an appointment
- Advising the originator what they would like to accomplish, such as what they qualify for.
- Providing personal and financial information.
- Carefully recording the options presented by the originator.
- Thanking the originator and waiting to see if the originator suggests follow up!
Once the tester is done with the meeting, they have an extensive survey to complete. Here are some of the sample questions and feedback.
- Who was tested and interactions with other employees in the office, including visual observation of their race, sex, age and gender.
- Whether the tester met with the person they spoke with or whether they met with someone that works for that person.
- Did the originator indicate that the tester should talk with a real estate agent before meeting with them?
- Did they suggest that the tester should contact another lender to inquire about financing? If so, why?
- Recording of very detailed information about what the tester was asked for, and whether they volunteered the information, or the originator asked for the information.
- Were there any products that the tester was told they did not qualify for? What products was the tester qualified for?
- Did the originator offer any other types of assistance such as down payment, closing costs or other ways that would assist the tester? The tester provides a high level of detail in the survey.
- The tester makes note about what loan amount they were qualified for, and what range of house they should buy based on their financial profile.
- What are the costs and payment associated with programs that were offered, along with any loan terms?
- Did the originator make any comments about the area the tester is looking in, or the area the tester should be looking in?
- Did the originator make disparaging remarks about one loan product over another?
- Was the tester invited to apply and if so, was there a fee?
- Did the originator being tested follow up and if so when?
This is just a small sampling on what the originator is tested on and what the tester reports. At the end of the testing, the data from all of the testers is either reviewed for reporting; or if the sampling is large enough sent to a statistician to evaluate the data. In the BancorpSouth case, originators were quoted specifically on disparate treatment concerns. For instance, one originator consistently gave a higher loan amount and higher sales price range to the white tester who made less, had a lower credit score, and less assets versus the minority tester. In addition, the originator discussed loan options that they did not discuss with the minority borrower, which would have cost the minority borrower more.
In my own experience in testing originators, it can be quite scary. This includes experiences with rookies and top producers. Yet I would rather have scary results with someone who is not a regulator, versus scary results from a regulator. At least I will know what I need to fix! Even if you don’t do any formal testing, just hang out and listen to the calls. You might be surprised at what you learn!
So as I complete this series, I sure do hope I gave you something to think about! There is so much more that good training covers on this subject that cannot be covered in a blog posting. We can all say that our originators would pass this test with flying colors, but until you hear them first hand, you don’t know what you don’t know!
Worried about your originators passing the Mystery Shopper Test? Contact me for my comprehensive training program on “Passing the Mystery Shopping Test-What Every Originator Needs to Know”. It makes a great conference training session!
