I recently joined the Visma.net Planning team in Helsinki to test a new (and hopefully improved) concept of creating and contributing to a budget process. Visma.net Planning is a budgeting tool where users create their company’s budget and subsequent forecasts.
We had the advantage of already knowing what the problem areas were in the current version of the application, so the goal with the test was to make sure that the new concept we had created would be easily understood by the customers, and of course, to gather their feedback in order to further improve.
Visma.net Planning team from left to right: Andrei Tomici (developer), Dmitry (developer), Stanislav Ploschansky (developer), Dmitri Khanoukaev (QA), Jan-Markus Viikari (Product Owner), Jonas Lögdberg (Team Manager), Dorin Potorac (developer) and on the bottom me, Raquel Ferreira (UX Designer) in Helsinki, Finland.
As a User Experience Designer, the worst mindset one can have before visiting customers to perform user tests is to:
a) be emotionally attached to your sketches and
b) think you did a great job.
Because in the end, if customers don’t understand it, it means the process for us starts all over again and we need to come up with new concepts, new prototypes and conduct more tests. In Visma’s Product Unit we live by the “Fail fast, fail often” mantra, and I personally complement it with the thought that “all ideas are good, even if they serve the purpose of showing you that you were wrong.” In this field we have to grow thick skin, because even though it’s called “user testing,” we are definitely not testing the user, but the system we have helped to create.
“All ideas are good, even if they serve the purpose of showing you that you were wrong.” – Raquel Ferreira
The process-prototype was divided in two parts: the budget details (such as name, dates, input form, etc.) and the contributions (like adding users to the process, sending for approval, etc.).
Since this is a multi-persona process, we had to create two scenarios, one for each persona; a controller and a contributor. The controller is the one setting up the budget details, adding the contributors to the process, setting up deadlines and then reviews their contributions. As for the contributors, they are responsible for filling up the budget values for their own cost center(s) and requesting approval.
An example of one of the prototype’s pages that was tested, in this case the contributors statuses on the Budgeting process.
We tested the prototype with both customers who are already users of the system, and with customers who were not familiar with it, but who still fit the personas. This gave us the chance to evaluate the level of difficulty on both ends – experienced user vs new user.
As for the methods used, we were not interested in metrics at this time since it’s an early stage of the development process. We were more interested in the user’s comments, behaviour and opinions throughout the tasks and questions they were given. We encouraged the participants to “think aloud” and used Lookback to record both the screens being tested, the users facial expressions and sounds.
A screenshot of Lookback in action (a tool used to record the screens being tested while recording the user’s facial expressions simultaneously).
We got really good feedback. And by good feedback I don’t mean only the praising part. As much as we love to hear “this looks easy” we can only improve if we know where the problems are, and how real users actually think. Even small details that our tunnel vision doesn’t allow us to pinpoint during the development process.
But one thing we know for sure: we are on the right track, and I came back to Oslo looking forward to work on the improvements and to start the implementation.
I can’t wait to test version 1!