Running large multi-team Product development

User experience and the customer journey is more than ever on the top of everyone’s mind in the software business. The feature-game of the past is being replaced by easy-to-use, easy-to-understand applications. Users of business applications expect simple applications focused on solving their tasks, in a simple way and without spending unnecessary time doing so. But how do you achieve this running large multi-team Product development organisations? 

In Visma, we are building critical business applications, with a mission to simplify everyone’s life by using technology. It’s a great mission, but building business-critical applications require that it is correct. Business sustainability and quality of people’s daily lives depend on correct calculations. Failure is not an option when delivering payroll and accounting systems. That is why our methodology is centred around the user, ensuring that the required validation steps are in place. 

True SaaS applications tend to be decomposed into smaller components often referred to as microservices. Whether to aim for true microservices or also allow macroservices, are a never-ending discussion. What we have experienced, is the fact that microservices bring some great benefits to the development organization:

Clear ownership of the source code

Our experience is that clear boundaries and ownership of the application is a critical success factor for highly efficient development organisations. We have found that ensuring that there is one, and only one, team that owns each component in the code is crucial. Shared responsibility does not bring the level of responsibility and engineering excellence we expect within our organisation. For this reason, we aim to have a one-to-many relationship between a scrum team and microservices. 

Clear organizational scale-up and down strategy

Having a clear team – micro services relationship, there is a clear strategy on how to scale the organisation. Usually, each team is responsible for more than one microservice. In need to scale up the development capacity in this part of the application, it is quite easy to extend and split the team. In case there is a need to scale down the development capacity, smaller teams can be combined by adding some more microservices to the new teams’ responsibility. For sure, the responsibility of the team increases – but in this situation, it is likely that the investment in the or at least some of the included microservices are slim. 

How to organise teams to give customers a holistic experience?

Now, a critical question arises; when your software development teams are organised according to microservices, e.g. small parts of the service you offer your customers, how do you organise yourself to make sure that the customer gets the best, and holistic customer experience? How do you organise yourself to ensure that the customer point of view is taken care of when each team is responsible for a small piece of the overall offering?

This was the fundamental question we were facing in the Visma.net HRM development organisation. Having multiple teams delivering the service to the end-user, the holistic product view required extra attention. We quickly realised that if we wanted to secure and deliver the best possible user experience – we needed to address this through the product management process as well as how we decided to organise our software delivery process – obviously, these are closely related. Working to improve our delivery processes we established the following critical success criteria:

  • Everything has to start with the customer and what we offer.
    To us, this means that we have decided to set up our organisation according to the offerings we bring to the market. 
  • Ensure full transparency in the organisation.
    It is very important to have one, and only one backlog of work. Further, this has to include all functional and non-functional work. This idea is easy to agree to, but often not so easy to live by. For any modern software development organisation there are multiple needs to be handled; security, performance, UIX improvements, and delivering the next mind-blowing feature is competing for time and attention in most development teams. This was also the situation for us. When not having a clear and transparent backlog, we found that in most teams there was happening Dark work. Dark work was introduced by Jeff Sutherland as zero value stories, e.g. activities initiated below the surface. Sometimes these activities were important to refactor work-items that were never prioritised. Sometimes they were of less value to the end-user. What therefore decided to ensure that all functional and non-functional work items were equally important delivering value to the customer.
  • “Simplicity, simplicity, simplicity”.
    Processes shall never, ever, be complicated. Processes are in place to support teams and organization delivering value. The processes in an organisation will never bring value to the customer. Its purpose is to make sure that the organisation applies best practices in the industry and is moving up the learning curve as quickly as possible. 

Based on these three success criteria – we found that our software delivery process consists of 3 key phases.

  1. “Solution scoping”
  2. “Software design”
  3. “Implementation” 

It is important to highlight that the described framework is meant to handle the bigger items delivered by the organisation. Our development effort in Visma is based upon true DevOps ensuring rapid turnaround and frequent deployments by the individual teams. Alexander Lystad has described “Our Large Scale DevOps Transformation” in his blog post. This framework will address cross-team deliveries that require a proper holistic product view.

Solution scope

The first phase has a clear purpose: to understand how we will solve the prioritised need or problem for the customer, what will be the steps to deliver the solution. Not forgetting to provide context and motivation for the need by answering why this is important for our customers and the success of our product. In this phase, there are obviously multiple iterations and customer testing is a critical part of the work. In our organisation, UIXes, Product managers and Subject matter experts are key people in this phase. We also prefer to include the software architect in the phase. This is to discover dependencies across multiple services as early in the process as possible. The work items being evaluated in this phase are first and foremost business requirements. Internally we refer to these as VIGT’s.

The target is to scope the deliverable to be possible to deliver to production in four weeks or less. The decided time-constraint added on the VIGT, puts a lot of attention towards breaking the delivery into small pieces. One of the challenges we have seen internally, is that it is easy to gold-plate and increase scope on any deliveries – not any different from most other software organisation, I guess? The time-constraint added to the time-to-production, also adds a new challenge onto Product management – when is the feature really ready to be released to all customers? An important part of this phase is also for Product management to start planning how to launch the new feature to the market. This consideration also aligns well with feature management best practices.  

Design

Having multiple software teams delivering the same product offering may lead to the high need for coordination between the software development teams. Such coordination tends to be time-consuming and really hard to get right. The key target in the Design-phase is to agree on what teams do what part of the business requirement. This is first-and-foremost a technical discussion about the architecture of the new deliverable. The Business requirement (VIGT) will in this phase be divided into one or more EPIC’s. Each EPIC belongs to one and only one development team. 

As we are based upon a microservice architecture, the API design on the microservices are also a critical element in this phase. Well-defined APIs are essential to allow the teams to be more independent of each other while developing the feature. Agreeing on the APIs also allows the teams to discuss between themselves how the customer experience will be properly handled. We also see a huge benefit using this phase to coordinate the development of the same Business requirement. Striving for efficient software development, our goal is to have a few parallel Business Requirements in delivery as possible.

Implementation

Best practices for how to efficiently implement the software by scrum-teams is highly documented. Moving towards true DevOps, KPI’s in our processes is the number of deployments to production and Lead time to production. These are identified as key metrics for Elite performers in the State of DevOps report. Alexander Lystad has also written a great blog-post on how Visma is handling DevOps in this Blog-post.

Summary

More than ever before, the user experience is crucial in the software business. Modern SaaS applications are often developed on top of a microservice architecture. In our situation, our experience is that the modularity a microservice architecture provides makes it harder to provide a holistic product experience. Our approach to this challenge was to:

  • Ensuring one transparent product backlog including both functional and non-functional needs.
  • Early in the product development spend effort on prototyping and customer validation on the planned new features.
  • Simplify the process, and make sure the customer has a critical position in the process.

Read more about technology

10