This article is about a very important question today’s innovators face when they develop new software products.
The question is:
“What are the appropriate technical skills, technology stack and architecture for the new product?”
This question is very hard to answer because, very often, it is not clear what features a product should have. Sometimes, it is not even clear what problem the company is trying to solve for its customers.
From startup to scale-up
Consider a development team that follows an iterative approach to test fundamental hypotheses about the product, the business model and the market.
Initially, during the startup phase, the team will attempt to build, test and adapt an MVP through an iterative process. With each iteration, the team will improve the MVP and adapt it to reflect the feedback received during testing.
Once the team verifies its fundamental hypotheses and validates the business model, the startup phase concludes and the scale-up phase starts. If the team disproves or fails to verify its hypotheses, it will attempt to pivot the strategy, correct its course, formulate new fundamental hypotheses and repeat the process.
By the time the team completes the MVP and validates the business model, the product may have gone through several fundamental changes and most likely already has many users. The next step is to scale up, roll out the product to significantly more users and enter new markets. At the same time, the team should continue operating the product for existing users.
Technology dilemmas
Following the above approach creates conflicting expectations regarding the appropriate technical skills, architecture and technology stack. The reasons are:
- The problem to solve and the solution are not always clear from the start.
- The requirements are discovered gradually and they may change after each iteration.
- Speed is of the essence. The hypotheses have to be verified ahead of the competition and before available funds are exhausted.
- Pivoting the strategy may happen at any time and the technical requirements may dramatically change.
- To solve the problem, the team may need skills that are initially unknown.
To put it another way, the technology dilemmas can be described as:
“A startup needs a technology strategy that not only allows for the fast delivery of value and supports growth, but can also fulfil unknown, diverse and possibly contradictory requirements.”
To put the technology dilemmas in a familiar and tangible context, the following two examples are based on actual situations that startups have encountered.
The e-commerce company
The first example is about an e-commerce company that operates in the B2C market.
Startup phase
Their product idea is great, but they have to gain significant market share very quickly because competition looms. The team decides to use a simple architectural pattern and an on-premises hosting solution. They make this choice because they are very familiar with these technology options and feel that this is the fastest way to build the product. Very soon, the first version of the website is ready, and the team continuously develops and tests its assumptions about the market.
Scale-up phase
After a couple of years, the user base in the company’s home country has grown significantly, as has the codebase of the B2C web application. It is now time to grow even further, enter new countries and expand into the B2B2C market.
Sadly, the team realizes that the application cannot handle the new traffic load and regularly crashes. To make matters worse, implementing requirements for other countries and the B2B2C market turns out to be very difficult. Their web application—a monolith with spaghetti code—cannot be easily scaled or extended.
The team now feels that it should have chosen a different approach to building the web application, but it would not have been able to move as quickly as required in the beginning. The team now proposes migrating the application to a modern microservice architecture and moving it to an external IaaS/PaaS.
However, this will require considerable time and effort, which the company cannot afford because the competition is catching up.
The Industry 4.0 startup
The second example is about a startup that builds a SaaS product for the Industry 4.0 market.
Startup phase
Its business model assumes that several thousand customers and tens of thousands of users will use the product concurrently.
The development team is excited and chooses a microservice architecture for the product. Work begins, and the team develops the product on the IaaS/PaaS of a major provider. The team is not familiar with these technologies. Progress is slow at the beginning, but its members learn fast. With each iteration, they incrementally improve their MVP. After several months of work, they begin testing it with users and potential customers. Everything appears to be going well, and they are finally making good progress.
Unfortunately, after a while, the team realizes that its target customers are not yet willing to adopt a SaaS solution. There are good reasons to believe that, in the future, customers will accept a solution running on a third-party cloud, but for now it is too early. Most potential customers have concerns because they plan to process intellectual property using the product and are not willing to allow this information to leave their own infrastructure.
Scale-up phase
The product team decides to pivot. Instead of offering the product as a service to thousands of customers and tens of thousands of users, it decides to offer it on-premises to a handful of large customers and later to a few hundred mid-sized customers. Consequently, every installation must take place on-site, as remote access is not always possible. Furthermore, because each installation is customer-specific, it will have only a few hundred users at most.
The team is now facing a difficult technical challenge because the requirements have fundamentally changed:
- The selected architecture is too complex for the new business case.
- The new requirements involve deploying and updating the application for multiple customers with diverse infrastructure. However, the technology choices the team made tie it to a specific IaaS/PaaS provider.
Re-architecting the application is possible, but it will require considerable effort, and the team still needs to continue developing new features for existing customers.
Modeling the technology dilemmas
Modeling the technology dilemmas can help teams better understand their dynamics and develop strategies to tackle the associated challenges.
The value a new product creates resembles an S-curve. Initially, as the team tests various hypotheses and builds the MVP, value creation is relatively flat. Once the right strategy is found and the business model is validated, the potential for value creation grows exponentially. To capitalize on this potential, the scale-up phase starts. Ultimately, as pressure from competition increases and the market becomes saturated, growth flattens out again. If it were possible to choose the perfect technical skills, technology stack and architecture, the development team would be able to develop the MVP and validate the business model quickly during the startup phase. Furthermore, development could transition smoothly to a scale-up phase, and the team could deliver in a way that optimally capitalizes on the exponential growth potential.
The imperfect technology match
In reality, because of uncertainty about the requirements and pressure to deliver, technology choices cannot always satisfy the conflicting demands of developing quickly enough to validate the strategy while retaining the ability to scale up.
The need to keep changing the MVP in response to market feedback and delivery pressure can lead to quick fixes, shortcuts and poor engineering choices. In other words, technical debt builds up. In addition, a pivot may set an entirely new direction for the product that is not compatible with the current technical solution.
As a result, product development cannot keep up with the demands created by the exponential growth potential. Technical debt constrains growth, and value creation flattens. The business encounters difficulties scaling up.
The technology pivot for scaling up
To avoid a premature plateau in growth, the development team may need to perform a technology pivot. The appropriate type of pivot—for example, re-engineering, migration or a rewrite—depends on the specific situation the startup faces. The point at which the team validates the business model may provide an appropriate opportunity for the pivot, as this is when it becomes evident that value creation has the potential to grow exponentially.
If the pivot is successful, the product will capitalize on its exponential growth potential. However, many teams have difficulty making this decision. Teams hesitate because the decision is counterintuitive: the pivot may temporarily delay growth at a moment when growth has, for the first time, demonstrated an exponential trend. In the model shown in Figure 3, pivoting promptly pays off and outperforms the decision to retain the existing technology choices.
Common pitfalls when pivoting the technology
What are common pitfalls when it comes to pivoting the technology, besides failing to recognize the need to pivot?
Delayed technology pivot
The first common mistake is delaying the pivot. Doing so may seem to be a better option than retaining the current technology choices. Even so, this decision will eventually perform worse than pivoting when the growth potential becomes exponential.
Slow technology pivot
Another danger is that if the pivot takes too long to complete, it will fail to support the growth potential. As a result, the product will miss the chance to scale.
Cumbersome technology choices
Finally, teams may be tempted to choose technologies optimized for scale from the start of development—for example, by choosing a highly scalable but complex architecture or a heavyweight technology stack. They may do so in the hope of avoiding a later pivot. However, this will most likely slow the development of the MVP and the validation of the business model. Figure 6 shows how such a choice can again lead to suboptimal growth. Demonstrating the feasibility of a scalable technical solution is vital, but implementing it too early is not.
Epilogue
This article described the technology dilemmas that innovators face and presented a model to explain how these dilemmas affect product development.
The next step would be to answer the question raised at the beginning of the article:
“What are the appropriate technical skills, technology stack and architecture for developing the new product?”
As already mentioned, this is a hard question to answer. Fortunately, the model presented here offers some insights that can help develop ideas to answer it.
The next step is to build on these ideas and develop a technology strategy for innovation.