Progressive delivery

What is progressive delivery? 

Progressive delivery is a shift in the way companies go about software delivery. It allows them to move fast by releasing code to small portions of their traffic and improve confidence in product decisions by testing their hypotheses with real customers.   

Earlier practices like agile and continuous delivery involved deploying changes to the entire user base simultaneously. Progressive delivery takes a different approach with its cloud-native capabilities. It validates quality and performance in production with feature flags.  

Instead of deploying new features or a new code to everyone, companies using progressive delivery techniques release them incrementally to a subset of their user base. It focuses on feature management and releasing changes via gradual rollouts, targeted feature flags, and A/B tests, A/B/n, etc., to control the release process and gather insights.   

This process helps product and engineering teams reduce uncertainty, make data-driven decisions, and deliver the right experience to end users faster. They gain confidence in their product decisions before making significant investments, avoiding costly rollbacks or hotfixes. After all, engaging customers with specific features and products ultimately drives retention and increased revenue for businesses.    

At the core of this practice is a product experimentation platform that gives teams the ability to control the release of new features to production, decouple deployment from feature enablement, and measure the impact of those changes with real users in production. With feature experimentation and feature flags, development teams gain the confidence of knowing they’re building the most impactful products and features because they can validate product decisions well before expensive investments are lost in pipelines. 

How does progressive delivery work? 

It’s no longer enough to just ship small changes quickly with continuous deployment strategies. By going from a foundational understanding to learning how to avoid pitfalls in process, technology, and strategy, teams can successfully deliver better software, products, and growth faster. 

An example of a progressive feature delivery could look like this:     

Team → Beta → Staged rollouts → Everyone   

It might include feature granular rollouts such as canary releases, A/B testing, and more, allowing teams to work more effectively while reducing risks. Here’s how it helps. 

1. Feature flags  

   Feature flags allow for granular feature rollouts. It means that new features can be released to a small percentage of users before being rolled out to everyone. This approach helps to identify any issues before they affect all users. 

2. Canary testing 

   Canary deployment is another important aspect of Progressive Delivery, as it allows for testing in production with a small percentage of users. 

3. A/B testing 

   A/B testing allows for experimentation and iteration on functionality and user experience. By testing different variations of a feature with different groups of users, teams can determine which version performs better and make data-driven decisions. 

4. Observability 

   Observability helps in monitoring and optimization of performance. This means that teams can quickly identify and resolve any issues that arise, ensuring that users have a smooth and seamless experience. 

Progressive delivery is a powerful approach to software and feature releases that helps teams reduce risks and ensure users still have the best possible experience. 

Why every team needs both progressive delivery and experimentation 

Every team needs both progressive delivery and experimentation to drive innovation, improve user experience, and stay ahead of the competition in the world of automation and modern software development. A combination of both can give you an efficient system for validating both quality and customer engagement across your development lifecycle.  

This approach avoids the need to decide upfront whether you are planning for a new release, to experiment, a phased rollout, or simply a "don't release it yet" flag. With this approach, you can start with a basic feature flag in your codebase and in time, turn it into a product decision that requires a full-blown A/B or multivariate test, allowing more sophisticated techniques for the features that need them. 

As software and feature delivery continue to become more complex with higher stakes, progressive delivery is quickly becoming the go-to technique for continuous integration. It offers a way to balance speed and agility with the need for reliability and stability, allowing teams to deliver features that meet the needs of their users.  

It is the best way to release features and updates confidently to users in a safe and controlled manner. 

Progressive delivery example experimentation flow 

As with any new process or platform, incorporating progressive delivery and experimentation into your software development and delivery process can bring up many questions for engineering and DevOps teams:  

• How can we start with feature flagging and A/B testing without creating technical debt down the line?   
• How can we scale to thousands of flags and still retain good governance and QA processes?   
• How can we adopt these new practices organization-wide without slowing down development and delivery?  

Here is a simple example workflow to get you started:

1. Generate insights 
   

   Before implementing progressive delivery, gather insights about user behavior and application performance. Use monitoring tools, user feedback, and analytics. 

2. What’s your hypothesis 
   

   Based on the insights gathered, define a hypothesis for the new version you want to test. Have a clear definition of success, such as improved user engagement, faster page load times, or increased conversion rates. 

3. Setup experiment 
   

   Define the criteria for success, such as the percentage of users in the experiment, the duration of the experiment, and the specific features or changes that will be tested. 

4. Run experiment 
   

   Launch the experiment by rolling out the changes to a small percentage of users. Monitor the results and make adjustments as needed. 

5. Analysis 
   

   Analyze the test results to check if the hypothesis was successful and it reached statistical significance. Use quantitative data, such as user engagement and conversion rates, and qualitative data, such as user feedback. 

6. Sharing and learning 
   

   Share the results of the experiment with stakeholders and team members. Use the insights to inform future development decisions and improve customer experience. 

7. Progressive rollout 
   

   If it was a successful experiment, gradually roll out the changes to a larger percentage of users. Monitor the results and make adjustments as needed. If it was a failed test, re-evaluate the hypothesis and make changes before launching a new experiment. 

Benefits of progressive delivery 

The progressive delivery methodology has several benefits and use cases that can help teams and individuals, especially product managers improve collaboration, reduce risks, and validate quality and performance in a production environment.

1. More effective team collaboration 

   By breaking down the delivery process into smaller, more manageable stages, teams can work together more closely to identify and address potential issues before they turn into problems. It aligns everyone to help them work towards the same goals, ultimately leading to a more efficient and effective development process.

2. Reduces risks in releases 

   By rolling out new features and updates in stages, teams can identify and address issues before they impact all users. It minimizes downtime, reduces the risk of data loss or corruption, and prevents other potential problems from larger, more complex software releases.

3. Validates quality and performance in production 

   By testing new features and updates in smaller, controlled environments, teams can monitor performance metrics and user feedback to identify challenges before they impact a larger audience. It verifies that new features and updates are working as intended and meeting user needs.

4. Targets specific audiences through multiple stages of release

   By rolling out new features and updates to a subset of users first, teams can gather feedback and make adjustments before releasing it to a wider audience.

By adopting progressive delivery, teams can create better features and software products to drive business success in real time. 

Progressive delivery success stories 

These A/B testing and progressive delivery case studies show the results the world's leading digital brands have achieved using Optimizely Feature Experimentation.