What is feature management and feature experimentation?

Feature management is the process of managing how features are developed and released progressively to a select few user segments or controlled user populations before making them available to the entire user base.

In contrast, feature experimentation means launching different versions of the same feature (via an experiment like an A/B test or an A/B/n test or a personalization) to see which version performs the best. Feature experimentation also includes launching a feature to a controlled (and targeted) user population to validate if the feature should be launched at all.

Feature management frees you from the "all or nothing" release problem as each feature has a flag that doubles up as a kill switch if you want to turn it off. You can roll back any feature you need to without impacting the rest of the solution or features.

In this guide, we’ll first go over feature management and feature flags. Then, we’ll explain how progressive delivery has enabled developers to scale their feature testing, eventually building up to the current best practice of feature experimentation.

1 What is feature management vs. feature experimentation?

Though still not as mainstream as feature management, feature experimentation is the natural progression of feature management into its current best practice. Therefore, anyone who works on software products in any capacity needs to understand how it works and why it matters.

Businesses that start with feature management naturally graduate to feature experimentation as feature experimentation offers the means to launch better features (and products). They do this by implementing techniques like A/B testing, personalization and segmentation for their product deployment. Advanced A/B testing tools now offer comprehensive feature experimentation solutions.

2 What is feature management?

Feature management is a modern feature development practice that overhauls traditional feature delivery. It uses the build-test-launch model, where features are built, tested, and launched to all users, all at once. The problem with this kind of delivery is that features are planned and launched on the basis of what the product team wants rather than user input. Furthermore, when it comes to iPhone apps, every new version requires another round of approval.

The main problem with the classic product release model is the risk of failure. A technical issue or bug can be catastrophic when a new feature is released to everyone. Additionally, if a new feature does not live up to its promise, it is back to the drawing board from scratch.

Feature management reduces risk and allows quality control. Features are continuously deployed to the app in whatever state they are, but they only get launched when you're ready and are only made available to the users you want to make them available.

Simply put, feature management gives your engineering and product/marketing teams granular control on building and launching new features (or updates). Less risk, more quality control.

3 What is feature experimentation?

Feature experimentation means deploying low-key feature launches where your engineering and product teams see whether features perform as intended. They assess if the features meet their goals.

Studies support the benefits of feature experimentation. For example, about 50% of the respondents in a 2021 Forrester survey called feature experimentation very important in their software development initiatives. More and more businesses are looking to move toward hypothesis-driven development. They only want to invest in features and product experiences that will positively impact their KPIs.

When Microsoft reviewed its online experimentation program, it found that for every 10 development ideas (like new features):

• 1/3 of new features can offer good results
• 1/3 can prove flat and fail to move any metrics
• 1/3 can negatively impact the target metrics.

Even at Amazon, where every feature is tested, the success rate stays below 50%. Feature experimentation enables you to find (and invest in) those features that will meet your business goals.

4 How feature management works

Feature management is a modern way to approach feature delivery — and by extension, software delivery. With feature management, businesses launch features and feature design improvements "progressively." They ship features when they're ready but don't enable them for all users by default.

Access is allowed to only a few select users. Once the features are tested (and validated), they're made available to the entire user base. Since the features are already tested in the live app by this time, there's no stress or anxiety around the launch. As such, it doesn't even count as a "launch" at this point.

5 Benefits of feature management

While your features are still in the development or in beta mode, feature release cycles enable you to:

• Find bugs;
• Catch performance issues; and
• Collect user feedback.

Looping these in lets you build better features, and ensure that only the best version reaches your full user population. You can also roll back your features if they're buggy or seem to impact the user experience negatively. Also, feature development and releases are decoupled from the general software development and release lifecycle with feature management.

With feature management practices in place, your IT team can keep working on multiple new features and adding them to the live app — just in an inactive state. It helps you get the innovation edge by letting you ship features faster.

6 Feature flags: The cornerstone of feature management

The main staple of feature management is feature flags. Feature flags (also called feature toggles, feature switches, and feature flippers) are if ... else logic code snippets that let you enable or disable features.

Feature flags offer a way to control the behavior of an application. They let you turn features on or off at runtime — enabling targeted feature delivery in real time.

The creation of a feature flag should be very simple.

1. Name your flag and associate it to a site(s) and add any relevant tags to easily catalog it.
2. Configure it.  Give it a unique ID so it can be identified in the source code. Add Variables depending on how you want the flag to affect a feature. Choose the SDK and the provided sample code to set up the SDK in your application. 
3. Choose the KPIs or goals that will determine if the flagged feature is successful or not 
4. Set which and how many of your users will see the flagged feature i.e. targeting.
5. Schedule when and how you want your feature to be released. Set up automatic alerts to let you know how it's performing.

If conditions for delivering a feature are met, then you make it available. Else, you don't.

 

The targeting conditions of these flags can be as simple or complex as your feature delivery needs. For example, you might want to deliver a feature to just 20% of your female US user base. Or to only 10% of your enterprise customers from Asia. Or to your entire beta user group. Your feature management solution needs segmentation capabilities to execute such targeting.

Targeting feature flags is done by segmentation. Advanced feature management and feature experimentation platforms like Kameleoon give product and dev teams many native segmentation criteria. They can use the segmentation criteria to target precise users based on their behavior, acquisition, geolocation, and much more. Kameleoon also enables brands to easily connect to their customer data platform (CDP) and add unlimited custom data to their feature flag deployments.

Teams can monitor the performance of their features with client-side reporting dashboards. In addition to their analytics solutions, product and dev teams can use client-side reporting tools and metrics to see how these server-side feature rollouts are performing. This is often described as "hybrid" feature flagging. 

7 How feature management enables progressive delivery

Feature management is one of the key modern software development best practices that comprise the umbrella term "progressive delivery". In fact, feature management is the key enabler for progressive delivery, especially as it relates to feature delivery.

 

 

Feature management allows businesses to keep adding features to their solution without actually deploying them. Features can be shipped to the production environment (the live app/solution) at any stage of their development. These features can be just 10% coded or only have their back-end ready or can be there without being tested at all. Feature management lets you progressively ship all the features in your product roadmap while you're still building them.

But by enabling progressive delivery, feature management improves the software delivery speed — a constant struggle for business leaders. This explains why businesses that use feature flags are able to add more features to their web product and/or mobile app (30 new ones each application each year, according to Atlassian) than the others.

Feature management lets you decouple your IT team's development lifecycle from its efforts on driving innovation. Features keep getting rolled out — in inactive states — without impacting the product's stability. You can allocate separate IT resources for both functions. Your feature team could work on driving innovation, whereas most of your IT or software development bandwidth can go into "keeping things running," where it typically goes.

Feature flagging also streamlines development for apps. It enables businesses to implement continuous delivery even for their mobile app experiences. That’s because toggling feature flags allows developers to control the behavior of the app, like enabling a feature for the beta user group, without going through an additional round of the App Store review process. This allows developers to start integrating testing into their feature deployment—leading to optimized experiences for users. Read on to see how testing works within the framework of feature management.

How A/B testing became part of feature management

 

 

Modern feature management brings A/B testing to the live production environment. As feature flagging works seamlessly with canary and blue-green deployments, it enables you to release multiple new features (or experiences) at once to targeted user population groups. Real users use (test) your features on your live app.

What is A/B testing? A/B testing uses experimentation practices that allow businesses to remove uncertainty and gut feeling when making marketing or design decisions. While this has previously been mostly used for websites, ads or other digital campaigns, it’s now possible for software products and their features.

A feature experimentation solution like Kameleoon can even automatically roll back a feature when certain conditions get triggered. Product managers can even create scenarios such as rolling back a feature if it negatively affects a direct or tangential KPI(s). Teams don’t have to wait for negative (or positive) results before taking action.

Also, the risk factor is significantly lowered with feature management as you only expose your new features to controlled user populations in such testing and deployments. This assists with modern DevOps software development cycles and makes them safer. Stable releases remain a concern for even the top business leaders that also happen to report a change failure rate of up to 15%.

8 The growth of feature experimentation

Feature experimentation is a use case of feature management, just like progressive delivery and testing are. But, feature experimentation is also the future of feature management.

 

Feature management sets the foundation for feature experimentation, which, in turn, changes how you develop features. Feature experimentation forces you to think about how you're going to measure the value of a new feature, which helps build more effective product roadmaps.

Think about it:

Why would you want to develop a feature when you aren't sure if your users need it? Or when you don't know if the way you implement it is indeed the most optimized version? Also, why would you build something with no way to measure its impact?

While feature management can help you deliver a feature or a product experience safely, feature experimentation gives you the data to decide if you should build it at all.

It's the most natural development practice to follow. We could go so far as to say that feature experimentation is a better way to manage features, because it helps businesses use their development resources only on features that they know will work. Now, you might say that you still are coding the features you're experimenting with. Agreed: you are.

But the real savings from such experiments come from NOT having to roll back features that hurt your KPIs. Plus, with feature experimentation, you avoid bloating your app with features that make no difference to your product experience. Imagine having to roll back six of every ten releases you make because of these reasons.

Feature experimentation helps you move toward a “hypothesize-driven” software development model—a model where you hypothesize new features based on your customers' needs and behaviors. You're not only looking at requirements as you would in traditional software development.

Also, feature experimentation accommodates both the IT and product teams in the feature development lifecycle. Once an experimental feature shows positive impact and the technical team gives it a go, the product team can take over. The product team can decide when they want to launch it and for which users. They can also plan marketing campaigns based on your feature release schedule.

9 Who needs feature experimentation?

Feature experimentation lets you shift the focus of your software and IT team from just keeping things running to driving innovation. A few benefits:

Optimizing your development bandwidth: Companies like Microsoft are all for launching early prototypes for experimentation purposes. Combined with targeting, this saves a lot of in-house development resources that get spent on coding, QAing, and deployment. For example, if you launch a feature for your web app for your Chrome users and it fails, you've wasted zero efforts for the Safari browser. Businesses save precious in-house development resources when they're able to discard feature ideas that fail to move the metrics or offer any tangible improvements.

Building better features: Feature experimentation allows businesses to compare features side by side with data pointing to the best implementation. This allows only the best features to reach the entire user base. As features are released progressively to controlled user groups, businesses get to capture user feedback in the interim and relay it back to the development team. This way, the improved or enhanced features that finally roll out for all offer the best possible user experiences. Indeed, you need to code all the versions, but experimentation gives you the data to launch the "winner" -- the feature version that you know to work the best (both in terms of performance and business).

Feature experimentation helps validate features and measure their business impact: Feature experimentation helps tie feature releases to goals that, in turn, have a direct bearing on business metrics. Feature experimentation solutions like Kameleoon easily translate the impact of such experimentation to a business's key KPIs. Validation also inspires a culture of experimentation. Brands can feature experiment i.e. work server side, and with a feature experimentation tool, view how the feature is performing using client side analytics. This hybrid approach helps marketers, product managers, and developers all work together to ensure their optimizations are positively affecting business KPIs.

How feature delivery looks like with feature flagging, feature management, and feature experimentation

Basically, you go from ad hoc feature releases to hypothesis-driven feature development with these practices.

Feature flagging: Loosely, starting with feature management means starting with feature flags. Most businesses (95%) are 1) already using feature flags, 2) in the process of implementing it, or 3) planning to implement it. At this point, most businesses generally experience slower timelines from the idea inception to production, however it is the first step.

Feature management: Here, your IT team can release features within well-defined release timelines. You're doing reasonably well with feature management if you can deliver a regular feature release cadence. You may not be shipping as fast as your business needs, but you'd still have a solid structure in place that you can build upon over time.

Feature experimentation: At the most mature level, a business progresses from feature management to feature experimentation. Here, you use feature management to ship innovation. You're at hypothesis-driven feature delivery in this case: You write a hypothesis for a feature, set up an experiment to launch it, measure its performance, and then launch it post validation. Experimentation is a part of your feature delivery cycle. This is the hallmark of innovative software development teams.

Final goal: In the end, your north star should be that all your feature development is based on solving customer pain points while also having clear proof that it is successful. With experimentation, your feature development goes from build, test, launch to hypothesize, build, experiment, validate, and launch:

10 A word of caution about technical debt

Feature flags that you use to implement feature management and experimentation can quickly spiral out of control. They're just if...else logic snippets in a way, so they're easy to add.

But the more feature flags that you add to your code — for better and more granular feature management — the more technical debt you add to your codebase, especially when you don't think through your feature flag's lifecycle.

In fact, if your organization has a culture of experimentation that encourages extensive testing, your technical debt will quickly grow to the point you cannot ignore it. That’s why Kameleoon helps you in managing your technical debt by alerting your team when feature flags have not been requested for a long time.

Remember: Once a feature flag serves its purpose (for example, when its feature goes through its entire lifecycle from QA, testing, and experimenting all the way to production), it is essential to remove the flag from the code.

11 Types of tools for feature management and experimentation

Because feature management and experimentation are such connected processes — and because businesses clearly need both to create better products — some feature management solutions are attempting to add on experimentation capabilities. Conversely, some experimentation vendors are expanding their functionalities to include feature flagging and management.

Forrester recommends an experimentation solution that now also supports feature flagging and management as the top choice.

The reason?

The science and practice of experimentation is highly complex and caters to marketers, product managers and developers alike. The need of each user group in regards to experimentation is extremely unique, unlike feature management, which caters more to developers and engineering-savvy product managers.

Marketers and many product managers require client-side analytics that reveal how campaigns and funnels are affecting business KPIs. They will also want to experiment with both the front-end (JavaScript and modern JS frameworks such as React) and the backend. Feature management tools are mostly built around back-end developer needs (providing SDKs, IDE support and more), but they often lack critical capabilities in terms of analytics and data reports.

On the other hand, the best experimentation platforms seamlessly integrate with your current tagging plan (which 95% of the time is implemented in JS on the front-end) and allow you to immediately analyze the business impact of your ideas and planned features.

Also, advanced experimentation solutions already offer a full-stack solution, which allows users to experiment with product features and functionality. Hence, adding feature management and feature flagging, which occur server-side, is a natural progression. Said another way: It's easier to move from feature experimentation to feature management than the other way round. Kameleoon is a full-stack A/B testing and personalization solution with feature flagging, feature management, and feature experimentation capabilities.

12 Recommended reads

Feature experimentation meets feature management

How server-side experimentation deepens testing programs

3 progressive delivery concepts everyone needs to understand

 

If you'd like to learn more about Kameleoon's Full Stack solution, which includes all the feature management, feature flagging, and feature experimentation described in this article, please contact us.  Or, if you're ready to see how we can help you build better product experiences, then Request a demo. We're happy to hear from you.