Companies need capabilities to evaluate the customer value of software-intensive products and services. One way of systematically acquiring data on customer value is running continuous experiments as part of the overall development process. This paper investigates the first steps of transitioning towards continuous experimentation in a large company, including the challenges faced. We conduct a single-case study using participant observation, interviews, and qualitative analysis of the collected data.  Results show that continuous experimentation was well received by the practitioners and practising experimentation helped them to enhance understanding of their product value and user needs. Although the complexities of a large multi-stakeholder B2B environment presented several challenges such as inaccessible users, it was possible to address impediments and integrate an experiment in an ongoing development project.

Developing the capability for continuous experimentation in large organisations is a learning process which can be supported by a systematic introduction approach with the guidance of experts. We gained experience by introducing the approach on a small scale in a large organization, and one of the major steps for the future work is to understand how this can be scaled up for the whole development organization.

Sezin Gizem Yaman, Fabian Fagerholm and Myriam Munzero (University of Helsinki), Jurgen Munch (University of Helsinki, Reutlingen University) Mika Aaltola and Christina Palmu (Ericsson), and Tomi Männistö (University of Helsinki): Transitioning Towards Continuous Experimentation in a Large Software Product and Service Development Organisation – A Case Study

http://link.springer.com/chapter/10.1007/978-3-319-49094-6_22