Solita developed a data science method in the N4S-program that helps the sale of different magazines increase by nearly a third.
Selecting the product assortment for sales outlets is challenging work. Traditionally, the purchase decision is based on experience and categorization of the products according to their expected target audiences. However, it is nearly impossible to know in advance how many copies of a certain product will be sold. This is particularly difficult when the assortment consists of periodical products, such as magazines. Should there be one or a hundred copies of some particular magazine? Is it a good idea to sell another particular magazine at all?
IT company Solita, specialized in data analytics, developed a method where the magazine sales are forecasted based on other similar magazines in similar magazine sales points. With this method, it is also possible to predict the sales numbers for a new magazine if it were added to the assortment.
If there are 1,500 items in the product assortment for the supply chain, what should be selected if there is only space for 250 products? Using Solita’s method, it is possible to predict the future sales of the magazine.
Which to choose?
“The method can predict new products for the assortment with moderate success. Preliminarily it would seem like the magazines found with this method should be added to the assortment. If, simultaneously, the same magazine is removed from a poorly selling sales outlet and the sales are compared, the sales increase by 29 %. The assortments are already highly optimized. Our approach serves as a good addition, as the model is completely different from the current ones,” states Senior Software Designer Timo Lehtonen, developer of the method.
Self-organizing maps in forecasting sales
The method was born when Solita decided to find out what could be extracted from the existing magazine sales data. The method takes into account the sales data of other similar products, which are modeled and visualized. This facilitates decision-making.
Solita introduced a method using self-organizing map (SOM) to optimize a supply chain of single-copy magazines in Finland. The self-organizing feature map (SOM) is an unsupervised learning algorithm which brings the original high dimensional data into two or three dimensions and, thus, available for visualizing. The basic idea of the method is to keep the relative distances between data points comparable. The SOM algorithm divides the data points into nodes. Each node holds data points that are similar to each other. Also points in nearby nodes have similarities. Solita uses SOM, since Solita is specifically searching for similarities between data points: either similar products or similar sales outlets.
In this method, the data is clustered visually, and thereby a map of the data streams for the sale of different products is obtained. The visualizations help demonstrate problems and the factual relationships hidden in the data mass. This was done with the help of a self-organizing map, SOM. The map highlights essential factors in the data, which are visually grouped in an order corresponding with their relations.
Modeling similarities is based on the well-known neural network algorithm, i.e. self-organizing map. This breakthrough algorithm was developed by the Finnish professor Teuvo Kohonen in the 1980s. In the self-organizing map, statistical connections are turned into simple geometrical relationships between the elements of the multidimensional data set, which can be shown, for example, as a two-dimensional map.
“The SOM is very suitable for explorative data analysis. The method is visual and intuitive while the map remains a bit of a mystery: what is the actual significance of the distance and the place on the map? In any case, here the 4500-dimensional data organized itself automatically, and the result was new insight into the cross-sale of products, which a human being would not discover without the computer,” Lehtonen says.
According to Lehtonen, the method is unique and experimental. Compared to the traditional methods, the strength of this approach is that it can find assortment relations that are difficult to find using only prior understanding on the problem area. Thus, it can give suggestions that not achieved by existing methodology.
“It works well in the sense that it finds obvious gaps in the data mass that a human being cannot find. In a way, the idea of similarity is used for “neural network deduction” that serves to assist a person. In principle, co-creative intelligence is a good term. A person who, for example, visually browses gene changes in the field of medicine can browse the assortments visually, organized by a self-organizing map. Together, a human being and the computer can create more understanding.”
“The result was new insight into the cross-sale of products, which a human being would not discover without the computer”
N4S-program helped with the experiment
According to Timo Lehtonen, the method developed is a good example of the culture of experimentation that is supported by the N4S-program. The method was studied in cooperation with the Tampere University of Technology, which is involved in the program. With the help of Professor Ari Visa and Doctor Timo Aho, the importance of planning the model and experiment was emphasized in the Big Data research. Such research complements well the business competence in the corporate world.
“Without the N4S-program, the development work would have ended with the first experiment in 2014. With the support of N4S, it was developed further to the extent that currently we perform business-oriented experiments in cooperation with the client.”
Lehtonen believes that a cloud-based solution of the method will be created for Amazon Web Service.
“It is certain that in this event also other methods of machine learning will be tried. Even now, we offer many clients smooth data analysis with different methods to help with continuous decision-making. The most functional methods can be found with the long-term culture of experimentation.”
Cooperative work was done with Åbo Akademi University and JAMK University of Applied Sciences. Thirty students at JAMK tested the new tool and gave feedback on the user experience.
“All the innovations come from the north,” they say, and the self-organizing map is one of them. It was good that Kohonen’s SOM_PAK code was being shared online. The 25-year-old C code was compiled and worked as good as new! Finnish technology companies and universities will make innovations regarding the application of such methods. As a research field, Data Science is good because, in principle, everything new that is developed will be utilized.”
Solita’s open source code algorithm is freely available at Github.
Teuvo Kohonen’s book on the SOM maps he has developed is also freely available for download online. It demonstrates how the Matlab programming language is used to write SOM algorithms.