Is AI more valuable than the E-commerce professional?

webshop

Artificial Intelligence. Unless you spent the last two years underneath a rock, you will have noticed that A.I. is at the verge of taking over the world. Or is it? In E-commerce it has become more common that different products are displayed at different moments. Is that A.I.? Do we no longer have to instruct the systems what to do, or are they not quite autonomous and is the E-commerce professional the real difference maker? 

How do you leverage technology to enhance your value as an E-commerce professional? If you don’t ask yourself that, chances are that you are in service of technology rather than the other way around. Not unlike the fireman feeding coal to a steam engine or the switchboard operator who was literally connecting callers. Understanding how A.I. or predictive technologies work enables you to stay connected to techies. This blog discusses how A.I. and predictive technologies are applied in E-commerce and how they fundamentally work. 

The changing role of data in E-commerce 

Data, Machine Learning, A.I. Five year ago the noise was all about ‘Big Data’. ‘Big Data’ in E-commmerce was mainly concerned with increasing data quality, capturing large amounts of data and exploring and analyzing data. Systems that have debuted since, execute tasks to some extend autonomously and are driven by data. Technology providers from all corners meanwhile have jumped on the A.I. marketing bandwagon regardless of the underlying technology. But hardly any application currently in E-commerce is fully autonomous A.I. Although there are plenty of systems that make smart use of data and respond to context. 

When should we consider a system to be A.I.? The complexity of the task to be automated and the degree of autonomy given to the system determine how much intelligence is required. Simple tasks that require little autonomy, such as making a database selection are better categorized as data driven than as A.I. driven. Boundaries are grey though, as the degree of autonomy differs per application. A.I. after all should not take over unless it performs better and more efficient than a knowledgeable person. A marketer or E-commerce professional who has a lot of expertise about customers can point out relationships that are more interpretable with far fewer data points than a system can. Leveraging this expertise with the proper technology can be of tremendous value.

Below we’ll discuss various ways systems can work with data. We start with database selections, a frequently recurring task with little autonomy. Next we’ll discuss the opposite: systems that apply unsupervised learning that, fully autonomously, learn to deliver better E-commerce results. However, most of these systems are still in experimental stages. For E-commerce professionals, the sweet spot is therefore between those two extremes, in the domain of predictive systems. Predictive systems can execute complex tasks and are capable of reinforced learning via supervised learning. The make use of complex models that usually perform best having the knowledge, intuition and creativity of E-commerce professionals as input.

Once upon a time, there were .... database selections 

Database selections are the mother of datadriven E-commerce. For example, it is quite common to make a selection of recipients for an email rather than send everyone the same email. Selections can be made using attributes such as ‘has someone viewed an item of the Adidas brand’. A number of attributes can be combined and the email is sent to the recipients that satisfy all conditions. The task to daily check which recipients should be sent the email can be automated. This is often called Marketing Automation, a function provided by most email service providers. Functionalities for retargeting or personalization functions that are packaged with E-commerce suites principally rely on database selections: manually defining target groups and follow-up actions. 

Database selections fit seemlessly with the traditional top-down approach in marketing. Online professionals have discretion which actions, campaigns or promotions are important enough to create and automate selections for. The flip side of course is that many times, despite signals from visitors, the system can take no action because the marketer has not defined the trigger and the action. Furthermore, to increase the success rate (e.g. a view or add-to-cart), you should make the selections quite narrow. But that necessitates a lot of different selections and follow-ups and quickly becomes unworkable. Lastly, database selections have the important flaw that a recipient is either in or out of the selection. Suppose that a person has viewed an Adidas product: he will receive an Adidas promotion email. However, if he has viewed ten Nike items and only one Adidas items, a Nike promotion would probably be more successful. 

selection

Has someone viewed brand X? We will send that person an email promoting that brand.  

Deep learning, tomorrow here today?

Explicit modeling has the drawback that people overlook relationships or complexity in relationships that exist in the real world. Or they may emphasize relationships that are irrelevant. A trained marketer may place a lot of importance on brands, while in reality webshop visitors may base their decisions on reviews and speed of delivery.  Methods such as Deep learning can circumvent the problem of preconceived but incomplete or incorrect models. Deep learning, in short, makes various combinations and abstractions of data and is suitable for unsupervised machine learning. By processing actions and outcomes, the system learns which action leads to better outcomes. This way, two neural networks can learn to play Go games by playing matches against each other. Only the moves and outcomes are explicit. The system learns autonomously, without a preconceived model. Deep learning is applied in various areas where humans find it difficult to formulate a precise and effective model (selfdriving vehicles, recognizing images, spoken language). 

In theory, a system can learn to display products that maximize the probability of a purchase in a webshop. However, stand-alone applications of Deep learning in E-commerce have not been successful yet. One of the major issues is that it is not transparent what the system is exactly learning. Brand for example, may be a factor. More likely the system relates combinations and abstractions of various attributes. Those have a high explanatory value numerically, but lack the intuitive interpretability of factors that are rooted in motives and behaviors of customers. There is also no resulting model whose behavior under various conditions can be studied. You’ll have to do with the outcomes and scoring of the system on the total test set. But who can tell how the system learned that the image below is of a man riding a skateboard on a ramp? 

ramp

An experiment by Samim Winiger, Samim recently asked a neural net to caption a series of pop culture videos and clips from movies to illustrate the huge variance in how accurate these algorithms are. 

Autonomously learning systems are also sensitive to bias that may be present in training sets. Large, diverse and high quality training sets are important therefor. Also, dealing with gaps in the data is a challenge, in particular if the gaps are structural. If there is no data about customers who visit competitors’ sites after you displayed a product recommendation, the system can’t learn that those recommendations were actually of interest to customers.    Given these obstacles, it is not strange that fully autonomous learning for E-commerce is mostly confined to lab settings. A very promising development to keep an eye on - some publications already have appeared, is autonomously creating and adapting digital content for individual ads. In this application there is a very clear relationship between display and click through action. There are also many users to train the system on. Otherwise, A.I. models in E-commerce are mainly used to complement other models, for example ones that are used in recommender systems. 

Predictive, best of both worlds

Let’s take a step back, from autonomously learning systems to the first example about making database selections based on brands. A recipient who has a preference for Nike is going to receive an email about Adidas. You may argue ‘But I should know if someone has viewed ten items of Nike, isn’t that in the data? I should be able to do something with it’. Yes, the views are in the data and moving towards the domain of Predictive, there are various ways to do something with it. Predictive is a collective name for using data with statistical approaches. A simple data driven set-up approaches the view of a product as a singular event (has or has not viewed). Predictive looks at all the views and expresses the collection in various statistics. Brand relevance to a visitor may be expressed as a percentage of items viewed of each brand over the total number of items viewed. It may even be broken down to relevance scores per product category. 

histogram

The more often viewed, the more relevant 

Making an email recipient selection can become a lot smarter this way. Rather than applying the selection ‘which recipients have viewed an Adidas item’, the selection becomes ‘which recipients have Adidas as the most viewed brand’. Or ‘which recipients have Adidas as most viewed brand within sport shoes’. The next step would be to skip Adidas as a planned campaign and to have banners prepared of various brands. Each recipient will receive the banner of only their favorite brand in the email, directing their attention to the new models. Common applications of predictive in E-commerce include displaying relevant images, content and products for a number of commerce situations such as product overviews, upsell and cross sell recommendations and promotions. Predictive can also be applied in service oriented tasks, such as displaying on top the relevant FAQ topics, how-to content and topics on user forums. You can find examples of more use cases here. 

Anything you can do with brands, you can also do with other attributes: sizes, colors, etc. etc. Statistics of multiple attributes can be combined to predict what is of interest to a certain visitor. 

dots

 

Customer data: Frequently viewed items of Nike result in a big blot. An attribute such as size:medium is also expressed in a score. 

You can use the statistics of one particular visitor to give that person a personalized experience. When the Nike brand for this person has a 30% relevance score, your model may display Nike items higher in product lister pages. However, it is not unusual to have little data of say visitor A. This may be because you cannot relate various sessions to a specific individual. It is also possible that you simply know an awful lot about which products are viewed together in the same sessions and may be good substitutes for each other. That could be a certain Mizuno shoe for a Nike shoe. In those cases you like to leverage the information you do have to the maximum extent. For example, your data indicates that visitor A shares similar traits to visitor B. Products of Under Armour that visitor B has viewed and A not (yet) may be of interest to A as well. Or, you utilize the relationship between the shoes and show Mizuno shoes to visitor A when he has viewed the related Nike shoes. Utilizing relationships between persons or products is called collaborative filtering and is frequently applied in E-commerce. 

What is great about predictive is that expert marketers can guide the predictive models. In fashion, a marketer can tell you from experience that brand, size and color determine if a customer will deem an item to be of interest or not. In a different category, e.g. electronics, it may revolve around the number and quality of customer reviews. Experience and analyses of the marketing or E-commerce teams can actively steer the models for personalization in the right direction. Some challenges still remain of course, such as what are the appropriate weights for each attribute (feature). You can optimize them with the help of (supervised) machine learning. Another challenge for various systems is to apply realtime contextual data (who is the visitor, which pages has just been viewed, what is currently in the cart, what is the visitor’s location, etc.) because this puts a lot of strain on the performance of the system. However, it is very important because context sensitivity determines to a large extend how ‘intelligent’ users experience the system to be. 

Practically all modern decision engines and recommender systems that companies develop in-house are predictive in nature. The big differences are in the type of algorithms used, the complexity and the context sensitivity of the applications. Furthermore, realtime personalization is not just about good models, but also about good engineering to ensure that users get the results practically immediately. 

 

Technology enhances the value of E-commerce professionals 

Technological advance is great for E-commerce professionals. They can add tremendous value by understanding customers and their behavior and spread this knowledge throughout the organization. The importance of data and technology increases by the day though. The era in which messages had be oversimplified so that they could be conveyed and executed (we have four personas with these typical characteristics – and BTW we don’t really know who belongs where) is practically over. Expertise required of marketers to successfully have 1-on-1 personalization on online channels has shifted to understanding which behaviors of customers yield relevant information, where the information is stored in the data and how to apply various methods to display relevant content at the right moment. 

Real A.I. in E-commerce is still a couple of years away, but applying available data in smarter ways than your competitor is here and now.