1. What are Recommender Engines/systems

Firstly, we need to understand what a recommendation system really consists of. Let’s take a real-life scenario of who uses recommendation systems in their infrastructure/product/services and understand the intuition behind it. Probably you may have encountered the option called the “suggested video list” in YouTube; that displays more related videos suggesting to play next. This feature is built using a recommendation engine/model, and implemented as a content recommender system attached to YouTube as a service.

The common norm is that a recommendation engine provides product/content suggestions for an end-user. However it can also function as a system that provides suggestions of end-user profiles to the product/content.

As an example let’s imagine that you are doing a marketing campaign/email marketing campaign for an upcoming product. You will need to target the correct set of the audience, otherwise you’ll end up spending more money than you’ll actually have to for the digital commercials/ads. The idea is to refrain from showing the ads to those who do not wanna buy your product and solely target the right audience.

In a scenario like this, the recommendations systems are used to target the correct audience. The ad platforms like Facebook have strategized the use of recommendation systems heavily in their business.

Recommendation systems are all about personalization. It’s about taking your product that works for everyone and personalizing it for an individual user which leads to a better user experience and match.

2. What are the types of Recommendation Systems

If you plan to implement a recommendation system to your system, there are four types of recommendation systems like; **content-based filtering, collaborative filtering, knowledge-based system and hybrid recommendation systems **

1. Content-based Recommendation

The main idea behind content-based recommendation systems is to recommend items to a customer similar to previous items rated highly by the same customer For an example: in movie recommendation system it might recommend movies with the same actor or actors directors genre, or news article recommendation systems that recommend articles with similar content or on the simple similar topics user may previously read about.

One of the drawbacks of content-based filtering that needs attention is the lack of ability to learn user preferences using the actions of the users on one particular content and if it’s able to replicate the similar approach across multiple other content types. If a recommender is limited to showing the contents that are similar to the same content a user is using, it may limit the business benefit of the recommender, as most businesses expect to have the ability to do the recommendations of different types of content for the same person. The best example of this is when a user is browsing through sport related articles, and then being able to suggest contents like music or videos that may really pique the user interest and drive them for conversions. Such a recommendation can really add value to the business.

Content based recommendation systems have many pros. They do not need the data about other users, instead it uses the data about the user of interest and then that data is used to recommend similar items. Such systems have the ability to recommend niche items, because the data indicates the users’ specific tastes which is not shared among other users. Therefore we can use such a recommendation to meet those users’ interest. However content-based recommendation systems have their cons such as the required domain knowledge needs to be taken by a human in order to label the movie genres. Apart from that it is always required a human to enter the text description of the items, to post the item picture, and to create the audio or video clip attached to the item. As you can see there is a lot of involvement by human experts in content-based recommenders who know their items very well. Also content-based recommendation systems tend to make only safe recommendations as they stay within a user safe bubble of the end in space. If the user has never expanded beyond this bubble, the content-based recommendation systems will only recommend similar things which will invariably end up being within the bubble or near the edge. There isn’t any information in a purely content-based recommendation system that can push a user outside their usual boundaries to explore items that they didn’t know they might actually like.

2 . Collaborative Filtering Recommendation

The basic idea behind collaborative filtering Recommendation systems is very simple. Let’s imagine that we have a user/customer X who wants to make recommendations. What you can do is to find a group of other users whose likes and dislikes are similar to user X. Lets this understand with example, suppose you are doing movie recommendations now this group of users you know like the same movies that X likes and dislike the same movies that X dislikes we call this set of users the neighborhood of user X once we find the set N of users or the neighborhood of user similar to user X we find other movies that are liked by a lot of users in the set N and recommend those items to the user X .

Collaborative filtering is very powerful as there is no need to have domain knowledge. The data is automatically generated itself when the users are interacting with content. We can harness that information to predict other favorable user content interactions through a recommendation and targeting. This can be either an explicit feedback such as the number of stars, thumbs up, or like -dislike button. It could also be implicit feedback such as pageviews, duration of watchtime etc. The collaborative filtering can solve the problem of safe recommendations that is inherent in content-based recommendation systems. This is because not only the ability that the collaborative filter has to identify the user of interest points in embedding space, it can also tune in to other users points in embedding space and find similarity between them. For instance, think about movie recommendations where there are 3 users. Let’s call them user A, B and C. The user A might love sci-fi but has never even thought about seeing anything outside their genre bubble. With the help of collaborative filtering user A is found to be very similar to user B and user C due to their shared passion for sci-fi. However user B and C also both love fantasy and action movies. Even though these recommendations could fall far outside of the user raised bubble and embedding space, it can be a matching recommendation for them to check out. Collaborative filtering isn’t a totally perfect method and does have drawbacks. It mainly suffers from the cold start problem. The cold-start problem is when we have fresh items or fresh users; for instance when an item is interacted with a lot of users we will have a better idea of what type of users will like that particular item/content. When there’s little to no interaction data for that particular content we don’t really have a good idea as the user sample size is so small or non-existent. We could hopefully use the item embedding in such a scenario to look at nearby users and see if those users share similarities. However a lack of interaction data could be found in that particular instance as well.

3. Knowledge-based systems

Knowledge-based recommendations is a method where we either take data from user surveys or entered-user-settings that show users preferences. Assuming it is legal and ethical for your model to use user-entered-data, one way of doing this is; finding demographics like where they live, age, gender and using those factors to find similarities. As an example; with age it would be possible to determine what most children would likely prefer similar to other children’s preferences, rather than what the elderly individuals enjoy. We can also ask users of their preferences, when building knowledge based recommendation systems. We should keep this in mind during the design phase, in particular where the point of registration or preferences of data is going to be taken. This could be asking users of what type of movies that they enjoy, what types of food they like and what activities they are involved in the most. This also could be set to ask users of what they don’t like, so that we can filter that out those criteria.

One of the benefits of knowledge based recommendation systems is not needing to have user item/content interaction data. We simply can rely on user centric data to link users with other users and recommend similar things that those users liked. This also doesn’t require human generated information about the items which is usually expensive and hard to generate properly. Also knowledge based recommendations use data edits of high fidelity, as the user of interest has self-reported their information and preferences. We can fairly and safely assume that those are true and it could give us more confidence about the data.

Unfortunately knowledge-based systems don’t work well if users don’t select their preferences or set their properties. Any machine learning a model is going to struggle to generate inferences if there is a major lack of data. However, the lack of user data can become a motivator when we design our collection processes. Perhaps we could be asking users the wrong questions or weren’t asking the right ones. Maybe we didn’t create enough fields in the profile page for users to fill out their information or maybe users don’t feel comfortable sharing their preferences with us. Therefore the lack of user data can be due to privacy concerns. As such during machine learning it is important that we act as responsible data stewards sending the right message to the community as well as with the right actions. There can be situations where the privacy concerns are greater than the solutions where it would be easier to recommend methods other than knowledge-based systems.

4. Hybrid Recommendation Systems

Hybrid recommendation systems are designed keeping in mind the both strengths and weaknesses of previous recommendation system approaches.

Although the hybrid recommendation systems sound more complex they can be rather simple. Imagine training a content based collaborative filtering and a knowledge based recommendation systems that each make a recommendation for an item for a user. All these models might recommend different items and some predictions may be better than others due to things like data size quality and model properties. A simple way to create a hybrid model is to just take things from each of the models and combine them all in a neural network. The idea is to eliminate the independent errors within each model so that we will be able to generate much better recommendations.

Context-Awareness Recommender Systems

Limitations of Hybrid Recommendation Systems

Hybrid models sometimes can generate somewhat stateless pieces of data. We don’t know what is going on when someone is interacting with an item that could have changed how they felt about it. The context, which defines what the user has experienced does matter. If the context was different and the perception of enjoyment could be different as well. This changed perception affects the sentiment which can change the feedback provided by the user about an item. Therefore an item/content is not just an item and a user is not just a user, as there are more nuances to them that can be derived from context.

Components of Context

The context is not simple, as it is built up upon many components that a user has remembered from the previous encounters, and experiencing them again in the present and looking forward to in the future

1. What is the mood?

For example, imagine you are watching a movie. What is your mood at the time? Are you super relaxed and curled up on the couch with a bowl of ice cream? Or are you completely tired from a long day at work, stressed from a long commute home and worried about finishing your project for the deadline tomorrow? This probably makes a difference on how you proceed and enjoy the movie

2. Who are you experiencing the item with?

It also depends on with whom you are going to be experiencing the content with. Back to our movie example; what if you were watching a movie with your kids on the couch? What if it is with your partner? What if it is while you are crammed into an economy seat on an airplane with a bunch of random strangers? Each experience will be totally different depending on who you are with and can affect how you would rate the product.

3. Where are you experiencing the item?

Another main factor is location where we experience things that matter a lot. When it comes to our perception, watching movies at home is a different experience from watching movies at a movie theater. Watching a movie on a pop-up big-screen under the stars camping in the desert is totally different and will affect user ratings.

4. When are you experiencing the item?

Well time is also a factor on how we judge our experiences. Imagine we can travel to the same place what we can go in a winter or summer- this can be drastically different experiences.

Let’s say we are enjoying a bed-and-breakfast in a hot summer and the air conditioning is broken. The entire time we will be giving them a low rating. However if we chose to go in the cold winter, we might give a higher rating. Because they always provide a delicious hot breakfast and the leisure house is kept nice and cozy and has a warm furnace. Even if the air conditioner had been broken in our winter example, that component of the bed breakfast we didn’t interact with, because it was not needed and therefore didn’t influence our rating. Same place, different times, different ratings!

5. Special occasions

On a special occasion can heighten our sensitivity to experience, as there are emotional expectations. Going to a restaurant for a quick lunch with a friend is completely different from going to the same place as your wedding rehearsal dinner with the entire family. Even going to a restaurant on your birthday is a special event. This is why a lot of restaurants offer free desserts, singing birthday songs for you. They want you to have a good experience especially because your special occasion changes your perception and you may be more likely to rate them strongly and positively.

3. Recommendation Systems in Action.

How Netflix adopts recommendations to their system.

Netflix is one of the major entities in the content business that uses the context awareness recommendations. They are able to generate recommendations on a variety of contents for the same user base from types of genres like movies, TV shows, anime, documentaries. As a result of the strategy of cross-genre promotion of contents, the platform has gained 203 million paid subscribers by the end of the year 2020. Netflix recommendation strategy is focused on optimizing the context awareness of the platform and maximizing the user experience as a result..

In the below table you can see how the total streaming hours are distributed among main Streaming Platforms [1]

As shown in the above chart, Netflix subscriber base continued to grow consecutively using their marketing campaigns[2]. You may think that this is a result of the brand attraction, but it is in fact the outcome of targeting the correct audience and giving better user experience with contents.

Don’t think it happened randomly and that they were lucky enough. The magic behind Netflix’s exponential growth is its personalized Recommendation systems that rely on a variety of factors. Below we list out a few of them with examples.

Context Awareness Factors

● Device

Netflix content is changing according to their device type they are using, because Mobile devices have a multitude of sensors. They acquire a variety of information about the user’s environment like light intensity, acceleration and information like whether they are in a vehicle or walking, and location information.

● According to Location

● Time (day, season, hour)

They have a Time-Dependent Context Awareness system and they have a process of seasonal recommendation of movies

● Language

App contents vary according to their language which is more valuable for giving relevant contents to the users.

● The “Continue Watching” row is based on the following factors:

• Time watch and remaining
• Based on the Device
• Popularity**

5. How to implement a recommendation system

There are no general evergreen approaches for building recommendation systems; they may vary according to the domain and the type of system. Here we are trying to demonstrate a higher view of user oriented collaborative filtering recommendations system building.

Approach

Assume we are building a movie recommendation system, and there is a particular set of users named A, B and C.

1. Collection and the structuring of the user information and products.

The first step for a recommender system development is determining the users and the content they are using and to understand their preferences towards the content. In movie content recommendations the ideal way of understanding the user preference is the movie reviews and ratings.

2. Comparison of multiple different user groups together.

The next crucial step is designing the process that generates comparisons between different users. We discovered that user A and B are most similar to user A, and that their previous rating about movies. There is a movie which the user A has never seen before, but A’s neighbours B and C highly rated the same movie.

3. Create a function that finds movies that user A has not seen before, but have been watched by other users.

4. Ranking and recommendations.

In order to pique the interest hospitalharrywilliams.org of user A for a new movie, it is necessary to increase the chances of success by assigning a higher ranking to the movie that customers like user A already have. During the ranking process, as the number of similar customers’ rankings increases on a particular Movie, the ranking system can be helpful for the recommender to get that movie assigned to like minded users..

5. Evaluation and testing.

It is not recommended to assume that the recommender system is going to get the accurate result in the first instance. To test the accuracy of such a recommendation system it needs to be evaluated under multiple iterations based on its original collection of contents and users with the reviews. This repetition of the pipeline process increases and maintains a constant level of accuracy.