A model-based collaborative filtering with dimensionality reduction

Abstract

In this day and age, the measure of data accessible online multiplies exponentially. With such development rate, it is getting to be distinctly troublesome for clients to approach things of interest subsequently bringing about information overload issue. This overload produces information in very high dimensions and makes it challenging for these systems to suit or accommodate this increment in data. One of the issues with high-dimensional datasets is that, in many cases, not all the measured factors are "vital" for comprehending the underlying phenomena of interest. The use of mathematical procedures to tackle these problems by reducing the dimensions of the data can successfully alleviate such problems and generate more accurate recommendations. This paper proposes a Model-Based Collaborative Filtering (CF) algorithm that integrates dimensionality reduction technique to lessen known limitations of collaborative filtering techniques. The algorithm consists of building a recommender system for movies using data from the MovieLens Recommender System containing 100,000 ratings. The analytic model was constructed using the standard CRISP-DM methodology. According to the experimental results obtained, the proposed algorithm proved to be very effective as far as dealing with both the sparsity and scalability problems and thus produced more accurate predictions and recommendations when contrasted with the standard Item-based CF technique and the random CF technique.

Keywords: collaborative filtering, dimensionality reduction, model-based, scalability, sparsity