Database management system for mobile crowdsourcing applications

Abstract

The evaluation of mobile crowdsourcing activities and reports require a viable and large volume of data. These data are gathered in real-time and from a large number of paid or unpaid volunteers over a period. A high volume of quality data from smartphones or mobile devices is pivotal to the accuracy and validity of the results. Therefore, there is a need for a robust and scalable database structure that can effectively manage and store the large volumes of data collected from various volunteers without compromising the integrity of the data. An in-depth review of various database designs to select the most suitable that will meet the needs of a real-time, robust and large volunteer data handling system is presented. A non-relational database was proposed for the mobile- end database: Google Cloud Firestore specifically due to its support for mobile client implementation, this choice also makes the integration of data from the mobile end-users to the cloud-hosted database relatively easier with all proposed services being part of the Google Cloud Platform; although it is not as popular as some other database services. Separate comparative reviews of the Database Management System (DBMS) performance demonstrated that MongoDB (a non-relational database) performed better when reading large datasets and performing full-text queries, while MySQL (relational) and Cassandra (non-relational) performed much better for data insertion. Google BigQuery was proposed as an appropriate data warehouse solution. It will provide continuity and direct integration with Cloud Firestore and its Application Programming Interface (API) for data migration from Cloud Firestore to BigQuery, and the local server. Also Google BigQuery provides machine learning support for data analytics.