Optimizing Fake News Detection in Resource-Constrained Devices Using Transformer Distillation

Abstract

In today's digital era, stopping fake news from spreading, especially on social media is crucial, given its potential to undermine public trust, influence elections, and cause social unrest. The quick sharing of information, enabled by technological improvements and the decreasing size of computational devices, highlights the need for effective techniques for detecting fake news. This study looks into the possibility of using Transformer Distillation to create a small but precise model for detecting fake news. The research compares the performance of TinyBERT, a simplified version of the base BERT model, with other well-known BERT versions, such as BERT Base, DistillBERT, and MobileBERT. The feasibility of using a resized BERT model for fake news detection, especially on resource-constrained devices like mobile phones, is carefully examined by analysing key variables including model size, training time, and accuracy. The results show that TinyBERT performs admirably accurate given its 80% smaller size compare to BertBase Model, making it a viable option for preventing the spread of false information in the era of portable electronics. The present study enhances the continuous endeavours to curb the dissemination of false information by offering a proficient and effective detection system.