CodeELECTRA: An ELECTRA-based approach for improved vulnerability detection in blockchain smart contracts

Abstract

Blockchain technology has gained significant traction due to its core features of immutability, transparency, and decentralization. Smart contracts, self-executing programs stored on blockchains, play a vital role in enabling secure and automated transactions. Secure and automated transactions are made possible by self-executing programs and smart contracts that are kept on blockchains. The rapid progress of blockchain technology has been linked to an increase in security concerns targeting smart contracts. In comparison to traditional approaches, deep learning and transformer-based approaches have recently demonstrated a number of advantages, such as the capacity to learn from enormous datasets of known vulnerabilities and adjust to novel attack patterns. But Masked token training is the source of inefficiency for transformer-based approaches like CodeBert, resulting in low accuracy and restricted vulnerability coverage. Furthermore, we propose a novel approach, CodeELECTRA, by utilizing the Electra approach and context-aware masking to discover vulnerabilities, The model first step involves compiling and labeling the dataset of Solidity code that is vulnerable, and this is known as preprocessed Solidity code. Next, the logic decides which tokens to mask, the contest-aware masking step which employs a technique known as context-aware masking to strategically mask specific portions of the code during training. In the third step, model will use the pre-trained ELECTRA model to learn contextual representations of the masked code. The masked code is fed into the ELECTRA encoder to generate contextual embedding, and the fully connected layer is employed in the final step to compare and adjust the ELECTRA models' output in order to classify vulnerabilities. The effectiveness of the chosen model in identifying vulnerabilities will evaluate using the Sodifi benchmark dataset. CodeELECTRA approach will improve vulnerability detection in blockchain smart contracts.