Size-Controlled Opcode Ablation for Smart Contract Vulnerability Detection

Astrid Pranadani; Dhani Ariatmanto

doi:10.32734/jocai.v10i2.26398

Authors

Astrid Pranadani Universitas Amikom Yogyakarta
Dhani Ariatmanto Universitas AMIKOM Yogyakarta https://orcid.org/0000-0001-8877-9941

DOI:

https://doi.org/10.32734/jocai.v10i2.26398

Keywords:

ablation study, DIVE Dataset, multi-label classification, opcode distribution, smart contract vulnerability detection

Abstract

Smart contract vulnerability detection requires evaluation protocols that separate real representation signal from dataset-specific artifacts. DIVE provides lifecycle-based tabular features for Ethereum smart contracts, but benchmark performance alone cannot show whether a dominant feature group is useful or only benefits from having many columns. This study examines Opcode Distribution features using 22,330 contracts, 397 processed features, and eight DASP-aligned vulnerability labels. Five multi-label learning configurations were evaluated under 3 x 5 repeated cross-validation, followed by global feature-group ablation, size-controlled random opcode ablation, per-label degradation analysis, cumulative stability analysis, and opcode-profile group-aware robustness checking. MultiOutput LightGBM achieved the best baseline performance, with Micro-F1 of 0.91396, Macro-F1 of 0.82464, and Macro-PR-AUC of 0.90146. Removing the full Opcode Distribution group reduced Macro-F1 to 0.78745, while removing a same-sized random opcode subset produced Macro-F1 of 0.82404. The findings indicate that Opcode Distribution acts as a collective predictive representation rather than a feature-count artifact, without implying causal vulnerability mechanisms.

Downloads

Download data is not yet available.

References

[1] S. S. Kushwaha, S. Joshi, D. Singh, M. Kaur, and H.-N. Lee, "Systematic review of security vulnerabilities in Ethereum blockchain smart contract," IEEE Access, vol. 10, pp. 6605-6621, 2022, doi: 10.1109/ACCESS.2021.3140091.

[2] S. J. Alsunaidi, H. Aljamaan, and M. Hammoudeh, "Leveraging machine learning models to improve smart contract security: A survey of vulnerabilities and detection methods," ACM Computing Surveys, vol. 58, no. 6, Art. no. 151, pp. 1-37, 2025, doi: 10.1145/3772367.

[3] F. Jiang et al., "Enhancing smart-contract security through machine learning: A survey of approaches and techniques," Electronics, vol. 12, no. 9, p. 2046, 2023, doi: 10.3390/electronics12092046.

[4] Z. Zheng et al., "DAppSCAN: Building large-scale datasets for smart contract weaknesses in DApp projects," IEEE Transactions on Software Engineering, vol. 50, pp. 1360-1373, 2024, doi: 10.1109/TSE.2024.3383422.

[5] C. S. Yashavant, S. Kumar, and A. Karkare, "ScrawlD: A dataset of real world Ethereum smart contracts labelled with vulnerabilities," arXiv, 2022, doi: 10.48550/arXiv.2202.11409.

[6] G. Ibba et al., "A curated Solidity smart contracts repository of metrics and vulnerability," in Proceedings of the 20th International Conference on Predictive Models and Data Analytics in Software Engineering, 2024, pp. 32-41, doi: 10.1145/3663533.3664039.

[7] S. HajiHosseinKhani, A. H. Lashkari, and A. M. Oskui, "Unveiling smart contracts vulnerabilities: Toward profiling smart contracts vulnerabilities using enhanced genetic algorithm and generating benchmark dataset," Blockchain: Research and Applications, vol. 6, no. 2, Art. no. 100253, 2025, doi: 10.1016/j.bcra.2024.100253.

[8] P. Qian, Z. Liu, Y. Yin, and Q. He, "Cross-modality mutual learning for enhancing smart contract vulnerability detection on bytecode," in Proceedings of the ACM Web Conference 2023, 2023, pp. 2220-2229, doi: 10.1145/3543507.3583367.

[9] W. Deng et al., "Smart contract vulnerability detection based on deep learning and multimodal decision fusion," Sensors, vol. 23, no. 16, p. 7246, 2023, doi: 10.3390/s23167246.

[10] F. Luo et al., "SCVHunter: Smart contract vulnerability detection based on heterogeneous graph attention network," in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, 2024, pp. 1-13, doi: 10.1145/3597503.3639213.

[11] Y. Wang, X. Zhao, L. He, Z. Zhen, and H. Chen, "ContractGNN: Ethereum smart contract vulnerability detection based on vulnerability sub-graphs and graph neural networks," IEEE Transactions on Network Science and Engineering, vol. 11, pp. 6382-6395, 2024, doi: 10.1109/TNSE.2024.3470788.

[12] X. Sun et al., "ASSBert: Active and semi-supervised BERT for smart contract vulnerability detection," Journal of Information Security and Applications, vol. 73, p. 103423, 2023, doi: 10.1016/j.jisa.2023.103423.

[13] B. Le Hung et al., "Contextual language model and transfer learning for reentrancy vulnerability detection in smart contracts," in Proceedings of the 12th International Symposium on Information and Communication Technology, 2023, pp. 739-745, doi: 10.1145/3628797.3628945.

[14] V. K. Jain and M. Tripathi, "An integrated deep learning model for Ethereum smart contract vulnerability detection," International Journal of Information Security, vol. 23, pp. 557-575, 2024, doi: 10.1007/s10207-023-00752-5.

[15] H. Lakadawala, K. Dzigbede, and Y. Chen, "Detecting reentrancy vulnerability in smart contracts using graph convolution networks," in 2024 IEEE 21st Consumer Communications & Networking Conference, 2024, pp. 188-193, doi: 10.1109/CCNC51664.2024.10454763.

[16] S. J. Alsunaidi, H. Aljamaan, and M. Hammoudeh, "DIVE: A multi-label smart contract vulnerability dataset," Scientific Data, vol. 13, p. 664, 2026, doi: 10.1038/s41597-026-07025-5.

[17] S. Alsunaidi, H. Aljamaan, and M. Hammoudeh, "DIVE: A multi-label smart contract vulnerability dataset," Zenodo, 2026, doi: 10.5281/zenodo.18519253.

[18] DIVE Framework, "DIVE, version v2.0.0," Zenodo, 2026, doi: 10.5281/zenodo.18779606.

[19] S. J. Alsunaidi, H. Aljamaan, and M. Hammoudeh, "MultiTagging: A vulnerable smart contract labeling and evaluation framework," Electronics, vol. 13, no. 23, p. 4616, 2024, doi: 10.3390/electronics13234616.

[20] B. Lashkari and P. Musilek, "Evaluation of smart contract vulnerability analysis tools: A domain-specific perspective," Information, vol. 14, no. 10, p. 533, 2023, doi: 10.3390/info14100533.

[21] R. Yu, J. Shu, D. Yan, and X. Jia, "ReDetect: Reentrancy vulnerability detection in smart contracts with high accuracy," in 2021 17th International Conference on Mobility, Sensing and Networking, 2021, pp. 412-419, doi: 10.1109/MSN53354.2021.00069.

[22] Z. Zheng et al., "Turn the rudder: A beacon of reentrancy detection for smart contracts on Ethereum," in 2023 IEEE/ACM 45th International Conference on Software Engineering, 2023, pp. 295-306, doi: 10.1109/ICSE48619.2023.00036.

[23] N. Ivanov et al., "Security threat mitigation for smart contracts: A comprehensive survey," ACM Computing Surveys, vol. 55, no. 14s, Art. no. 326, pp. 1-37, 2023, doi: 10.1145/3593293.

[24] H. Zhou, A. Milani Fard, and A. Makanju, "The state of Ethereum smart contracts security: Vulnerabilities, countermeasures, and tool support," Journal of Cybersecurity and Privacy, vol. 2, no. 2, pp. 358-378, 2022, doi: 10.3390/jcp2020019.

[25] Z. Liu et al., "Rethinking smart contract fuzzing: Fuzzing with invocation ordering and important branch revisiting," IEEE Transactions on Information Forensics and Security, vol. 18, pp. 1237-1251, 2023, doi: 10.1109/TIFS.2023.3237370.

[26] L. Zhang et al., "SPCBIG-EC: A robust serial hybrid model for smart contract vulnerability detection," Sensors, vol. 22, no. 12, p. 4621, 2022, doi: 10.3390/s22124621.

[27] Z. Yang, W. Zhu, and M. Yu, "Improvement and optimization of vulnerability detection methods for Ethereum smart contracts," IEEE Access, vol. 11, pp. 78207-78223, 2023, doi: 10.1109/ACCESS.2023.3298672.

[28] M. Eshghie, C. Artho, and D. Gurov, "Dynamic vulnerability detection on smart contracts using machine learning," in Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering, 2021, pp. 305-312, doi: 10.1145/3463274.3463348.

[29] A. Mezina and A. Ometov, "Detecting smart contract vulnerabilities with combined binary and multiclass classification," Cryptography, vol. 7, no. 3, p. 34, 2023, doi: 10.3390/cryptography7030034.

[30] S. Rawlekar, S. Bhatnagar, V. P. Srinivasulu, and N. Ahuja, "Improving multi-label recognition using class co-occurrence probabilities," in Pattern Recognition: 27th International Conference, ICPR 2024, Proceedings, vol. 15310, pp. 424-439, Springer, 2025, doi: 10.1007/978-3-031-78192-6_28.

[31] R. Croft, M. A. Babar, and M. M. Kholoosi, "Data quality for software vulnerability datasets," in 2023 IEEE/ACM International Conference on Software Engineering, 2023, pp. 121-133, doi: 10.1109/ICSE48619.2023.00022.

[32] S. Huang, W. Hu, B. Lu, Q. Fan, X. Xu, X. Zhou, and H. Yan, "Application of label correlation in multi-label classification: A survey," Applied Sciences, vol. 14, no. 19, p. 9034, 2024, doi: 10.3390/app14199034.

[33] A. N. Tarekegn, M. Giacobini, and K. Michalak, "A review of methods for imbalanced multi-label classification," Pattern Recognition, vol. 118, p. 107965, 2021, doi: 10.1016/j.patcog.2021.107965.

[34] J. Read, B. Pfahringer, G. Holmes, and E. Frank, "Classifier chains: A review and perspectives," Journal of Artificial Intelligence Research, vol. 70, pp. 683-718, 2021, doi: 10.1613/jair.1.12376.

[35] L. Grinsztajn, E. Oyallon, and G. Varoquaux, "Why do tree-based models still outperform deep learning on typical tabular data?" in Advances in Neural Information Processing Systems, vol. 35, pp. 507-520, 2022.

[36] V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, and G. Kasneci, "Deep neural networks and tabular data: A survey," IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 6, pp. 7499-7519, 2024, doi: 10.1109/TNNLS.2022.3229161.

[37] H. Tiittanen, L. Holm, and P. Törönen, "Novel split quality measures for stratified multilabel cross validation with application to large and sparse gene ontology datasets," Applied Computing and Intelligence, vol. 2, no. 1, pp. 49-62, 2022, doi: 10.3934/aci.2022003.

[38] I. Covert, S. Lundberg, and S.-I. Lee, "Explaining by removing: A unified framework for model explanation," Journal of Machine Learning Research, vol. 22, no. 209, pp. 1-90, 2021.