A Natural Language Processing Model Based on Pre-trained Deep Learning Models
Aviation College, AnYang University, Anyang 455000, China
| Abstract: | Natural language processing (NLP) is a field of artificial intelligence (AI) whose primary purpose is to give computers the ability to understand written and spoken language in a human-like manner, mainly by creating computers that can read and respond to information in a human-like manner and then generate their text or speech as a response. However, the process of natural language processing may encounter the problem of inefficiency, so how to make the process of natural language processing more efficient is a direction that is currently being studied. This paper aims to derive a set of programmable rules from helping NLP describe human language by combining computational linguistics and statistics, machine learning and deep learning models. In this way, when text and speech data are combined, computers can understand human language in the form of text or audio data, such as the intent and emotion of the speaker or author. This paper introduces various types of deep learning systems used in NLP analysis and research and describes a pre-training-based approach to natural language processing. The final findings can improve operational efficiency, increase employee productivity, and streamline mission-critical business operations. |
| Keywords: | Deep Learning; Pre-Trained Deep Learning Models; Natural Language Processing |
| DOI: | 10.57237/j.cst.2022.01.007 |
View:
Downloads:
Received: 24 September 2022
Accepted: 20 October 2022
Published Online: 5 January 2023
References
| [1] | Haque N I, Islam M, Ahsan M M. A comparative analysis on suicidal ideation detection using NLP, machine, and deep learning [J]. Technologies, 2022, 10 (3): 57. |
| [2] | Lauriola A L, Aiolli F. An introduction to deep learning in natural language processing: Models, techniques, and tools [J]. Neurocomputing, 2022, 470: 443-456. |
| [3] | Alaidi M, Salim H, Aljazaery I A, Abbood S H. Dark web illegal activities crawling and classifying using data mining techniques [J]. International Journal of Interactive Mobile Technologies, 2022, 16 (10). |
| [4] | Aljazaery A, ALRikabi H T S, Alaidi A H M. Encryption of Color Image Based on DNA Strand and Exponential Factor [J]. International Journal of Online Biomedical Engineering, 2022, 18 (3): 101-113. |
| [5] | ALRikabi S, Qateef J S, Al_airaji R M. Face Patterns Analysis and recognition System based on Quantum Neural Network QNN [J]. International Journal of Interactive Mobile Technologies, 2022, 16 (9). |
| [6] | Alseelawi H T H, ALRikabi H T S. A Novel Method of Multimodal Medical Image Fusion Based on Hybrid Approach of NSCT and DTCWT [J]. International Journal of Online Biomedical Engineering, 2022, 18 (3). |
| [7] | Azeez A, Abdul-Hussein M K, Mahdi M S. Design a system for an approved video copyright over cloud based on biometric iris and random walk generator using watermark technique [J]. Periodicals of Engineering Natural Sciences, 2022, 10 (1): 178-187. |
| [8] | Charniak. Parsing as language modeling [C] // Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. |
| [9] | Goodfellow Y B, Courville A. Deep learning [M]. MIT press, 2016. |
| [10] | Goldberg. A primer on neural network models for natural language processing [Z]. |
| [11] | Guida, Mauri G. Evaluation of natural language processing systems: Issues and approaches [J]. Proceedings of the IEEE, 1986, 74 (7): 1026-1035. |
| [12] | Joshi K, Weinstein S. Control of Inference: Role of Some Aspects of Discourse Structure-Centering [M] // IJCAI. |
| [13] | Koskenniemi. Two-level morphology: A general computational model for word-form recognition and production [M]. Finland: University of Helsinki, Department of General Linguistics Helsinki, 1983. |
| [14] | Jozefowicz O V, Schuster M, Shazeer N, Wu Y. Exploring the limits of language modeling [Z]. |
| [15] | Mezaal S, Abdulkareem S F. Affine cipher cryptanalysis using genetic algorithms [J]. JP Journal of Algebra, Number Theory Applications, 2017, 39 (5): 785-802. |
| [16] | Mezaal S, Hammood D A, Ali M H. OTP encryption enhancement based on logical operations [C] // 2016 Sixth International Conference on Digital Information Processing and Communications (ICDIPC. IEEE, 2016: 109–112. |
| [17] | Salah, Khairy R. The Detection of Counterfeit Banknotes Using Ensemble Learning Techniques of AdaBoost and Voting [J]. International Journal of Intelligent Engineering and Systems, 2021, 14 (1): 326-339. |
| [18] | Shareef S, Abd T, Mezaal Y S. Gender voice classification with huge accuracy rate [J]. ℡KOMNIKA, 2020, 18(5): 2612-2617. |
| [19] | Sundaram S, Gurajada S, Fisichella M, Abraham S S. Why are NLP Models Fumbling at Elementary Math? A Survey of Deep Learning based Word Problem Solvers [Z]. |
| [20] | TH., Hazim H T. Enhanced Data Security of Communication System Using Combined Encryption and Steganography [J]. International Journal of Interactive Mobile Technologies, 2021, 15 (16). |
| [21] | Vinyals Ł K, Koo T, Petrov S, Sutskever I, Hinton G. Grammar as a foreign language [J]. Advances in neural information processing systems, 2015, 28. |
| [22] | Abbas F, Yasmin M, Fayyaz M, Elaziz M A, Lu S, El-Latif A A A. Gender Classification Using Proposed CNN-Based Model and Ant Colony Optimization [J]. Mathematics, 2021, 9 (19): 2499. |
| [23] | Abd-El-Atty A M I, Alaskar H, El-Latif A A A. A Robust Quasi-Quantum Walks-based Steganography Protocol for Secure Transmission of Images on Cloud-based E-healthcare Platforms [J]. Sensors, 2020, 20 (11): 3108. |
| [24] | Abdullah S, Abed M A, Barazanchi I A. Improving face recognition by elman neural network using curvelet transform and HSI color space [J]. Period. Eng. Nat. Sci, 2019, 7 (2): 430-437. |
| [25] | Ahmed S M, Ahmad M, El-Latif A A A. Intelligent Driver Drowsiness Detection for Traffic Safety Based on Multi CNN Deep Model and Facial Subsampling [J]. IEEE Trans. Intell. Transp. Syst, 2021: 1-10. |
| [26] | Elgendy A, Zhang W-Z, He H, Gupta B B, El-Latif A A A. Joint computation offloading and task caching for multi-user and multi-task MEC systems: reinforcement learning-based algorithms [J]. Wirel. Networks, 2021, 27 (3): 2023-2038. |
| [27] | Hammad. Deep Learning Models for Arrhythmia Detection in IoT Healthcare Applications [J]. Comput. Electr. Eng, 2022, 100: 108011. |
| [28] | Johnson. How the statistical revolution changes (computational) linguistics [C] // Proceedings of the EACL 2009 Workshop on the Interaction between Linguistics and Computational Linguistics: Virtuous, Vicious or Vacuous? |
| [29] | Rashid A A, Barazanchi I A, Jaaz Z A. Clustering algorithms subjected to K-mean and gaussian mixture model on multidimensional data set [J]. Period. Eng. Nat. Sci, 2019, 7 (2): 448-457. |
| [30] | Resnik. Four revolutions [J]. Language Log, February, 2011, 5. |
| [31] | Sabir F S. An Automated Real-Time Face Mask Detection System Using Transfer Learning with Faster-RCNN in the Era of the COVID-19 Pandemic [J]. Comput. Mater. Contin, 2022, 71 (2): 4151-4166. |
| [32] | Salih K, See O H, Yussof S, Iqbal A, Salih S Q M. A proactive fuzzy-guided link labeling algorithm based on MIH framework in heterogeneous wireless networks [J]. Wirel. Pers. Commun, 2014, 75 (4): 2495-2511. |
| [33] | Salih Q, Alsewari A R A, Yaseen Z M. Pressure vessel design simulation: Implementing of multi-swarm particle swarm optimization [Z] 10.1145/3316615.3316643. |
| [34] | Schank C, Abelson R P. Scripts, plans, goals, and understanding: An inquiry into human knowledge structures [M]. Psychology Press, 2013. |
| [35] | Tawalbeh F M, Tawalbeh M, Quwaider M, El-Latif A A A. Edge enabled IoT system model for secure healthcare [J]. Measurement, 2022, 191: 110792. |
| [36] | Trabelsi P K, Komurcugil H. Mitigation of grid voltage disturbances using quasi-Z-source based dynamic voltage restorer [C] //2018 IEEE 12th International Conference on Compatibility, Power Electronics and Power Engineering (CPE-POWERENG 2018. IEEE, 2018: 1–6. |
| [37] | Turchin, Builes L F F. Using natural language processing to measure and improve quality of diabetes care: a systematic review [J]. Journal of Diabetes Science Technology, 2021, 15 (3): 553-560. |
| [38] | Winograd. Procedures as a representation for data in a computer program for understanding natural language [M] // MASSACHUSETTS INST OF TECH CAMBRIDGE PROJECT MAC1971. |
| [39] | Niu Y, Korneev A. Identification Method of Power Internet Attack Information Based on Machine Learning [J]. Iraqi Journal for Computer Science and Mathematics, 2022, 3 (2): 1–7. |
Join the Editorial Board
We invite active, qualified and high profile scientists and researchers to join as Editorial Board Members.
Join UsBecome a Reviewer
Scholars with a strong interest in reviewing are invited to join the reviewer panel to ensure the quality of the research to be published.
Join Us