https://doi.org/10.31449/inf.v47i9.5217 Informatica 47 (2023) 123–132 123 Fine-Tuning BERT for Aspect Extraction in Multi-domain ABSA Arwa Akram, Aliea Sabir Faculty of Computer Science and Information Technology, Computer Information System Dept., University of Basrah, Basrah, Iraq E-mail: itpg.arwa.akram@uobasrah.edu.iq, aliea.sabir@uobasrah.edu.iq Keywords: aspect extraction, BERT, bidirectional encoder representations from transformer, deep learning, aspect- based sentiment analysis Received: September 21, 2023 Aspect extraction plays a crucial role in understanding the fine-grained nuances of text data, allowing businesses and researchers to gain deeper insights into customer opinions, sentiment distributions, and preferences. This study presents a BERT-based framework for aspect extraction in ABSA and evaluates its performance. Our research focuses on the comprehensive analysis of aspect extraction as we test our method using the SEMEVAL dataset of various consumer evaluations across diverse domains, including laptops, restaurants, and Twitter. By fine-tuning BERT on a large dataset, we aim to overcome the limitations of traditional approaches and improve the accuracy and efficiency of aspect extraction in ABSA. The experimental findings provide evidence of the efficacy of our methodology with a noteworthy aspect extraction accuracy of 0.98, highlighting its capacity to properly and consistently extract features. The article also explores the applicability of our approach to new domains and its possible applications in real-world scenarios. Povzetek: Ta študija predstavlja okvir za ugotavljanje aspekta v ABSA, ki temelji na sistemu BERT. 1 Introduction Aspect extraction, also known as feature extraction is a fundamental task in natural language processing (NLP) that involves identifying and extracting specific aspects or features from text data. These aspects refer to the different elements, attributes, or components that are being discussed or evaluated within a given context. The background of aspect extraction can be traced back to the broader field of sentiment analysis, which aims to understand and interpret subjective information, such as opinions, sentiments, and emotions, expressed in text. While sentiment analysis provides an overall sentiment polarity (positive, negative, neutral) associated with a piece of text, aspect extraction goes further by identifying the specific aspects or features that contribute to the expressed sentiment. Aspect-Based Sentiment Analysis (ABSA) is an essential subdomain of natural language processing (NLP) that concentrates on the extraction and analysis of particular aspects included in textual material [1]. This approach enables a deeper understanding of the attitudes expressed towards these specific features. The primary obstacle in ABSA (Aspect-Based Sentiment Analysis) is to the accurate detection and extraction of aspects, which serves as the fundamental basis for sentiment analysis at the aspect level. This work’s central focus is optimizing aspect extraction in the field of Aspect-Based Sentiment Analysis (ABSA), with a specific emphasis on situations that include multiple domains. Although ABSA has made significant advancements in recent years, the extraction of aspects continues to be a complex and context-dependent task. Conventional techniques for aspect extraction were labor- intensive and frequently failed to capture the underlying complexity and diversity of language since they mostly depended on manually created features and domain- specific rules. Considerable progress has been made in natural language processing tasks such as sentiment analysis and aspect extraction with the development of deep learning models such as BERT [2]. The main goal of this work is to optimize the BERT algorithm's capacity for aspect extraction in consumer reviews. BERT is a pre- trained transformer-based language model that has shown impressive performance on a range of natural language processing tasks by making use of its attention mechanisms and contextual embeddings. Our goal is to overcome the drawbacks of existing methods and improve aspect extraction accuracy and efficiency in Aspect-Based Sentiment Analysis (ABSA) by fine-tuning BERT on a large dataset of customers reviews. The challenge lies in the identification and isolation of elements within textual data that are not only related to items of interest, such as computers, restaurants, and Twitter debates but 124 Informatica 47 (2023) 123–132 A. Akram et al. also correspond to the particular attitudes expressed towards them [3]. The importance of this work expands to enhancing aspect extraction, the main task of Aspect- Based Sentiment Analysis (ABSA). This improvement in aspect extraction facilitates more accurate sentiment analysis, hence helping businesses [4] and organizations to acquire a more comprehensive understanding of consumer opinions and preferences across many domains. Additionally, this work highlights the effectiveness of BERT, a cutting-edge transformer-based model, in tackling aspect extraction difficulties. Finally, our research covers a wide range of domains, emphasizing the adaptability and cross-domain versatility of our methodology, which in the end offers a useful tool for thorough sentiment analysis in a variety of textual contexts. 2 Related work A comparative study conducted in 2018 [5] investigated language rule-based aspect extraction methods, highlighting the significance of rule completeness and accuracy in achieving successful outcomes. Moreover, the same year [6] witnessed a study on aspect extraction from financial microblogs, comparing supervised and unsupervised methods for explicit and implicit aspect identification, showing promising results. Additionally, 2019 [7] saw a study introducing classification methods based on stacked auto-encoders for aspect extraction in Bangla reviews, outperforming the state-of-the-art methods. In 2022 [8] , researchers explored the benefits of leveraging aspect extraction knowledge in improving aspect-level sentiment analysis across different domains, demonstrating the viability of transferring knowledge from aspect extraction to sentiment analysis. In 2021 [9], a noteworthy study proposed an innovative methodology that combined aspect-based sentiment analysis with the use of BERT, refined aspect extraction rules, and a modified rank-based TF-IDF approach, resulting in enhanced performance compared to state-of-the-art methods. More recently, in 2021 [10], a fine-tuning BERT based method was presented for extracting implicit aspects from Chinese online clothing reviews, achieving improved identification of implicit aspects. Another research [11] effort in the same year focused on incorporating emotional affects into aspect extraction through a novel supervised approach, providing more comprehensive information for decision-making. Furthermore, a domain-independent dynamic ABSA model [12] was introduced in 2021, automating the aspect extraction and sentiment analysis process using Efficient Named Entity Recognition (E-NER) guided dependency parsing and Neural Networks (NN). In 2022 [13], efforts were made to enhance aspect-based sentiment analysis for reviews in the Indonesian language through a deep learning approach in semi-supervised graph-based, such as GCN and GRN for aspect and opinion relationships detection, while polarity classification in CNN and RNN demonstrating superior performance over existing models. These studies collectively As shown in table (1) showcase the ongoing advancements in aspect extraction techniques for sentiment analysis, offering valuable insights into the current state-of-the-art and laying the groundwork for further research in this evolving field. Overall, these papers show that there are different approaches to supervised aspect extraction, including language rule-based methods [14], unsupervised methods, and neural network-based methods. The accuracy of these methods depends on the completeness and accuracy of the rules, the use of a stock-investment taxonomy, the transfer of knowledge across different domains, and the use of stacked auto-encoders. Fine-Tuning BERT for Aspect Extraction in Multi-domain ABSA Informatica 47 (2023) 123–132 125 3 Discussion and comparison We observe a significant disparity between the results obtained in our research and those reported in the related worktable above. It is worth mentioning that our aspect extraction accuracy, which is 0.98, exceeds the accuracies reported in all the relevant literature. The significant improvement in precision highlights the efficacy and excellence of our methodology. In order to comprehend the disparities in performance, it is imperative to take into account the fundamental aspects that contribute to this gap. Our study presents an innovative methodology that involves the fine-tuning of BERT (Bidirectional Encoder Representations from Transformers) to enhance aspect extraction capabilities over a wide range of domains. The process of fine-tuning improves the model's capacity to catch subtleties and context unique to a particular domain, resulting in more accurate extraction of aspects. In contrast, some previous studies have mainly utilized conventional techniques, such as linguistic rule-based approaches or unsupervised methods, which could encounter challenges in accommodating the complexities inherent in diverse domains. Moreover, the significance of our carefully optimized BERT methodology in terms of its impact on the precision and effectiveness of Aspect-Based Sentiment Analysis Study Year Methodology Effectiveness Metrics Current SOTA Gaps Proposed BERT-Based Approach Justification [5] 2018 Rule-based Rule completeness, accuracy Lacks scalability and adaptability across domains BERT provides context-awareness, addressing domain variations and improving aspect extraction accuracy. [6] 2018 Supervised, Unsupervised Explicit and implicit aspect identification Limited scalability for different text types BERT offers a unified approach, improving both explicit and implicit aspect identification across diverse domains. [7] 2019 Stacked Auto-encoders Enhanced Bangla aspect extraction Limited applicability beyond Bangla language BERT's pre-trained multilingual models extend applicability and enhance performance in aspect extraction tasks. [8] 2022 Aspect Extraction Knowledge Transfer Aspect-level sentiment analysis improvement Limited exploration of knowledge transfer techniques BERT's fine-tuning capabilities facilitate knowledge transfer from aspect extraction to sentiment analysis, enhancing results. [9] 2021 BERT, Refined Rules, Rank-based TF-IDF Enhanced performance May not generalize well to diverse datasets BERT's contextual embeddings and adaptability enhance generalization to different datasets and languages. [10] 2021 Fine-tuning BERT Improved implicit aspect identification Limited exploration of fine-tuning for implicit aspects Fine-tuned BERT models have demonstrated superior performance in identifying implicit aspects, addressing this gap. [11] 2021 Supervised Emotional Affect Integration Comprehensive aspect information Limited exploration of emotional affect integration BERT's contextual understanding can be leveraged for improved integration of emotional affects into aspect extraction. [12] 2021 Domain-independent ABSA Model Automation and efficiency Limited attention to cross-domain adaptability BERT's multilingual and context- aware capabilities enhance cross- domain applicability and automation in ABSA. [13] 2022 Graph-based Semi- supervised Learning Superior performance for Indonesian-language ABSA Limited exploration of graph-based approaches BERT's embeddings can enhance graph-based methods, further improving Indonesian ABSA results Table 1: Related work summary table showcasing the results of research studies reviewed, methodologies employed, the effectiveness metrics achieved, the gaps and how BERT overcame these gaps. 126 Informatica 47 (2023) 123–132 A. Akram et al. (ABSA) should not be understated. Accurate aspect extraction is a crucial component of Aspect-Based Sentiment Analysis (ABSA), since it directly influences the overall quality of sentiment analysis outcomes. Our methodology not only exhibits exceptional precision but also showcases notable flexibility to other domains, rendering it an adaptable solution that may be applied across several fields. The capacity to adapt is of utmost importance in the contemporary dynamic online landscape, whereby debates and evaluations encompass a wide range of subjects and industries. In summary, our research represents a ground-breaking addition to the area of aspect extraction in ABSA, providing a method that is both extremely accurate and adaptable to other domains. The process of fine-tuning BERT, along by its corresponding enhancements, demonstrates the capacity to significantly transform sentiment analysis by offering more dependable insights into user-generated material across diverse areas. The significant improvements in accuracy shown in our study highlight the importance of utilizing cutting-edge deep learning methods in the ever-changing field of online interactions and analysis of consumer feedback. 4 Methodology In this section, we provide a comprehensive overview of the methodology employed in our research. We cover various facets of our approach, including the architecture of the BERT-based model, pre-training and fine-tuning processes, adaptation for aspect extraction, and critical pre-processing steps. 4.1 BERT-Based model architecture A BERT-based model architecture is used in (ABSA) aspect-based sentiment analysis for the task of aspect extraction. In the proposed model, we utilize the pre- trained BERT variation, namely "bert-base-uncased," as its underlying framework to effectively collect contextual information from the input text data. In addition to the pre- trained BERT base model, a linear layer with two output units is incorporated to facilitate aspect classification. This enables the differentiation between the presence and absence of aspects. To achieve multi-class classification, the training aim makes use of cross-entropy loss. The cross-entropy loss [15] quantifies the discrepancy between the anticipated probability and the true labels and it is calculated as follow: Loss = -Σ(y_i * log(p(y_i))) During the process of a forward pass, the model receives input token IDs (ids_tensors) and attention masks (masks_tensors). It then utilizes BERT to construct contextual embeddings and applies a linear layer to generate predictions. The training loss is calculated using aspect tags (tags_tensors) if they are available. This architectural design integrates the contextual comprehension capabilities of BERT with the process of fine-tuning for aspect extraction. Consequently, it serves as a robust and effective instrument for Aspect-Based Sentiment Analysis (ABSA) assignments. 4.2 Pre-training and fine-tuning processes for BERT The next critical aspect of our methodology revolves around the pre-training and fine-tuning of the BERT model. The code implements the utilization of the SEMEVAL dataset, which includes data from laptops, restaurants, and Twitter, to perform pre-training and fine- tuning [16] operations for BERT. These processes are applied to construct a customized aspect extraction model. During the pre-training phase, a BERT model as shown in fig (1), precisely referred to as "Bert-base-uncased," undergoes training using a large corpus of textual data. This training aims to enable the model to profoundly comprehend language, context, and semantics. The model acquires the ability to make predictions on words masked inside sentences and comprehend the links between sentences by engaging in activities such as next- sentence prediction [17]. After the completion of pre- training, the process of fine-tuning is conducted on domain-specific datasets pertaining to laptops, restaurants, and Twitter. During this stage, task-specific layers are included in BERT. The inclusion of these further layers facilitates the adaptation of the model to effectively carry out aspect extraction. The process of fine-tuning involves utilizing the language comprehension capabilities of the pre-trained BERT model and applying this acquired knowledge to the task of identifying aspects within the varied textual sources found in the SEMEVAL dataset. This process yields a highly efficient model for extracting aspects, specifically designed to account for the domain- specific intricacies present in laptop, restaurant, and Twitter data. Fine-Tuning BERT for Aspect Extraction in Multi-domain ABSA Informatica 47 (2023) 123–132 127 Rationale Behind Fine-Tuning BERT: • Domain adaptation is a crucial process in the context of aspect extraction tasks in the Bangla language, whereby fine-tuning BERT plays a pivotal role. This essay will examine the difficulties associated with employing a pre-existing model on a distinct linguistic or subject-specific context. It is important to highlight that the process of fine-tuning plays a crucial role in enabling the model to effectively adapt to the unique linguistic and semantic intricacies associated with Bangla aspect extraction. • Enhanced Performance: Elucidate the advantages of fine-tuning through an examination of its role in augmenting aspect extraction accuracy. Please provide concrete examples or findings that demonstrate the improvements in performance attained via the process of fine-tuning. • Novelty: Highlighting the originality in the application of fine-tuning BERT for aspect extraction in the Bangla language. Please indicate whether this is one of the initial endeavors to employ this methodology within this particular field. This strategy exhibits notable distinctions from prior methodologies and is deemed more efficacious. • Efficiency: In the event that the process of fine-tuning BERT yielded enhancements in terms of processing time or resource needs, it is vital to acknowledge these improvements. The attainment of increased efficiency can confer a notable benefit in real-world scenarios. • 4.3 Pre-processing, tokenization In the context of our research, the implementation of various pre-processing and tokenization procedures plays a vital role in the data preparation phase for aspect extraction utilizing BERT. The initial stage involves pre- processing the input text data, which is obtained from the SEMEVAL dataset. The data is initially saved in the form of strings that represent lists. These strings are subsequently subjected to a cleaning procedure to remove undesirable characters. Figure 2: A tokenized sentence and its corresponding tags determining aspects presence. Following the cleaning process, the strings are divided into separate tokens as shown in Fig (2) to facilitate the next step. The process of tokenization[18] is carried out by employing the BERT tokenizer (`tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")`) in order to break down the text into subword tokens, which is a fundamental element of the input format utilized by the BERT model. In the course of this procedure, the text is divided into subwords, and afterward, the tokens are transformed into their respective token IDs. The process of tokenization ensures that the input data is structured in a manner that conforms to the required format of BERT. This allows the model to accurately assess the text and carry out aspect extraction with high efficiency. 5 Experiments 5.1 Dataset The dataset utilized in the code is the SEMEVAL- 2014 dataset [19], which consists of data from three separate domains: Laptops, Restaurants, and Twitter as shown in Table (1). The dataset has 24,321 occurrences. The allocation of data in the data division adheres to an 80-20 split, where about 80% of the data is designated for training purposes and the remaining 20% is reserved for testing within each respective domain. Table 2: Cross-domain SEMEVAL dataset Laptops, Restaurants, and Twitter train test train test train test train test Laptops 993 341 870 128 464 167 2327 636 Restaurants 2162 728 807 195 633 196 3602 1119 Twitter 1561 173 1560 172 3126 346 6247 691 Dataset Positive Negative Neutral Total Figure 1: BERT model and its pre-training and fine- tuning process 128 Informatica 47 (2023) 123–132 A. Akram et al. Table 3: Several samples of the tokenized SEMEVAL datasets. The data points inside these CSV files are structured into three distinct columns. • The tokens, refer to tokenized phrases. • The tags, indicate the annotations of aspect terms. Non-aspect terms are marked with '0', the beginning of aspect terms are marked with '1', and aspect terms are marked with '2'. • The polarities are represented using numerical values, where '0' indicates negativity, '1' denotes neutrality, '2' represents positivity, and '-1' is used for non-aspect phrases. The dataset presented in this study has a well-organized structure, enabling accurate analysis of sentiment based on certain aspects and extraction of such aspects across diverse domains. Benchmark Standard: The SEMEVAL dataset has become known as a standard in the domain of aspect-based sentiment analysis. SEMEVAL has been widely utilized in several prior research investigations and contemporary models for the objectives of assessment and comparison. By using SEMEVAL as the evaluation framework, researchers may establish congruence with the prevailing body of literature and guarantee comparability of their findings with those of other scholars, so enhancing the credibility of their work. 5.2 Hyperparameters The implemented method utilizes many hyperparameters in order to configure and train the aspect extraction model based on BERT. One of the crucial hyperparameters [20] is the learning rate (lr), which has been assigned a value of 2e-5. This hyperparameter plays a significant role in determining the step size throughout the process of gradient descent. In addition, the batch size is a parameter that specifies the quantity of data samples that are processed during each iteration of the training process. For the training dataset, a batch size of 5 is chosen, while for the testing dataset, a batch size of 50 is selected. This choice of batch size has implications for both the efficiency of the training process and the amount of memory utilized. The training epochs, which are set to a value of 3, determine the number of iterations during which the model's weights are updated using the whole training dataset. This parameter significantly impacts the duration of the training process. The determination of hyperparameters is of utmost importance in influencing the behavior of the model. These values are established through a process of testing and meticulous adjustment, with the aim of attaining the highest possible performance in aspect extraction on the SEMEVAL dataset. 5.3 Evaluation metrics The evaluation metrics [21] used to measure the performance of the aspect extraction model in our study are as follows: Precision is a metric that quantifies the ratio of accurate positive predictions (TP) to the total number of expected positive occurrences (TP + false positives, FP). Precision=TP/(TP+FP) The recall metric is computed by dividing the number of true positive predictions (TP) by the sum of true positive predictions and false negative occurrences (TP + FN). Recall=TP(TP+FN) The F1-Score is a metric that quantifies the performance of a model by calculating the harmonic mean of precision Tokens Tags Polarities ['Set', 'up', 'was', 'easy', '.'] [1, 2, 0, 0, 0] [2, 2, -1, -1, -1] ['No', 'installation', 'disk', '(DVD)', 'is', 'included', '.'] [0, 1, 2, 2, 0, 0, 0] [-1, 1, 1, 1, -1, -1, - 1] ['It', "'s", 'fast', ',', 'light', ',', 'and', 'simple', 'to', 'use', '.'] [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0] [-1, -1, -1, -1, -1, -1, - 1, -1, -1, 2, -1] ['Keyboard', 'responds', 'well', 'to', 'presses', '.'] [1, 0, 0, 0, 0, 0] [2, -1, -1, -1, -1, -1] ['It', 'is', 'super', 'fast', 'and', 'has', 'outstanding', 'graphics', '.'] [0, 0, 0, 0, 0, 0, 0, 1, 0] [-1, -1, -1, -1, -1, -1, - 1, 2, -1] ['But', 'the', 'mountain', 'lion', 'is', 'just', 'too', 'slow', '.'] [0, 0, 1, 2, 0, 0, 0, 0, 0] [-1, -1, 0, 0, -1, -1, - 1, -1, -1] ['The', 'baterry', 'is', 'very', 'longer', '.'] [0, 1, 0, 0, 0, 0] [-1, 2, -1, -1, -1, -1] ['Its', 'size', 'is', 'ideal', 'and', 'the', 'weight', 'is', 'acceptable', '.'] [0, 1, 0, 0, 0, 0, 1, 0, 0, 0] [-1, 2, -1, -1, -1, -1, - 1, -1, -1, -1] Fine-Tuning BERT for Aspect Extraction in Multi-domain ABSA Informatica 47 (2023) 123–132 129 and recall. This measure aims to provide a fair evaluation of the model's effectiveness. F1−Score = 2 ⋅Precision ⋅Recall /(Precision +Recall ) The measure of accuracy quantifies the ratio of correctly categorized cases, including both true positives and true negatives, relative to the total number of examples present in the dataset. Accuracy= (TP+TN)/ (TP+TN+FP+FN ) The aforementioned equations serve the purpose of quantifying the performance of the model by evaluating precision, recall, F1-Score, and overall accuracy. This allows for a full evaluation of the model's ability to extract aspects. 5.4 Results This section reports the results of aspect extraction task. Our findings on the SEMEVAL dataset, as illustrated in Table 3, provide evidence of the precision and efficacy of our methodology in relation to aspect extraction. The model demonstrated a remarkable capacity to properly detect aspects within the text, as evidenced by its attainment of high accuracy, recall, F1-score and percision as shown in table (3) below: In addition, our model demonstrated greater performance in aspect extraction tasks compared to all presented baselines for this dataset, highlighting its superiority. This achievement is especially remarkable considering the variety of review sources, such as laptops, restaurants, and Twitter as shown in Figure 3. We demonstrate the resilience and flexibility of our technique with consistent results across different areas. To the best of our knowledge, this study represents a significant step forward in aspect extraction on the SEMEVAL dataset. Figure 3: Bar chart representing the model performance 5.5 Real-world applications The investigation conducted on BERT-based aspect extraction in ABSA holds substantial significance for both enterprises and researchers in practical contexts. This is a potential strategy for utilizing our research findings: • Improved customer insights: Precise aspect extraction helps firms identify product or service aspects that generate the most client happiness or discontent, enabling targeted improvements. • Identifying consumer preferences helps product developers prioritize improvements and innovations. • Comparative Analysis: Scholars and businesses can compare aspect-based sentiment among brands or products on a topic. This can help a company outperform its rivals. • Aspect extraction can influence and shape marketing. Customizing communication and support tactics based on customer feedback can improve problem-solving and message delivery. • Domain Adaptation: Our methodology allows organizations to analyze various industries Our research can improve decision-making, product development, and marketing, increasing consumer happiness and competitiveness. 5.6 Future work The ABSA aspect extraction approach based on BERT has demonstrated promising results. There exist several possible further research and developments in this area such as: • Multilingual Aspect Extraction: By extending our method to many languages, businesses may evaluate customer satisfaction on a worldwide scale. • Cross-Domain Transfer Learning: In scenarios with insufficient labeled data, studying methods for transferring information between domains might improve our approach's flexibility. • Robustness and Bias Mitigation: Aspect extraction models must address robustness, bias, and fairness challenges to give accurate and impartial insights. 0 0,2 0,4 0,6 0,8 1 1,2 Precision Recall F1-score 130 Informatica 47 (2023) 123–132 A. Akram et al. Our aspect extraction method has exhibited a wide range of potential applications that extend beyond the specific domains we have investigated. These applications encompass several sectors like healthcare, e-commerce, hotel, financial services, education, and government/public services. Researchers and companies may gain from this adaptability since it offers insights on consumer attitudes and views across a range of sectors. 6 Conclusion This method implements a BERT-based model for aspect extraction on the SEMEVAL dataset, which includes reviews from various domains such as computers, restaurants, and Twitter. Our work exemplifies the whole pipeline encompassing several stages such as data pre-processing, model training, and assessment. Through the process of fine-tuning BERT, the model demonstrates notable levels of accuracy, recall, and F1- score, hence highlighting its efficacy in the extraction of features from textual data. The efficacy of our proposed methodology is demonstrated through experimental findings on three benchmark datasets, showcasing its ability to deliver unprecedented outcomes and establish a new standard in the field. In addition, we incorporate BERT as an additional feature extractor, hence enhancing our approach. This work offers a useful tool for aspect- based sentiment analysis and lays the groundwork for more complex natural language comprehension tasks that make use of BERT's capabilities. References [1] G. Jacobs and V. Hoste, "Fine-grained implicit sentiment in financial news: Uncovering hidden bulls and bears," Electronics, vol. 10, no. 20, p. 2554, 2021. https://doi.org/10.3390/electronics10202554 [2] A. Torfi, R. A. Shirvani, Y. Keneshloo, N. Tavaf, and E. A. Fox, "Natural language processing advancements by deep learning: A survey," arXiv preprint arXiv:2003.01200, 2020. https://doi.org/10.1063/pt.5.028530 [3] A. Akram and A. S. Sabir, "Explicit aspect extraction techniques," Journal of Al-Qadisiyah for computer science and mathematics, vol. 14, no. 4, pp. Page 90-98, 2022. https://doi.org/10.29304/jqcm.2022.14.4.1116 [4] M. Hajiali, "Big data and sentiment analysis: A comprehensive and systematic literature review," Concurrency and Computation: Practice and Experience, vol. 32, no. 14, p. e5671, 2020. https://doi.org/10.1002/cpe.5671 [5] F. Z. Ruskanda, D. H. Widyantoro, and A. Purwarianti, "Comparative study on language rule-based methods for aspect extraction in sentiment analysis," in 2018 International Conference on Asian Language Processing (IALP), 2018: IEEE, pp. 56-61. https://doi.org/10.1109/ialp.2018.8629140 [6] T. Gaillat, B. Stearns, G. Sridhar, R. McDermott, M. Zarrouk, and B. Davis, "Implicit and explicit aspect extraction in financial microblogs," in 1st Workshop on Economics and Natural Language Processing (ECONLP 2018), 2018, vol. 56, no. 1: Association for Computational Linguistics, pp. 55-61. https://doi.org/10.18653/v1/w18-3108 [7] M. Bodini, "Aspect extraction from bangla reviews through stacked auto-encoders," Data, vol. 4, no. 3, p. 121, 2019. https://doi.org/10.3390/data4030121 [8] N. Majumder, R. Bhardwaj, S. Poria, A. Gelbukh, and A. Hussain, "Improving aspect-level sentiment analysis with aspect extraction," Neural Computing and Applications, pp. 1-11, 2022. https://doi.org/10.1007/s00521-020-05287-7 [9] W. Ansar, S. Goswami, A. Chakrabarti, and B. Chakraborty, "An efficient methodology for aspect-based sentiment analysis using BERT through refined aspect extraction," Journal of Intelligent & Fuzzy Systems, vol. 40, no. 5, pp. 9627-9644, 2021. https://doi.org/10.3233/jifs-202140 [10] L. Yu and X. Bai, "Implicit aspect extraction from online clothing reviews with fine-tuning BERT algorithm," in Journal of Physics: Conference Series, 2021, vol. 1995, no. 1: IOP Publishing, p. 012040. https://doi.org/10.1088/1742-6596/1995/1/012040 [11] J. Z. Maitama, N. Idris, A. Abdi, and A. T. Bimba, "Aspect extraction in sentiment analysis based on emotional affect using supervised approach," in 2021 4th International Conference on Artificial Intelligence and Big Data (ICAIBD), 2021: IEEE, pp. 372-376. https://doi.org/10.1109/icaibd51990.2021.9458996 [12] M. Syamala and N. Nalini, "ABSA: Computational Measurement Analysis Approach for Prognosticated Aspect Extraction System," TEM Journal, vol. 10, no. 1, 2021. https://doi.org/10.18421/tem101-11 [13] A. A. Chamid, Widowati, and R. Kusumaningrum, "Graph-Based Semi-Supervised Deep Learning for Indonesian Aspect-Based Sentiment Analysis," Big Data and Cognitive Computing, vol. 7, no. 1, p. 5, 2022. https://doi.org/10.3390/bdcc7010005 [14] F. Z. Ruskanda, D. H. Widyantoro, and A. Purwarianti, "Sequential covering rule learning for language rule-based aspect extraction," in 2019 Fine-Tuning BERT for Aspect Extraction in Multi-domain ABSA Informatica 47 (2023) 123–132 131 International Conference on Advanced Computer Science and information Systems (ICACSIS), 2019: IEEE, pp. 229-234. https://doi.org/10.1109/icacsis47736.2019.8979743 [15] L. Hou, C.-P. Yu, and D. Samaras, "Squared earth mover's distance-based loss for training deep neural networks," arXiv preprint arXiv:1611.05916, 2016. https://doi.org/10.1063/pt.5.028530 [16] Y. Tay et al., "Scale efficiently: Insights from pre-training and fine-tuning transformers," arXiv preprint arXiv:2109.10686, 2021. https://doi.org/10.1063/pt.5.028530 [17] Y. Sun, Y. Zheng, C. Hao, and H. Qiu, "NSP- BERT: A Prompt-based Zero-Shot Learner Through an Original Pre-training Task--Next Sentence Prediction," arXiv e-prints, p. arXiv: 2109.03564, 2021. https://doi.org/10.48550/arXiv.2109.03564 [18] J. Singh, B. McCann, R. Socher, and C. Xiong, "BERT is not an interlingua and the bias of tokenization," in Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), 2019, pp. 47-55. https://doi.org/10.18653/v1/d19-6106 [19] D. Kirange, R. R. Deshmukh, and M. Kirange, "Aspect based sentiment analysis semeval-2014 task 4," Asian Journal of Computer Science and Information Technology (AJCSIT) Vol, vol. 4, 2014. https://doi.org/10.15520/ajcsit.v4i8.9 [20] A. J. Quijano, S. Nguyen, and J. Ordonez, "Grid search hyperparameter benchmarking of BERT, ALBERT, and LongFormer on DuoRC," arXiv preprint arXiv:2101.06326, 2021. https://doi.org/10.48550/arXiv.2101.06326 [21] M. Hossin and M. N. Sulaiman, "A review on evaluation metrics for data classification evaluations," International journal of data mining & knowledge management process, vol. 5, no. 2, p. 1, 2015. http://dx.doi.org/10.5121/ijdkp.2015.5201 132 Informatica 47 (2023) 123–132 A. Akram et al.