Conference Agenda

Automated analysis of scientific literature represents an opportunity to discover relevant knowledge in a specific domain. The main challenge of Natural Language Processing (NLP) is the morphological descriptions of technical language and their scattered location, which hinders systematic processing.

This work proposes the design and implementation of an NLP method such as bidirectional encoder representation from transformers (BERT) for the automatic analysis and visualization of information extracted from scientific articles on artificial intelligence technologies used to support veterinary pathology. Techniques such as Named Entity Recognition (NER) and semantic representations are used, supported by pre-trained models from the scientific domain. The work was approached from the construction of a specialized corpus, text processing, fine-tuning of the BERT and SciBERT models, and evaluation using standard metrics. In the first experiment, the best configuration obtained in fine-tuning was SciBERT_lrst1, which achieved an F1-score of 0.8262, with a learning rate of 5e-05, weight decay of 0.1, 100 warmup steps, and a linear scheduler. The second experiment maintained the same hyperparameters and the best model, BERT_lrst1, obtained an F1-score of 0.7573. It can be observed that the incorporation of warmup steps and a higher level of regularization promote training stability and improve generalization capacity, which is particularly relevant in scenarios with limited corpus size. SciBERT's specialization in scientific texts gives it an additional advantage in NER tasks in specialized domains such as the one analyzed in this study.