My research focuses mainly on the adaptation of natural language processing (NLP) techniques to domain-specific scenarios, where the general-domain models fail to deliver satisfactory results or the availability of labeled data is not guaranteed. In particular, I am interested in introducing domain-specific knowledge into NLP models, to improve their ability to represent knowledge.

In particular, the following NLP tasks are of my interest:

• Few-Shot Named Entity Recognition
• Nested Named Entity Recognition
• Open Information Extraction
• Injecting knowledge into language models

Additionally, I am interested in complementing the NLP tasks with the analysis of other data sources like pictures or accompanying images. Specially, I intend to combine the analysis of both text and images in art-historic documents, where the pictorial representation of the artwork is accompanied by textual descriptions. In such documents, the combination of computer vision techniques like vision transformers and NLP techniques like language modeling and information extraction could benefit from each other to produce meaningful representations for downstream tasks like information retrieval and knowledge graph construction.

Artworks are an essential entity in the art domain, and their titles are the surface form used to mention these entities in art-historic documents. However, the nature of artwork titles makes their recognition a difficult task because they might be ambiguous, they contain mentions of other entities like locations and persons, and often they are composed of tokens that without enough context could be categorized like other syntactic constructs. Take for example "Guernica" by Pablo Picasso: without the proper context, a mention to this artwork might be confused for the place instead of the artwork depicting the events which took place there.

Although deep learning models can improve the performance for the task of NER, these models require large amounts of labeled data, which can be costly and time-consuming to obtain. Therefore, one of the approaches which we are experimenting with is to adapt models and datasets used in different domains, to reduce the amount of labeled data needed to recognize artwork titles. This approach has been previously defined as Cross-domain NER.

The transformer model, which is the basis of most recent pre-trained language models (PLMs), encodes multiple aspects of text into rich contextualized vectors, representing lexical, syntactic, and semantic information captured during pre-training.

We define a model to train the disentanglement of a latent space from pre-trained transformer encoders, in which multi-word named entities are represented in terms of token distances. We define a loss function which clusters the tokens of an entity together in the latent space, optimizing the intra- and extra-cluster distances.