Advancements in the NLP have skyrocketed in the past couple of years. In the research, machine learning and deep learning paved the way to new techniques and possibilities. Many of these techniques already went mainstream like ELMo, PyText, BERT, etc. These techniques provided us with many features, like predicting text, detecting entities in the text, and probably the most important – understanding context of the text.

The industry followed this and various frameworks and tools are developed based on the new research techniques. Today we have several options when it comes to different NLP frameworks. In this article, I cover one of those frameworks – Flair. I was impressed by its ease of use and the possibilities it provides, and how you can achieve state-of-the-art results in just a matter of minutes. So, let’s dive into it.

In this article, you will find out:

1. What is Flair?
2. Installation & Prerequisites
3. Flair Basics
4. Embeddings
5. Named Entity Recognition (NER) with Flair
6. Text Classification with Flair
7. Flair Datasets
8. Training Flair Model

1. What is Flair?

Flair is an amazing NLP framework built on top of PyTorch deep learning framework. It is built and outsourced by the Humboldt University of Berlin and Zalando Research. In essence, it provides three big families of functionalities. First, it exposes state-of-the-art natural language processing (NLP) models so you can easily use them for transfer learning.

Second, it provides various embeddings, so you can use Flair as an embedding library. Finally, it exposes various datasets and allows you to train your own models, so you can use it as a deep learning framework. Flair is a multilanguage library, so it is very good if you work with multiple languages, especially if you need to work with German.

1.1 Flair SOTA NLP Models

As we already mentioned, the first big Flair functionality is that exposes various SOTA NLP models, so you can use them via transfer learning. Some of the models that can be used out of the box are:

1. Name-Entity Recognition (NER): Detect words in the text which represent a person, location, or name in the text.

2. Parts-of-Speech Tagging (PoS): Mark all the words in the given text based on the “part of speech” they belong to.

3. Text Classification: Classify text based on the defined criteria.

These models are really good and if not SOTA, then they are very close to SOTA. Check out the results of the models that Flair provides against current SOTA models for NER:

1.2 Flair as Embedding Library

We will explore this topic in the following chapters in more detail, however, it is important to realize that Flair provides various word and document embedding techniques, like BERT embeddings, ELMo, etc. It is especially interesting that it provides its own embeddings – Flair Embeddings or Contextual String Embeddings.

This is a novel type of word embedding which is character-based. These embeddings are trained without any explicit notion of words and thus fundamentally model words as sequences of characters. Also, they are contextualized by their surrounding text, meaning that the same word will have different embeddings depending on its contextual use.

Find out more about Contextual String Embeddings in this paper.

2. Installation & Prerequsites 

Before installing Flair, make sure that you have installed Python 3.6 or higher, and PyTorch 1.5 or higher. Once you have installed these just run following command: 

3. Flair Basics

In general, there are several important Flair data types. Let’s start with the most simple one and work our way from there. 

3.1 Sentence Class

In the core of the Flair library you can find Sentence class:

This data type is used for the majority of operations in this library. As you probably realize it represents one sentence. Every Sentence uses tokenization, so each sentence is split into tokens automatically. Default tokenization is done by segtok library under the hood: 

You can access each token of the sentence either by using an index or by using method get_token:

Of course, you can choose not to use tokenization and split the sentence just by words, meaning split just by whitespace:

If you need some other tokenization, for example for Japanese, you can do that as well:

3.2 SequenceTagger and TextClassifier Classes

To load all those neat SOTA models you can use two classes: SequenceTagger and TextClassifier. The SequenceTagger is used for NER and POS operations, while TextClassifeier is used, as the name suggests, for text classification. We will see how these classes are used for these operations in the following chapters, but the most important method of both classes is the load method:

4. Flair Embeddings

As we mentioned, Flair provides various embeddings types through their flair.embeddings module. You can use simple GLoVe embedding, novel Flair embedding, or stacked them together. Here is what you need to import first:

In the snippet above we also initialized the sentence “I love Rubik’s Code blog!”. To load various embeddings we use WordEmbeddings class. For example, if we want to load GLoVe embeddings and apply them to previously initialized Sentence, all we have to do is:

The results can be accessed by using the embedding property of a token in a sentence:

If we want to use novel Flair Embeddings we can use the class of the same name. This class also provides various options, like forward and backwards embeddings:

The cool thing is that you can stack both of those embeddings together, if you want to use them like that:

5. Named Entity Recognition (NER) with Flair

Ok, let’s finally see how we can use some of those Flair state-of-the-art NLP models that we talked so much about. We start with Named Entity Recognition or NER. Here is how Wikipedia defines NER:

In an essence, it is a process of detecting words in the unstructured text that are representing names of organizations, persons, cities, etc. It is a commonly used and very powerful NLP technique.  We already know that we can use SequenceTagger class for this. Very cool thing is that Flair is multilingual.

This turned out really important in the project I am working on at the moment, so I am really happy that there is support for other languages. However, let’s start with English and see how we can do the same with other languages.

5.1 Named Entity Recognition (NER) in English

First, we import all necessary modules and initialize the Sentence:

Then we load the English NER model using SequenceTagger:

The first time you run the previous function, the NER model will be downloaded from HuggingFace. Then the model is loaded into en_ner variable. After that, we can run the model on defined text and see how it performed:

Note that we used the to_tagged_string method to retrieve necessary information from the tagged sentence. The model successfully identified and tagged Rubik’s Code as an organization and Berlin as a location. The same information can be extracted from the sentence like this:

Here we can also see the probability, ie. confidence with which the model identified previous entities. This is amazing. The code is so clean and easy to use. Even better, we can do the same thing in other languages too.

5.2 Named Entity Recognition (NER) in German and French

The code for German NER is almost the same:

It is interesting that the German model didn’t identify Rubik’s Code as an organization, but it identify Rubik as a person. 

We can do the same thing in French:

6. Text Classification with Flair

Flair provides many ways for text classification. In this chapter, we learn how we can implement simple sentiment analysis with it. For this purpose, we need to use TextClassifier class. Similar to NER, we can do this in multiple languages.

In the past couple of years, sentiment analysis became one of the essential tools to monitor and understand customer feedback. This way detection of underlying emotional tone that messages and responses carry is fully automated, which means that businesses can better and faster understand what the customer needs and provide better products and services.

Sentiment Analysis is, in a nutshell, the most common text classification tool. It’s the process of analyzing pieces of text to determine the sentiment, whether they’re positive, negative, or neutral. Understand the social sentiment of your brand, product, or service while monitoring online conversations is one of the essential tools of the modern business and sentiment analysis is the first step towards that.

6.1 Simple Sentiment Analysis in English

Let’s start with importing all necessary modules and initializing, obviously positive sentence:

Now, let’s load the model:

Same as in previous examples, the model will be downloaded in the first run. Here is how we use that model:

We can see that the model classified the previous sentence as POSITIVE with pretty high confidence. This is really good, let’s check if it works with negative sentences as well:

6.2 Detecting Offensive Language in German

This is not the only type of classification which can be done with Flair. We can, for example detect offensive language in German:

7. Flair Datasets

Flair library provides a number of datasets and for that, it uses Corpus class and object. This object is constructed of a list of train sentences – train dataset, a list of dev sentences – validation datasets, and a list of test sentences – test dataset. Using this object you can either load the dataset directly from Flair or load your own dataset (more on that later). Here is how you can load English corpus:

After this you can easily access sentences from this corpus with indexes:

The German corpus is also available:

8. Training a Model

Finally, let’s explore how you can train a model using Flair. Technically, you can use Flair as a standard deep learning framework. Here we train a model that can classify messages from Spam SMS Dataset. Spam detection was one of the first Machine Learning tasks that were used on the Internet. This task falls under NLP and text classification jobs which sound like the perfect match for the Flair framework. It is heavily used in literature and it is great for beginners. First, let’s import all necessary modules:

We use various modules that we already explored in previous chapters. The Sentence, TextClassifier and embedding classes we already learned how to use. The Corpus object we use more explicitly here. Also, we use ClassificationCorpus for loading data from .csv. The ModelTrainer class is used to train the model, as the name sugessts.

8.1 Load and prepare Data

Ok, so let’s load and pre-process the data from the dataset:

Ok, now we need to transfer this data from Pandas dataframe to Corpus object. For that we need to split it into train, validation and test subsets. Also, for this we utilize ClassificationCorpus class:

Finally, we need to build word and document embedings from this data. We use GLoVe and Flair embeddings for word embeddings and DocumentRNNEmbeddings  for document embeddings:

8.2 Train a Model

We put all that together and train a model:

First, we create an object of TextClassifier class. We pass along embeddings, labels and label types. Then we create a ModelTrainer object using the previously created classifier object and corpus object. Finally, we run the train method.

The output will be located in the model folder. There the information about each epoch, weights values and the model itself are stored. Output of this process looks something like this:

Conclusion

In this article, we explored many possibilities of Flair – a powerful NLP framework. We had a chance to learn its basics and to learn how to use it for various NLP tasks such as NER and Sentiment Analysis.