You can download Frog, manually compile and install it from source. However, due to the many dependencies and required technical expertise this is not an easy endeavor.
Linux users should first check whether their distribution’s package manager has up-to-date packages for Frog, as this provides the easiest way of installation.
If no up-to-date package exists, we recommend to use LaMachine. Frog is part of our LaMachine software distribution and includes all necessary dependencies. It runs on Linux, BSD and Mac OS X. It can also run as a virtual machine under other operating systems, including Windows. LaMachine makes the installation of Frog straightforward; detailed instructions for the installation of LaMachine can be found here: http://proycon.github.io/LaMachine/.
Manual compilation & installation¶
The source code of Frog for manual installation can be obtained from Github. Because of file sizes and to cleanly separate code from data, the data and configuration files for the modules of Frog have been packaged separately.
- Source code repository: https://github.com/LanguageMachines/frog/
- Stable releases : https://github.com/LanguageMachines/frog/releases/
- Frog data repository: https://github.com/LanguageMachines/frogdata/ (required dependency!)
To compile these manually, you first need current versions of the following dependencies of our software, and compile and install them in the order specified here:
ticcutils - A shared utility library
libfolia- A library for the FoLiA format
ucto - A rule-based tokenizer
timbl - The memory-based classifier engine
mbt - The memory-based tagger
frogdata - Datafiles needed to run Frog
You will also need the following 3rd party dependencies:
- icu - A C++ library for Unicode and Globalization support. On
Debian/Ubuntu systems, install the package
- libxml2 - An XML library. On Debian/Ubuntu systems install the
- textcat - A library for language detection. On Debian/Ubuntu systems install the
- A sane build environment with a C++ compiler (e.g. gcc or clang), autotools, autoconf-archive, libtool, pkg-config
The actual compilation proceeds by entering the Frog directory and issuing the following commands:
- $ bash bootstrap.sh $ ./configure $ make $ sudo make install
/usr/local/by default), you may use the
–prefixoption in the configure step:
Quick start guide¶
Frog aims to automatically enrich Dutch text with linguistic information of various forms. Frog integrates several NLP modules that perform the following tasks: tokenize text to split punctuation from word forms (including recognition of sentence boundaries and multi-word units), assignment of part-of-speech tags, lemmas, and morphological and syntactic information to words.
We give a brief explanation on running Frog to get you started quickly, followed by a more elaborate description of using Frog and how to manipulate the settings for each of the separate modules in Chapter :[modulesDescription].
Frog is developed as a command line tool. We assume the reader already has at least basic command line skills.
frog -h on the command line results in a brief overview of all
available command line options. Frog is typically run on an input
document, which is specified using the -t option for plain text
documents, or -x for documents in the FoLiA XML format. It is, however,
also possible to run it interactively or as a server. We show an example
of the output of Frog when processing the contents of a plain-text file
test.txt, containing just the sentence In ’41 werd aan de stamkaart
een z.g. inlegvel toegevoegd.
We run Frog as follows: $ frog -t test.txt
Frog will present the output as shown in example [ex-frog-out] below:
The ten TAB-delimited columns in the output of Frog contain the information we list below. This columned output is intended for quick interpretation on the terminal or in scripts. It does, however, not contain every detail available to Frog.
- Token number
- (Number is reset every sentence.)
- The text of the token/word
- The lemma
- Morphological segmentation
- A morphological segmentation in which each morpheme is enclosed in square brackets
- PoS tag
- The Part-of-Speech tag according to the CGN tagset [POS2004].
- in the PoS tag, a number between 0 and 1, representing the probability mass assigned to the best guess tag in the tag distribution
- Named entity type
- in BIO-encoding 
- Base phrase chunk
- in BIO-encoding
- Token number of head word
- in dependency graph (according to the Frog parser)
- Dependency relation type of the word with head word
For full output, you will want to instruct Frog to output to a FoLiA XML file. This is done using the -X option, followed by the name of the output file. https://github.com/proycon/pynlpl, supports both Python 2 and Pytho
To run Frog in this way we execute: $ frog -t test.txt -X test.xml The result is a file in FoLiA XML format [FOLIA] that contains all information in a more structured and verbose fashion. More information about this file format, including a full specification, programming libraries, and other tools, can be found on https://proycon.github.io/folia. We show an example of the XML structure for the token aangesneden in example [ex-xml-tok] and explain the details of this structure in section [sec-usage]. Each of these layers of linguistic output will be discussed in more detail in the next chapters.
<w xml:id="WP3452.p.1.s.1.w.4" class="WORD"> <t>aangesneden</t> <pos class="WW(vd,vrij,zonder)" confidence="0.875" head="WW"> <feat class="vd" subset="wvorm"/> <feat class="vrij" subset="positie"/> <feat class="zonder" subset="buiging"/> </pos> <lemma class="aansnijden"/> <morphology> <morpheme> <t>aan</t> </morpheme> <morpheme> <t>ge</t> </morpheme> <morpheme> <t>snijd</t> </morpheme> <morpheme> <t>en</t> </morpheme> </morphology> </w>
By default the output of Frog is written to screen (i.e. standard output). There are two options for outputting to file (which can also be https://github.com/proycon/pynlpl, supports both Python 2 and Pytho called simultaneously):
-o <filename>– Writes columned (TAB delimited) data to file.
-X <filename>– Writes FoLiA XML to file.
We already saw the input option -t <filename> for plain-text files. It is also possible to read FoLiA XML documents instead, using the -x <filename> option.
Besides input of a single plain text file, Frog also accepts a directory of plain text files as input –testdir=<directory> , which can also be written to an output directory with parameter –outputdir=<dir>. The FoLiA equivalent for –outputdir is –xmldir. To read multiple FoLiA documents, instead of plain-text documents, from a directory, use -x –testdir=<directory>.
Frog can be started in an interactive mode by simply typing
the command line. Frog will present a
frog> prompt after which you
can type text for processing. By default, you will press ENTER at an
empty prompt before Frog will process the prior input. This allows for
multiline sentences to be entered. To change this behavior, you may want
to start Frog with the -n option instead, which tells it to assume each
input line is a sentence. FoLiA input or output is not supported in
To exit this mode, type CTRL-D.
Frog offers a server mode that launches it as a daemon to which multiple
clients can connect over TCP. The server mode is started using the
-S <port> option. Note that options like
valid in this mode too.
You can for example start a Frog server on port 12345 as follows:
$ frog -S 12345.
The simple protocol clients should adhere to is as follows:
- The client sends text to process (may contain newlines)
- The client sends the string
EOTfollowed by a newline
- The server responds with columned, TAB delimited output, one token per line, and an empty line between sentences.
- FoLiA input and output are also possible, using the
-Xoptions without parameters. When
-Xis selected, TAB delimited output is suppressed.
- The last line of the server response consists of the string
READY, so the client knows it received the full response.
Communicating with Frog on such a low-level may not be necessary, as there are already some libraries available to communicate with Frog for several programming languages:
- Python – pynlpl.clients.frogclient 
- R – frogr  – by Wouter van Atteveldt
- Go – gorf  – by Machiel Molenaar
The following example shows how to communicate with the Frog server from
Python using the Frog client in PyNLPl, which can generally be installed
with a simple
pip install pynlpl, or is already available if you use
our LaMachine distribution.
from pynlpl.clients.frogclient import FrogClient port = 12345 frogclient = FrogClient('localhost',port) for data in frogclient.process("Dit is de tekst om te verwerken.") word, lemma, morph, pos = data[:4] #TODO: Further processing per word
Do note that Python users may prefer using the
instead, which will be described in Section :[pythonfrog]:. This binds
with Frog natively without using a client/server model and therefore has
|||The source code repository points to the latest development version by default, which may contain experimental features. Stable releases are deliberate snapshots of the source code. It is recommended to grab the latest stable release.|
|||B (begin) indicates the begin of the named entity, I (inside) indicates the continuation of a named entity, and O (outside) indicates that something is not a named entity|
|||https://github.com/proycon/pynlpl, supports both Python 2 and Python 3|
|[POS2004]||Van Eynde, Frank. 2004. Part of speech tagging en lemmatisering van het corpus gesproken nederlands. Technical report, Centrum voor Computerlinguıstiek, KU Leuven, Belgium.|