Icegrams: A fast, compact trigram library for Icelandic

Overview

Icegrams is an MIT-licensed Python 3 (>=3.10) package that encapsulates a large trigram library for Icelandic. (A trigram is a tuple of three consecutive words or tokens that appear in real-world text.)

Over 78 million unique trigrams and their frequency counts are heavily compressed using radix tries and quasi-succinct indexes employing Elias-Fano encoding. This enables the ~213 megabyte compressed trigram file to be mapped directly into memory, with no ex ante decompression, for fast queries (typically ~10 microseconds per lookup).

The Icegrams library is implemented in Python and C/C++, glued together via CFFI.

The trigram storage approach is based on a 2017 paper by Pibiri and Venturini, also referring to Ottaviano and Venturini (2014) regarding partitioned Elias-Fano indexes.

You can use Icegrams to obtain probabilities (relative frequencies) of over 1.7 million different unigrams (single words or tokens), or of bigrams (pairs of two words or tokens), or of trigrams. You can also ask it to return the N most likely successors to any unigram or bigram.

Icegrams is useful for instance in spelling correction, predictive typing, to help disabled people write text faster, and for various text generation, statistics and modelling tasks.

The Icegrams trigram corpus is built from the Icelandic Gigaword Corpus (Risamálheild), which is collected and maintained by The Árni Magnússon Institute for Icelandic Studies, supplemented by a corpus of recent news articles collected by Miðeind. A weighted sample of the corpora, containing about 1 billion tokens of text from 1980 through July 2026, was used as the source of the token stream. Every sentence was corrected with Málfríður, Miðeind's neural spelling and grammar correction model. Trigrams that only occurred once in the stream were eliminated before creating the compressed Icegrams database. The creation process is further described here; the previous (2019) model is described here. The 2019 model itself also remains available: it is bundled inside icegrams releases up to and including 1.1.7 on PyPI, and can be retrieved from this repository's git history, where it was tracked via Git LFS as src/icegrams/resources/trigrams.bin until version 2.0.0.

Example

>>> from icegrams import Ngrams
>>> ng = Ngrams()
>>> # Obtain the frequency of the unigram 'Ísland'
>>> ng.freq("Ísland")
708104
>>> # Obtain the probability of the unigram 'Ísland', as a fraction
>>> # of the frequency of all unigrams in the database
>>> ng.prob("Ísland")
0.00023349672913028182
>>> # Obtain the log probability (base e) of the unigram 'Ísland'
>>> ng.logprob("Ísland")
-8.362342488960794
>>> # Obtain the frequency of the bigram 'Katrín Jakobsdóttir'
>>> ng.freq("Katrín", "Jakobsdóttir")
47918
>>> # Obtain the probability of 'Jakobsdóttir' given 'Katrín'
>>> ng.prob("Katrín", "Jakobsdóttir")
0.1746471994635099
>>> # Obtain the probability of 'Júlíusdóttir' given 'Katrín'
>>> ng.prob("Katrín", "Júlíusdóttir")
0.027305595241566307
>>> # Obtain the frequency of 'velta fyrirtækisins er'
>>> ng.freq("velta", "fyrirtækisins", "er")
15
>>> # adj_freq returns adjusted frequencies, i.e incremented by 1
>>> ng.adj_freq("xxx", "yyy", "zzz")
1
>>> # Obtain the N most likely successors of a given unigram or bigram,
>>> # in descending order by log probability of each successor
>>> ng.succ(10, "stjórnarskrá", "lýðveldisins")
[('Íslands', -1.4328143767547825), ('.', -2.4118815147731096),
    (',', -2.960989325110117), ('og', -3.4164648537929434), ('að', -4.693559922947841),
    ('sem', -4.728651242759112), ('er', -5.016333315210893), ('í', -5.49590639547278),
    ('en', -5.575949103146316), ('?', -5.575949103146316)]
>>> ng.succ(1, "Ég", "hlýði")
[('Víði', -0.9694005571881035)]

Reference

Initializing Icegrams

After installing the icegrams package, use the following code to import it and initialize an instance of the Ngrams class:

from icegrams import Ngrams
ng = Ngrams()

Now you can use the ng instance to query for unigram, bigram and trigram frequencies and probabilities.

An Ngrams instance can be shared by any number of threads, also on free-threaded Python (3.14t and 3.15t), where queries run in parallel. Call ng.close() only when no other thread is using ng.

On Python with the GIL, threads take turns at queries, and several busy threads are slower than one: there, run parallel queries in separate processes, each with its own Ngrams. The processes share the memory-mapped model file.

By default, probabilities of bigrams and trigrams that do not occur in the database are estimated by backing off to the next lower order (see prob() below). To obtain the behavior of versions before 2.1.0, where no backoff was done, pass backoff=False:

ng = Ngrams(backoff=False)

The trigram model is installed with the icegrams-data package, a dependency of icegrams; see the Installation section. If the model is not present, the Ngrams() constructor raises icegrams.ModelNotFoundError.

The Ngrams class

• __init__(self)

  Initializes the Ngrams instance.

• freq(self, *args) -> int

  Returns the frequency of a unigram, bigram or trigram.

  • str[] *args A parameter sequence of consecutive unigrams to query the frequency for.
  • returns An integer with the frequency of the unigram, bigram or trigram.

  To query for the frequency of a unigram in the text, call ng.freq("unigram1"). This returns the number of times that the unigram appears in the database. The unigram is queried as-is, i.e. with no string stripping or lowercasing.

  To query for the frequency of a bigram in the text, call ng.freq("unigram1", "unigram2").

  To query for the frequency of a trigram in the text, call ng.freq("unigram1", "unigram2", "unigram3").

  If you pass more than 3 arguments to ng.freq(), only the last 3 are significant, and the query will be treated as a trigram query.

  Examples:

  >>>> ng.freq("stjórnarskrá")
  107427
  >>>> ng.freq("stjórnarskrá", "lýðveldisins")
  3167
  >>>> ng.freq("stjórnarskrá", "lýðveldisins", "Íslands")
  755
  >>>> ng.freq("xxx", "yyy", "zzz")
  0

• adj_freq(self, *args) -> int

  Returns the adjusted frequency of a unigram, bigram or trigram.

  • str[] *args A parameter sequence of consecutive unigrams to query the frequency for.
  • returns An integer with the adjusted frequency of the unigram, bigram or trigram. The adjusted frequency is the actual frequency plus 1. The method thus never returns 0.

  To query for the frequency of a unigram in the text, call ng.adj_freq("unigram1"). This returns the number of times that the unigram appears in the database, plus 1. The unigram is queried as-is, i.e. with no string stripping or lowercasing.

  To query for the frequency of a bigram in the text, call ng.adj_freq("unigram1", "unigram2").

  To query for the frequency of a trigram in the text, call ng.adj_freq("unigram1", "unigram2", "unigram3").

  If you pass more than 3 arguments to ng.adj_freq(), only the last 3 are significant, and the query will be treated as a trigram query.

  Examples:

  >>>> ng.adj_freq("stjórnarskrá")
  107428
  >>>> ng.adj_freq("stjórnarskrá", "lýðveldisins")
  3168
  >>>> ng.adj_freq("stjórnarskrá", "lýðveldisins", "Íslands")
  756
  >>>> ng.adj_freq("xxx", "yyy", "zzz")
  1

• prob(self, *args) -> float

  Returns the probability of a unigram, bigram or trigram.

  • str[] *args A parameter sequence of consecutive unigrams to query the probability for.
  • returns A float with the probability of the given unigram, bigram or trigram.

  The probability of a unigram is the frequency of the unigram divided by the sum of the frequencies of all unigrams in the database.

  The probability of a bigram (u1, u2) is the frequency of the bigram divided by the frequency of the unigram u1, i.e. how likely u2 is to succeed u1.

  The probability of a trigram (u1, u2, u3) is the frequency of the trigram divided by the frequency of the bigram (u1, u2), i.e. how likely u3 is to succeed u1 u2.

  If a bigram or trigram does not occur in the database, its probability is estimated by backoff: the probability of the next lower order n-gram, i.e. (u2, u3) for a trigram or (u3) for a bigram, multiplied by icegrams.BACKOFF_FACTOR (0.4, a read-only constant). The backoff is applied repeatedly if needed, so an unseen trigram whose bigram (u2, u3) is also unseen is estimated from the unigram probability of u3 multiplied by the factor squared. This is the "stupid backoff" scheme of Brants et al. (2007), with one addition: the backed-off estimate is capped at 1 / (context frequency + 1), i.e. what a single unseen continuation of the context would be worth. An unseen n-gram therefore never scores higher than a seen n-gram in the same context, while unseen continuations are still ranked against each other by their lower-order probabilities. The resulting values are not normalized to sum to exactly 1 over the vocabulary, but are suitable for ranking and scoring.

  If the Ngrams instance was created with backoff=False, no backoff is done and the add-one estimates described below under logprob() are returned instead. Note that in that case an n-gram whose context (u1 or (u1, u2)) is itself unseen gets a probability of 1.0.

  If you pass more than 3 arguments to ng.prob(), only the last 3 are significant, and the query will be treated as a trigram probability query.

  Examples:

  >>>> ng.prob("stjórnarskrá")
  3.542424727548586e-05
  >>>> ng.prob("stjórnarskrá", "lýðveldisins")
  0.02948951856126892
  >>>> ng.prob("stjórnarskrá", "lýðveldisins", "Íslands")
  0.23863636363636387
  >>>> # Unseen trigram, backed off to 0.4 * prob("át", "Ísland")
  >>>> ng.prob("hundurinn", "át", "Ísland")
  3.735947666084515e-05

• logprob(self, *args) -> float

  Returns the log probability of a unigram, bigram or trigram.

  • str[] *args A parameter sequence of consecutive unigrams to query the log probability for.
  • returns A float with the natural logarithm (base e) of the probability of the given unigram, bigram or trigram.

  The probability of a unigram is the adjusted frequency of the unigram divided by the sum of the frequencies of all unigrams in the database.

  The probability of a bigram (u1, u2) is the adjusted frequency of the bigram divided by the adjusted frequency of the unigram u1, i.e. how likely u2 is to succeed u1.

  The probability of a trigram (u1, u2, u3) is the adjusted frequency of the trigram divided by the adjusted frequency of the bigram (u1, u2), i.e. how likely u3 is to succeed u1 u2.

  Unseen bigrams and trigrams are backed off to the next lower order as described under prob() above, unless the Ngrams instance was created with backoff=False.

  If you pass more than 3 arguments to ng.logprob(), only the last 3 are significant, and the query will be treated as a trigram probability query.

  Examples:

  >>>> ng.logprob("stjórnarskrá")
  -10.248114021011704
  >>>> ng.logprob("stjórnarskrá", "lýðveldisins")
  -3.523720381779265
  >>>> ng.logprob("stjórnarskrá", "lýðveldisins", "Íslands")
  -1.4328143767547825

• succ(self, n, *args) -> list[tuple]

  Returns the N most probable successors of a unigram or bigram.

  • int n A positive integer specifying how many successors, at a maximum, should be returned.
  • str[] *args One or two string parameters containing the unigram or bigram to query the successors for.
  • returns A list of tuples of (successor unigram, log probability), in descending order of probability.

  If you pass more than 2 string arguments to ng.succ(), only the last 2 are significant, and the query will be treated as a bigram successor query.

  Examples:

  >>>> ng.succ(2, "stjórnarskrá")
  [('.', -1.8955821526777576), ('og', -2.45003747615368)]
  >>>> ng.succ(2, "stjórnarskrá", "lýðveldisins")
  [('Íslands', -1.4328143767547825), ('.', -2.4118815147731096)]
  >>>> # The following is equivalent to ng.succ(2, "lýðveldisins", "Íslands")
  >>>> ng.succ(2, "stjórnarskrá", "lýðveldisins", "Íslands")
  [(',', -1.799606749884445), ('nr.', -1.8759797286690185)]

Notes

Icegrams is built with a sliding window over the source text. This means that a sentence such as "Maðurinn borðaði ísinn." results in the following trigrams being added to the database:

   ("", "", "Maðurinn")
   ("", "Maðurinn", "borðaði")
   ("Maðurinn", "borðaði", "ísinn")
   ("borðaði", "ísinn", ".")
   ("ísinn", ".", "")
   (".", "", "")

Trigrams that occur only once in the source text are left out of the database (a frequency cutoff), which keeps it at a manageable size.

Bigram and unigram counts are derived from the stored trigrams: each stored trigram adds its count to both of its bigrams and to all three of its unigrams. In the example above, ("Maðurinn", "borðaði") is counted both as the start of ("Maðurinn", "borðaði", "ísinn") and as the end of ("", "Maðurinn", "borðaði"). This is necessary because the cutoff may remove every trigram that starts with a given bigram, while trigrams that end with it are kept. It means that:

• A trigram's count is its number of occurrences in the source text.
• A bigram's count is about twice its number of occurrences, and a unigram's count about three times. For example, ng.freq("í", "dag") returns about 2 million for a bigram that occurs about 1 million times.
• Bigrams that start a sentence, such as ("", "Maðurinn"), are the exception: their count is the number of sentences that start with the word, as is the count used by succ(N, "").

Comparisons between counts of the same order, and the ranking of successors by succ(), are therefore hardly affected. Probabilities from prob() and logprob() are relative frequencies of these counts, so a trigram probability is about half of the corresponding relative frequency in the text, and a bigram probability about two thirds of it, except directly after the start of a sentence.

The tokenization of the source text into unigrams is done with the Tokenizer package and uses the rules documented there. Importantly, tokens other than words, abbreviations, entity names, person names and punctuation are replaced by placeholders. This means that all numbers are represented by the token [NUMBER], amounts by [AMOUNT], dates by [DATEABS] and [DATEREL], e-mail addresses by [EMAIL], etc. For the complete mapping of token types to placeholder strings, see the documentation for the Tokenizer package.

Prerequisites

This package runs on CPython 3.10 or newer, free-threaded CPython 3.14t and 3.15t, and PyPy 3.11 and 3.12. Binary wheels are published for Linux (x86-64 and aarch64, gcc), macOS (x86-64 and arm64, clang) and Windows (x86-64, MSVC).

If a binary wheel package isn't available on PyPI for your system, you may need to have the python3-dev package (or its Windows equivalent) installed on your system to set up Icegrams successfully. This is because a source distribution install requires a C++ compiler and linker:

# Debian or Ubuntu:
sudo apt-get install python3-dev

Installation

To install this package:

pip install icegrams

This also installs the trigram model, which is published on PyPI as a package of its own, icegrams-data (a 192 MB wheel, holding the 225 MB model file trigrams.bin). icegrams requires the icegrams-data release of the model it was tested with, e.g. icegrams-data == 2026.11.* for the model release of November 2026, so lock files, wheelhouses and container images capture the model together with the code. pip and uv cache the wheel, so it is downloaded once per machine. Creating an Ngrams instance never accesses the network.

The model is also published as an asset of a GitHub release of this repository. Where icegrams-data can't be installed (for instance with pip install --no-deps, or on a machine that cannot reach PyPI), the model can be downloaded from there once instead:

python -m icegrams.download

This stores the model in a per-user cache directory (on Linux typically ~/.cache/icegrams/), verifies its checksum, and is a no-op if the model is already there, or if the installed icegrams-data package holds it. The same step is available from Python as icegrams.download.download_model(). This was the way to install the model for versions 2.0 and 2.1.

The model is looked up in this order: the file named by ICEGRAMS_MODEL_FILE, a model built in place in the package's resources directory, the icegrams-data package, and the download cache. If none holds the model, Ngrams() raises icegrams.ModelNotFoundError. These environment variables affect where the model is stored and looked up:

• ICEGRAMS_MODEL_FILE: path of an existing model file to use directly, skipping the lookup entirely (useful for offline or air-gapped environments).
• ICEGRAMS_MODEL_DIR: base directory for the downloaded model, instead of the per-user cache directory. The model is stored in a subdirectory named after the model release (e.g. model-2026.11), so a package upgrade that ships a new model requires running the download step again.
• ICEGRAMS_MODEL_URL: alternative URL for the download step to fetch the model from. The pinned checksum is only verified for the official URL.

Run python -m icegrams.download --help for the corresponding command-line options (--dir, --url and --force).

If you want to be able to edit the source, do like so (assuming you have git installed):

git clone https://github.com/mideind/Icegrams
cd Icegrams
uv sync --extra dev
uv run python -m pytest

The package source code is now in ./src/icegrams. uv sync installs icegrams-data, with the model, from PyPI (about 192 MB, which uv then caches), as pip install -e ".[dev]" does.

Tests

To run the built-in tests, install pytest, cd to your Icegrams subdirectory (and optionally activate your virtualenv), then run:

python -m pytest

Changelog

• Version 2.3.0: The whole model is now memory-mapped and shared between processes. Model file format 2 stores the vocabulary uncompressed, where format 1 made every process unpack it into 22 MB of private memory. Loading the model takes a tenth of the time, and looking up the words of succ() results is about ten times faster. The model is unchanged (the same words and counts), published as icegrams-data 2026.11.0 and as the GitHub release model-2026.11. Model files in format 1 can still be loaded, and tools/upgrade_model.py converts them. The Notes section now describes how bigram and unigram counts are derived. (2026-11-05)
• Version 2.2.0: The trigram model is installed from PyPI as the package icegrams-data (version 2026.8.0, the unchanged model of 2.0.0), which icegrams now depends on, so the separate download step python -m icegrams.download is no longer needed. It remains as a fallback for installations without icegrams-data. (2026-11-04)
• Version 2.1.0: prob() and logprob() now back off to the next lower n-gram order for bigrams and trigrams that do not occur in the database ("stupid backoff", factor BACKOFF_FACTOR = 0.4, capped so that an unseen n-gram never outscores a seen one in the same context). Previously an n-gram whose context was itself unseen was given a probability of 1.0. Probabilities of n-grams that do occur are unchanged. The previous behavior is available via Ngrams(backoff=False). Runs on free-threaded CPython (3.14t and 3.15t) without the GIL, and an Ngrams instance can be shared by threads. Support for Python 3.15. Wheels added for free-threaded CPython, PyPy 3.12 and Linux aarch64. Now requires Python 3.10+ and cffi 2.0+. succ() is about four times as fast, and no longer allocates memory in proportion to the number of successors, which is in the hundreds of thousands for common words. (2026-10-16)
• Version 2.0.0: New trigram model built from a ~1 billion word corpus (IGC-2022 and IGC-2024ext plus recent news through July 2026), corrected with Miðeind's Málfríður neural spelling and grammar correction model. The model file is no longer bundled in the package; it is fetched from a GitHub release in a separate one-time step, python -m icegrams.download, and Ngrams() raises ModelNotFoundError if it isn't present. (2026-09-03)
• Version 1.1.7: Published abi3 wheels; fixed C++ linking in source builds. (2026-06-11)
• Version 1.1.6: Added abi3 wheel support for smaller release size. (2025-12-12)
• Version 1.1.5: Fixed PEP 561 compliance (py.typed). Fixed ruff linting in CI. (2025-12-12)
• Version 1.1.4: Added support for Python 3.14 and Windows. Improved CI with PyPI trusted publishing. (2025-12-12)
• Version 1.1.3: Minor tweaks. Support for Python 3.13. Now requires Python 3.9+. (2024-08-27)
• Version 1.1.2: Minor bug fixes. Cross-platform wheels provided. Now requires Python 3.7+. (2022-12-14)
• Version 1.1.0: Python 3.5 support dropped; macOS builds fixed; PyPy wheels generated
• Version 1.0.0: New trigram database sourced from the Icelandic Gigaword Corpus (Risamálheild) with improved tokenization. Replaced GNU GPLv3 with MIT license.
• Version 0.6.0: Python type annotations added
• Version 0.5.0: Trigrams corpus has been spell-checked

Copyright and licensing

Icegrams is Copyright © 2020-2026 Miðeind ehf.. The original author of this software is Vilhjálmur Þorsteinsson.

This software is licensed under the MIT License:

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.