Fennica metadata conversions: statistical monitoring and analysis

Authors

Julia Matveeva (julia.matveeva@utu.fi)

Akewak Jeba (akewak.k.jeba@utu.fi)

Leo Lahti (leo.lahti@utu.fi)

Published

November 18, 2026

1 Preface chapter

This book is a work in progress. More fields will be added to the dataset as they go through data cleaning and harmonization.

Within the FIN-CLARIAH Metadata Harmonization and Analysis work package, we use the Finnish national bibliography (FNB) Fennica to build a harmonized dataset. It serves research and lays the groundwork for further infrastructure work, and it supports the DHL-FI project funded by the Research Council of Finland. Fennica holds metadata for over a million documents, including books, newspapers and maps, with records from 1488 to the present. For more about Fennica, see the National Library of Finland website.

The harmonization runs in two pipelines: the Complete FNB Pipeline and the 1809 to 1917 Period Pipeline. The chapters of this book cover the individual metadata categories.

1.1 The harmonized dataset

The choice of fields to harmonize is guided by the needs of the Digital History for Literature in Finland (2022-2026) project funded by the Research Council of Finland. The project uses digital collections and methods to expand the prevailing understanding of Finnish literary history.

The current release covers the complete digital Fennica, 1,318,062 records. From the MARC 21 source data, 19 harmonized analytical fields were produced. They are distributed over 92 columns, three of which are record identifiers. Most fields keep the source value alongside the harmonized value and, where relevant, add a primary-value column, a multiple-value indicator and derived variables. Publication language, for example, occupies 12 columns: the combined value and its two sources (041,a and 008/35–37) each come with these companion columns.

1.1.1 Fields and columns

Table 1.1: Harmonized fields and their columns in the released dataset.
# Field Source Columns n
– Identifiers – id, id2, data_element 3
1 Record type Leader/06 record_type 1
2 Bibliographic level Leader/07 biblio_level 1
3 Publication status 008/06 publication_status 1
4 Genre (008/33) 008/33 genre_008 1
5 Main author 100 author_name_100, first_100, last_100 3
6 Other contributors 700 author_orig_700, author_name_700, first_700, last_700 4
7 Author authority attributes Kanto (Asteri) author_birth, author_death, author_age, n_authors, author_keys, author_names_all, author_genders_all, author_births_all, author_professions_all, n_female_authors, n_authors_with_gender, author_key_100, author_name_db_100, author_gender_100, author_birth_100, author_death_100, author_profession_100 17
8 Title 245,a–b title, title_remainder, title2, each with _length and _word 9
9 Publication language 041,a; 008/35–37 language_full, language_041, language_008, each with _primary, _multi and lang_*_orig 12
10 Original language 041,h language_original, language_original_primary, language_original_multi, lang_original_orig 4
11 Publication time 008/06–14 publication_year_from, publication_year_till, publication_year, publication_decade 4
12 Publication place 260,a; 264,a publication_place 1
13 Publication country Geographic linkage publication_country 1
14 Publisher 260,b; 264,b publisher 1
15 Signum NLF signum_original, signum_harmonized 2
16 Genre/form 655,a genre_655 1
17 UDC 080 udk_orig, udk_harm, udk_aux, udk, udk_primary, udk_multi 6
18 Physical dimensions 300,c original, gatherings, width, height, obl, area, gatherings.original, width.original, height.original, obl.original 10
19 Physical extent 300,a pagecount, pagecount.multiplier, pagecount.squarebracket, pagecount.plate, pagecount.arabic, pagecount.roman, pagecount.sheet, volnumber, volcount, parts 10
Total 92

1.1.2 Column descriptions

Identifiers and record-level codes

Column Description
id Melinda record identifier
id2 Other system identifier(s)
data_element MARC 008 data element values
record_type Record type (Leader/06), as a descriptive category
biblio_level Bibliographic level (Leader/07)
publication_status Publication status (008/06)
genre_008 Literary form from 008/33, as one of 13 categories

Authors

Column Description
author_name_100, first_100, last_100 Main author (100): full name, first name, last name
author_orig_700 Other contributors as recorded in 700
author_name_700, first_700, last_700 Other contributors (700): full name, first name, last name
n_authors Number of named contributors (100 and 700 combined)
author_keys Authority keys of all contributors
author_names_all, author_genders_all, author_births_all, author_professions_all Names, genders, birth years and professions of all contributors
n_female_authors Number of contributors recorded as female
n_authors_with_gender Number of contributors with a gender value
author_key_100, author_name_db_100 Authority key and authorized name of the main author
author_gender_100, author_birth_100, author_death_100, author_profession_100 Gender, birth year, death year and profession of the main author
author_birth, author_death Author birth and death year
author_age Calculated lifespan (death year minus birth year); present only when both are known

Title

Column Description
title Title proper (245,a)
title_remainder Remainder of title, subtitle (245,b)
title2 Title and remainder combined
*_length, *_word Number of characters and words in each of the three title columns

Language

Column Description
language_full Publication language(s), combined from 041,a and, where 041 is missing, 008/35–37
language_041, language_008 Publication language(s) from each source separately
*_primary First (primary) language
*_multi TRUE if the record has more than one language
lang_*_orig Language code(s) as recorded in the source
language_original (with _primary, _multi, lang_original_orig) Original language of a translated work

Publication

Column Description
publication_year_from, publication_year_till First and last year; publication_year_till is filled only for date ranges
publication_year Harmonized publication year
publication_decade Publication decade
publication_place Harmonized place of publication
publication_country Country of publication
publisher Harmonized publisher name
signum_original, signum_harmonized Library call number as supplied by the NLF, and harmonized

Classification

Column Description
genre_655 Genre/form terms from 655,a that match the Finto SLM vocabulary
udk_orig UDC notation as recorded
udk_harm Harmonized UDC notation
udk_aux UDC auxiliary notation
udk UDC classes converted into descriptive categories
udk_primary Primary UDC class
udk_multi TRUE if the record has more than one UDC class

Physical description (MARC 300)

Column Description
original Dimension statement (300,c) as recorded
gatherings.original, width.original, height.original, obl.original Format, width, height and oblong flag as stated in the source
gatherings Harmonized bibliographic format (e.g. 4to, 8vo); 20 categories
width, height Width and height in cm, partly inferred from format
obl Oblong format (1 = yes)
area Page area in cm²
pagecount Estimated total page count
pagecount.arabic, pagecount.roman Pages numbered with arabic and roman numerals
pagecount.squarebracket Unnumbered pages (given in square brackets)
pagecount.plate, pagecount.sheet Plates and sheets
pagecount.multiplier Multiplier applied to the page count
volnumber, volcount, parts Volume number, number of volumes and number of parts (rarely filled)

1.1.3 Downloads

Download harmonized_fennica.feather.

Harmonized data is also available for a subset of years 1809-1917:

Download harmonized_fennica19.feather.

For more information contact Julia Matveeva (yulmat@utu.fi).

How to reproduce the workflow

1.2 Metadata availability by field

The share of records with at least one harmonized value varies widely between fields (Figure 1.1, Table 1.2).

Figure 1.1: Share of records with at least one harmonized value, by field.
Table 1.2: Records with at least one harmonized value, by field. Tiers as defined in the text.
Tier Field Source Records Share (%)
1 Record type Leader/06 1,318,062 100.0
1 Bibliographic level Leader/07 1,318,062 100.0
1 Title 245,a-b 1,318,062 100.0
1 Publication status 008/06 1,318,010 100.0
1 Publication language 041,a; 008/35-37 1,317,656 100.0
1 Genre (008/33) 008/33 1,314,561 99.7
2 Publication time 008/07-14 1,265,311 96.0
2 Publication place 260,a; 264,a 1,224,289 92.9
2 Physical extent 300,a (parsed) 1,140,335 86.5
2 Publication country derived from place 1,139,850 86.5
3 Call number NLF signum 981,713 74.5
3 Publisher 260,b; 264,b 949,057 72.0
3 Physical dimensions 300,c 868,365 65.9
3 UDC 080 859,589 65.2
3 Author names 100; 700 (authors) 854,774 64.9
4 Asteri identifier Kanto 854,774 64.9
4 Author gender inferred from name 792,495 60.1
4 Birth year Kanto 465,326 35.3
4 Profession Kanto 353,324 26.8
4 Genre/form (655) 655,a (SLM) 248,020 18.8
4 Original language 041,h 196,600 14.9
4 Death year Kanto 192,554 14.6
4 Calculated lifespan derived 190,555 14.5

1.3 Metadata availability over time

Availability of authority-dependent and classification fields changes strongly over time (Figure 1.2). Decades with fewer than 100 records are omitted; the shaded band marks the Grand Duchy period (1809–1917).

Figure 1.2: Availability of selected harmonized fields by publication decade. Shaded: 1809–1917.
Table 1.3: Availability of selected fields (%) by period.
Period Records Author names Asteri identifier UDC Genre/form (655) Physical dimensions Original language
before 1809 43,533 55.1 55.1 46.1 34.3 50.7 3.2
1809–1917 80,636 61.9 61.9 41.9 24.0 41.1 16.7
1918–1944 114,407 59.7 59.7 30.2 12.3 38.0 12.6
1945–1999 521,977 54.4 54.4 75.5 9.4 78.4 15.1
2000– 504,758 76.6 76.6 74.2 28.6 68.6 16.5