With digital editions creating connections to other projects and contexts and with their integration in the linked data landscape, matching and comparing information across different projects and datasets becomes an important aspect of digital editorial practice: Annotation of entities in TEI XML documents allows both complementing and verifying a project's information and interpretations. In serial or tabular data cleaning contexts, it is common to rely on technical mechanisms and procedures to assist this process of entity reconciliation. This submission argues that the very same mechanisms and procedures can also benefit digital editions and that they should be more than ad hoc solutions, tailored to a particular project and a specific authority database. It presents the Reconciliation API specification that has its origins in the Freebase/OpenRefine software and community, and it discusses how an implementation of the API proposal in the TEI Publisher (TP) platform allows a digital edition to operate both as a client and as a server for reconciliation operations.

The Reconciliation API spec defines ways of matching entities using various hints or conditions, and it includes additional endpoints for providing preview UI components or autocompletion strings. It also includes a data extension facility with which clients can query and adopt arbitrary property values of the entities hosted by the service. The spec follows W3C best practices and is currently at version 1.0-draft, while the W3C Entity Reconciliation Community Group is in the process of transitioning to a W3C Working Group (eventually getting the spec on the W3C standards track).

The TP implementation picks up on work already available in TP's annotation module, where a reconciliation connector has facilitated access to a wide range of authority databases [1] for a while. Besides updating this (client) connector, the presentation introduces a reconciliation server component for TP. This allows the TP instance to act as an authority database for other TP instances' (or its own) annotation connector, or for data cleaning and curation processes in OpenRefine, R, Python etc.

The presentation mentions typical usage scenarios, but it also invites discussions about the definition of entities, properties and concrete heuristics for matching and ranking: While the API spec provides for many entity types and complex matching logic, the current implementation is a proof of concept, restricted to persons and places and to exact, wildcard and regex string matching. Extending this is obviously very welcome, but it is not clear how far a generic component can go and what should be left to each project to define. In that sense, the presentation also seeks to unsettle established reconciliation practices.

[1] See a list of reconciliation services on the Reconciliation service test bench: https://reconciliation-api.github.io/testbench/. This table in effect gets reconciliation service endpoints via a Wikidata SPARQL query and then runs them through a validator to report on the supported API versions and optional features. Future TP-based digital edition projects can thus easily be included.

References

- OpenRefine software: https://openrefine.org/
- Reconciliation Spec Proposal: https://reconciliation-api.github.io/specs/1.0-draft/
- Reconciliation API testbench: https://reconciliation-api.github.io/testbench/0.2/#/
- W3C Entity Reconciliation Community Group: https://www.w3.org/groups/cg/reconciliation/

It has been 11 years[1] since the TEI Processing Model was first presented at the TEI Conference in Lyon and shortly after became part of the TEI Guidelines. It's not easy to say how many digital editions and other scholarly publications are using it, but certainly enough to draw some conclusions,[2] relevant to shaping the TEI's future.

In a decade of writing ODDs with TEI Processing Models for a great variety of projects, it seems the main challenges of using TEI for publishing and interoperability do not lie where they are commonly perceived: definitely not in the large number of available elements nor the complexity of the TEI schema. The recently released teipublisher.odd,[3] a successor of the teiSimplePrint.odd originally released by TEI-C, serves as a base for Processing Model customizations for all TEI Publisher editions, which is to say it tries to cover the sensible default handling for a large body of possible TEI documents —an aim comparable with the familiar TEI Stylesheets.

This comprehensive TEI ODD contains around 250 model specifications for just over a 100 TEI elements. Numbers average to around 2.5 models per element but such a picture is misleading: over 60% of elements needs just one model, and only 15 elements require 4 or more models. Closer examination of model and behaviour distributions across the TEI elements, in conjunction with analysis of customization needs for editions identifies clear directions for the TEI as a standard to provide its community a strong, interoperable basic framework, without compromising on its winning formula for flexibility and extendability.

This paper is going to discuss in detail where exactly the sensitive areas lie (e.g. formalization of generalization, specialization and meronymic relations), in the hope that future TEI development will address these as a priority for the benefit of the larger community.

[1] Eleven years “[...] what do you get? Another day older and deeper in debt” (M. Travis, Sixteen tons); see https://en.wikipedia.org/wiki/Sixteen_Tons.

[2] Largest registry of editions using the TEI Processing Model is the Project Registry maintained by e-editiones: https://www.e-editiones.org/map/

[3] https://github.com/eeditiones/jinks/blob/main/profiles/base10/resources/odd/teipublisher.odd

We conventionally think of computer text and document formats in terms of hierarchically layered functionality: Unicode encodes individual glyphs, HTML specifies the abstract structure of underlying plain text, and CSS concretely styles an HTML document. Unfortunately, these tidy abstractions collapse upon contact with the actual technologies. In reality, Unicode, HTML, and CSS can each (partially) encode, structure, and style text.

What we require is a proper accounting of the remit of a digital text or document format. The problem lies in there being no obvious boundary between contentful text in itself and formatting or structure imposed upon it. Bolding may be a mere visual flourish in a novel, but essential to a mathematical treatise. So, should bolding be handled at the markup or character set level? TEI P5 offers some guidance here, but both positions come down to taste. This stratification of text into character sets and markup is, I argue, unnatural and historically contingent; no single, universally applicable boundary between text and formatting or character and markup can exist. The salience of concrete, visual characteristics or structure to the meaning of text is always indexed to a particular text.

Instead, we might imagine the space of document formats as a continuum between the fully concrete, such as a typeset PDF, and fully abstract, such as an application-specific XML schema. TEI is unique in its ability to run this gamut to nearly both extremities within a single mechanism, and in this respect sits at an undertheorized position in this space. Indeed, TEI even includes a facility to define new characters altogether and thus is, in a remote sense, a character set encoding unto itself! While TEI serves as an embryonic model of what a truly universal text and document format could be, it is unsuited to this role in its present form due to, among other things, the rather extreme syntactic overhead of XML itself.

In this talk, I introduce Telecode: a new, single standard for encoding, structuring, and styling text, harmonizing the capabilities of Unicode and TEI. Telecode includes both a new character set and an XML-like data model which subsumes XML-based document formats such as TEI—all TEI documents can be expressed as Telecode text—but expands its scope to cover encoding all text at all levels of formatting. I further describe a novel binary encoding for Telecode text, which is competitive with UTF-8 Unicode in compactness. I finally give an overview of Telecode tooling, including an open source editor and software library that I have authored.

Telecode's design is particularly attuned to the needs of minority and minoritized writing system communities. Text as a category is not and cannot be a static thing. Text encodings must meet the needs of an ever increasing number of computerized scripts, but the expansion of many such technologies in functionality is metered by standardization bodies. Telecode unifies character encoding itself with a more advanced form of TEI's character declaration features and is thus readily adaptable to new scripts in a decentralized manner.