This session aims to explore the opportunities, challenges and methodological consequences related to the Linked Open Data approach for the study of the ancient world. We welcome multi-disciplinary submissions dealing with the following or related aspects of Linked Open Data: URIs for Cultural Heritage objects, methodological consequences of LOD, projects publishing data as LOD, relevant tools and live applications based on LOD, digital libraries and their content in relation to ancient world objects, other approaches of making data interoperable and interlinked.

The deadline for submission has been extended to Dec 7 (11:59pm GTM). Here you can find more details about the conference and read the call for paper, and there you can submit your abstract.

Linked Open Data for the Ancient World (abstract)

[session code: Data1]

The study of the Ancient World is by nature a rich soil for the adoption and exploitation of the Linked Opden Data (LOD) approach. Indeed its long tradition, the diversity of materials and resources as well as the high level of disciplinary specialisation lead to a situation where silos of knowledge, even when available online and under open access licenses, are isolated from each other. This situation is also reflected by the segmentation that the study of the Ancient World has reached with the inevitable tendency to favour one single perspective in despite of others. On the contrary, the LOD approach allows us to integrate heterogeneous sources of information by means of links and persistent identifiers while preserving the disciplinary specificity of data.

The recent adoption of the LOD principles by projects such as Pelagios [1], SQPR [2] and the British Museum [3], in acceptance of the CIDOC-CRM’s Linked Open Data Recommendation for Museums [4], are important steps towards a future of interoperable data in archaeology and classics. There is a variety of ways in which different resources are related to each other: an inscribed stone, for instance, will be linked to the edition of the text, to the building and location it belonged to, to different photographs of the object, to a record in the museum catalog and to related literature. Having those different pieces of information interconnected would allow us to overcome to some degree the mentioned fragmented view on antiquity by rendering a more wholistic image of the past.

In this session we shall discuss the advantages and disadvantages of LOD for the study of the Ancient World, look at available data, existing tools and live applications (beyond the status of being testbeds) and question which steps should be taken to overcome existing obstacles to increase the amount of LOD. Furthermore we welcome reflections on the opportunities, challenges and methodological consequences for the disciplines involved. In continuity with past sessions of the conference on related topics, this section addresses issues including but not limited to:

* URIs for Cultural Heritage objects

* methodological reflections on consequences of LOD

* experiences of projects publishing their data as LOD

* discussion of relevant tools and live applications based on LOD

* digital libraries and their content in relation to Ancient World objects

* other approaches of making data interoperable and interlinked

 References

[1] http://pelagios-project.blogspot.com/

[2] http://spqr.cerch.kcl.ac.uk/

[3] http://collection.britishmuseum.org/About

[4] http://www.cidoc-crm.org/URIs_and_Linked_Open_Data.html

At the end of the day what I got done was:

• setup a local instance of OJS (version 2.3.6) using MAMB;
• give a quick try to the OJS Voyeur plugin, which unfortunately for me is working only with version <=2.2.x;
• create the bare-bone of the plugin, whose code is up here (for my personal record rather than for other’s use, at least at this early stage);
• write a PHP class to query a web service (that I’m developing) to extract citations of ancient works from (plain) texts;
• come up with two possible scenarios for further implementation of the plugin, to happen possibly earlier than next year’s PKP hackfest

1. Client-side centric

The first scenario looks rather heavy on the client-side. The plugin is packaged as an OJS plugin and what it does is essentially as follows:

1. after an article is loaded for view, a javascript (grab.js) gets all the <p> elements of the HTML article and send them over ajax to a php page (proxy.php);
2. a php class act as a proxy (or client) for a 3rd party NER web service;
3. the data that are received from via the ajax call are passed on to the web service via XML-RPC;
4. the response is returned by the web service as JSON or XML format…
5. … and then processed again by the JS script, ideally using a compiled template based on jquery’s template capability. Finally, the citations that were extracted are display as a summary box alongside the article.

2. Server-side centric

Instead, in the second scenario that I envisaged most of the processing happens on the server-side.

1. before being displayed, the article is processed to extract <p> elements;
2. the main plugin class (plugin.php) takes care of sending the input to and receiving a response from the NER service;
3. the response is then ran through a template (template.tpl) by exploiting OJS’s templating functionalities;
4. the formatted summary box is injected into the HTML which is now ready to be displayed to the user.

All in all, I think that I came up with (1) mainly because my PHP is rather rusty at the moment . Therefore, although I’m quite reluctant to admit so, I might decide to go for (2). However, a good point to opt for the former is the case where the user can decide for each paper whether to enable this feature or not.

The idea it’s quite simple, but my knowledge of OJS’ is not (yet) such to allow me to have a clear idea of how to implement it. The plugin should enable the detection and markup of certain bits and pieces (read “named entities”) of articles from an OJS installation. Although my application of the plugin is (originally) targeting a specific type of named entities, citations to ancient texts, to be found mainly in Classics journals, it’d possible to generalise the idea for a wider application. Indeed, the plugin could be thought of as applicable to any named entities contained in journal articles, provided that a web service for that is available.

As an example, let’s suppose to have an existing installation of OJS, where an article contains the following paragraph (which is actually taken from a real world article appeared in Greek, Roman, and Byzantine Studies):

Thus, in the paragraphê speeches ([Dem.] 37.58–60, 38.21–22), a binding settlement is sometimes described as a “boundary marker (horos); in an inheritance dispute (40.39), the binding decision is a telos or peras.

The text in bold contains two references to Demosthenes’ works, respectively (1) a reference to lines 58-60 of the speech Against Pantaenetus and (2) another to lines 21-22 of the one  Against Nausimachus and Xenopeithes. The plugin would parse each paragraph and then produce a result somewhat similar to this, where the cited texts are displayed alongside the text article. All in all the whole idea is not much different from OJS’s citation markup assistant, although at the same time it can be generalised to cover other kind of named entities (people, organisations, etc.).

Some aspects that I believe are important for the implementation of such a plugin are:

• client/server architecture: the plugin should act as a client with respect to the Named Entity Recognition web service; I have already a working prototype for a web service (based on XML-RPC) performing the extraction of citations as described above;
• the markup of the extracted named entities should be customisable, ideally based on a template rewrite system, and should allow one to output RDFa or microformatted markup.
• being able to review, correct and therefore store the output of the automatic extraction will be a plus (possibly including interaction with authority lists to which the named entities can be linked to).

So, this is the idea in a nutshell. I’m looking forward to discuss it together with interested OJSers next week in Berlin and I hope there will be a follow-up post with some updates on the hackfest’s outcome.

But why is this actually not that straightforward and worth writing about it? Essentially because to do so we need to go from a hierarchical  format (XML) to a kind of flat one (BIO or IOB). During this transition all the information about the XML tree is irremediably lost unless we do something to avoid it. And once it’s lost there is no way to inject it back into the XML.

I am aware of the existence of SAX, of course. However SAX is not that suitable for my purpose since it allows me to keep track of of the position in the file being parsed just in terms of line and column number (see this and that). [I have to admit that at this point I did not look into other existing XML parsers.] Instead I just wanted to access for each node or text element its start and end position in the original file. The solution I found it’s probably not the easiest one but at the same time it’s quite interesting (I mean, in terms of the additional knowledge I acquired while solving this technical problem). The solution was to use ANTLR (ANother Tool for Language Recognition).

An explanation of what is ANTLR and how does it work it’s out the scope of this post. But let’s put it simple: ANTLR is a language to generate parsers for domain specific languages (see also here). It is typically used to write parsers to process programming languages and it’s based on few core concepts: grammar, lexer, parser and Abstract Syntax Tree (AST). Therefore, it is possible to write an ad-hoc XML/SGML parser using this language. To be honest, the learning curve is pretty steep, but one of most rewarding things to ANTLR is that it’s possible to compile the same grammar into different languages (like Python and Java in my case) with just few (and not substantial) changes, with consequent great benefits in terms of code reusability.

The parser I came up with (source code here) is based on some other code that was developed by the ANTLR community. Essentially,I did some hacking on the original to allow for tokenising the text element on the fly while parsing the XML. During the parsing process, the text elements in the XML are tokenised by space characters and split into tokens of which the start and end positions are kept.

My ANTLR XML/SGML parser does on the fly another couple normalisations in order to produce an output that is ready to be consumed by a Named Entity Recogniser:

1. resolving SGML entities into Unicode;
2. transcoding BetaCode Greek into Unicode;
3. tokenising text by using the non-breaking space (&nbsp;) in addition to normal spaces: this task in particular, although it may seem trivial, implies recalculating the position of the new token in the input file and it required a bit more thinking through;

[{"start": 2768, "end": 2778, "utext": "\u0153uvre", "otext": "œuvre"},
{"start": 2780, "end": 2782, "utext": "par", "otext": "par"},
{"start": 2784, "end": 2790, "utext": "Achille", "otext": "Achille"}]

The work presented in this paper is related to the Hellespont project, an NEH-DFG founded project aimed at joining together the digital collections of Perseus and Arachne [1]. In this paper we present ongoing work aimed at devising a Virtual Research Environment (VRE) that allows scholars to access to both archaeological and textual information [2].

An environment integrating together these two heterogeneous kinds of information will be highly valuable for both archaeologists and philologists. Indeed, the former will have easier access to literary sources of the historical period an artifact belongs to, whereas the latter will have at hand iconographic or archaeological evidences related to a given text. Therefore, we explore the idea of a VRE combining archaeological and philological data with another kind of textual information, that is secondary sources and in particular journal articles. To develop new modes of opening up and combining those different kinds of sources, the project will focus on the so called Pentecontaetia of the Greek historian Thucydides (Th. 1,89-1,118).

As of now, we do not dispose (yet) of an automatic tool capable of capturing passages of Thucydides’ Pentecontaetia that are of importance to our knowledge of Athens and Greece during the Classical period. For the identification of such “links” we totally rely on the irreplaceable, manual and accurate work of scholars. For this reason some preliminary work has been done by A. Thomas to manually identify within the whole text of Thucydides’ Pentecontaetia entities representing categories in the archaeological and philological evidence (e.g. built spaces, topography, individual persons, populations). However, what instead can be done at some extent by means of an automatic tool is extracting and parsing both canonical and modern bibliographic references that express the citation network between ancient texts (i.e. primary sources) and modern publications about them (i.e. secondary sources).

As corpus of secondary sources the journal articles available in the JSTOR and made recently available to researchers via the Data for Research API [3] are being used. Apart from JSTOR classification of such articles into the separate categories of archaeology and philology, those articles are likely to contain references to common named entities that make them overlap at some extent. As an example of what we are aiming to, in Th. I 89 the author refers to the rebuilding of the Athenian city walls – after the Persian War in the beginning of the 5th century BC – as a result of the politics of the Athenian Themistocles. Within our VRE, the corresponding archaeological and philological metadata [4,5] will be presented to the user along with JSTOR articles from both archaeological and philological journals related to the contents of this text passage.

From a technical point of view, we are applying Named Entity Recognition techniques to JSTOR data accessed via the DfR API. References to primary sources, that are usually called “canonical references”, and bibliographic references to other modern publications are to be extracted and parsed from JSTOR articles and will be used to reconstruct the above mentioned citation networks [6,7]. Semantic wise, the CIDOC-CRM will provide us with a suitable conceptual model to express the semantics of complex annotations about texts, archaeological findings, physical entities and abstract concepts that scholars might want to create using such a VRE.

References

For my research project I’m digging into the JSTOR archive via the Data for Research API. And I realised soon to what extent scalability matters when trying to process all the data contained in JSTOR related to scholarly papers in Classics. There are ~60k of them.

The workflow I decided to go for basically consists in retrieving the data from JSTOR, making them persistent via Django (+ MySQL database backend) and then processing iteratively the data. The automatic annotation about those data (mainly Named Entity Recognition) that I’ll be producing is to be stored in the same Django DB.

After having ran the first batch to load my data into my Django application the situation was as follows: 7k documents processed and DB size of ~600MB. By the end of my data loading process the DB will grow up to approximately 6GB (just the data, without any annotation). And it’s at this stage that the cloud (or the grid) comes in handy.

I run my process locally but the remote DB is somewhere on the NGS grid (in my case it’s on the Manchester node). This is of great relieve to my and my machine of course in terms of disk space, speed in accessing the DB and system load. Whenever I need I can dump the DB and installing it locally in case I find myself in the need of accessing it and without an internet connection. Not to mention the fact that the batch processed to load the data could be ran from the grid. Finally, to give public access to the data I’m using  the same django application that pulls out the data from the remote MySQL db.

Having free access to the national grid as UK researcher is absolutely essential, also for someone – like me – who does not work in one of those fields that are known to be benefitting most from grid infrastructure. Even if digital I’m nevertheless still a humanist.

But it’s lot of fun though and finding a technical solution is doable. This post is to share problems, ideas and solutions about this technical aspect of doing NLP on (*)ML-encoded texts and will be in two parts.

Materials

A little while ago I was given an SGML file (~12MB) to process. My idea was to try out on it a Named Entity Recogniser that I have been working on, which extracts standard references to ancient Classical (Greek and Latin) texts. My recogniser is written in Python and accepts as input a file encoded in the IOB format (a format used for the CoNLL-2003 shared task on language-independent named entity recognition). In the IOB format instances are separated by blank lines. Each instance is then tokenised and the resulting tokens are written one per line. Each line contains a number of space-separated column: in the example above the first contains the token itself whereas the second contains a label (category) assigned to the token. *-CRF indicates that a given token is part of a given Named Entity, in this case CRF is used to indicate the presence of a Canonical ReFerence.

This is what an example instance looks like:

this	O
is	O
a	O
canonical	O
reference:	O
Hom.	B-CRF
Il.	I-CRF
1,	I-CRF
Hom.	B-CRF
Il.	I-CRF
1,	I-CRF
477;	I-CRF
24,	I-CRF
788;	I-CRF

This format is used both to store the training sets and as output of the recogniser. In other words, the recogniser takes as input an IOB-encoded file where each token is initially assigned the label O (Other) and outputs the same file but with the new labels properly assigned.

Now, the main problem I was faced with is how to tag in the original SGML file those tokens that my recogniser had identified as being part of a named entity. In order to be able to do so, one needs to keep track of the token position within the XML file.

To sum up, these are the steps that I wanted to be able to perform:

1. parse the XML and keep only the text content of some elements;
2. tokenise the text extracted from the XML (while keeping a reference to the token position within the file): the result will be a list of instances (the text content of given elements) where each instance is a list of tokens;
3. the list of instances is then processed by the Named Entity Recogniser which assigns each token one of the following labels [ O | B-CRF | I-CRF ];
4. the original XML is then re-processed: the subsequent tokens that were previously labelled as B-CRF or I-CRF are to be included within a new XML element;
5. the resulting new XML file (i.e. the original document plus the automatically tagged information) is written to the memory.

[To be continued...]

Basically this script groups different calls to single software components (ParsCit, bibutils, Saxon) into a single pipeline.

The source code is hosted at GitHub and is likely to be quite buggy (particularly the XSLT transformation from ParsCit’s XML into MODS has not been thoroughly tested yet). So feel free to fork the repository and improve the code where needed.

In more detail what the script does is:

1. takes as input a plain text bibliography with one entry per line;
2. parses the input using a ParsCit engine;
3. outputs an intermediate mods encoding of the bibliography;
4. finally transforms the intermediate mods into a BibTeX file;
5. your bibliography is now ready to be imported in to Zotero!

A big CAVEAT about the accuracy of the BibTeX output: since the parsing of the plain text input is done automatically by ParsCit, some bibliographic fields might result to be incorrect and thus some manual editing may be needed.

The result won’t be perfect, but at least I don’t have to input everything manually from scratch.

The asynchronous highlights from the conference here presented were selected according to my personal interests. For a complete overview please refer to the program and the full list of presentations.

A bunch of presentations was related to epigraphy: Anelli, Muscariello and Sarullo talked about “The Digital Edition of Epigraphic Texts as Research Tool: the ILA Project”; Farina presented an “Electronic Analysis and Organization of the Syro-Turkic Inscriptions of China and Central Asia” and finally …

Barbera (hand out not available) and Tomatis presented the advancements of the Corpus Taurinense project, a corpus of texts written in XIII century Italian. After Barbera’s brilliant introduction to the corpus, Tomatis focussed on the problem of disambiguating POS tagging.