Stylometry, or: from experiments to widened horizons

(Eine deutsche Fassung dieses Beitrags erscheint zeitgleich bei “Philologeek“.)

Last week, at the Göttinger philologisches Forum, I had the occasion to report on the current state of my “stylometric experiments” involving above all the relationship between authorship and genre in French classical theater. (I know this sounds like its always the same topic, but I keep making small advances.) Without entering into any details here, I would like to simply take up some points from my conclusion, which I called somewhat grandishly “Reflexions on the philologies in the digital age”.

The conclusion itself was actually pretty short, and primarily consisted in admitting the obvious, namely that my stylometric experiments had not yet been crowned by success, i.e. had not solved the Molière/Corneille controversy. One may even ask whether solving this case is at all possible (and, as we discovered in the discussion, it may not even be desirable to do so; in any case, authorship attribution is not the most important application of stylometry, it seems to me. At the same time, I do think that a solution of the case is not possible without solving the issue of how to separate the signals of authorship and genre and, possibly, form (verse or prose). The stylometric procedures I currently know of do not seem to offer what is needed to decide the controversy. However, right now several publications are about to appear, by Maciej Eder, Matthew Jockers and Mike Kestemont, which each suggest relevant and exciting approaches to the issue (it seems).

Beyond the particular Corneille/Molière case and its specific methodological and theoretical issues, however, stylometric methods of text classification also raise some more general questions which can be seen and discussed in the wider context of how digitization changes our methods in the philologies. I would like to raise three of these issues.

First, the question of the relevance of stylometric methods for literary studies. For the theory of literary genre, it would probably be interesting to find out which textual features, on a micro-level, are responsible for the stylometric classification of texts according to genre. However, I would not put too many hopes in this, because the strength of the stylometric method is precisely to take into account and cumulate a huge amount of minimal differences in word frequency to create an overall significant difference. So it will most likely not be possible to connect the stylometric results with any personal reading experience or observation. More interesting and more promising, it seems, especially if the factor of the individual author could be excluded somehow (or even if is simply relegated to a secondary category), would be to compare the results of such genre or sub-genre classifications to established genre and sub-genre distinctions, or of other categories of literary history. Where is overlap, and where are the differences? Gerhard Lauer and Fotis Jannidis recently found out, for example, that in the body of German literature from around 1800 they analysed stylometrically, there is a group of female writers consistently clustering together which has not been recognized as a group by literary historians. Things like that show a potential for stylometry to enable rewriting literary history based not on narrow canonizations, but on the entire literary production of a literary tradition.

Second, the question of the transparency of the stylometric procedures. As long as we are doing “close reading”, we seem to retain control about how we move from one or several observations in a text to an interpretation of that text, because in a way, we “do it all by ourselves”. In reality, the hermeneutic act, which combines complex mental syntheses, long chains of arguments and creative acts of ascribing meaning, is much less transparent and intelligible to ourselves and to others than we may want it to be. It is true that digital philology means to formalise many things, making them explicit or formulating them overtly, and hence seems to be more easily intelligible to others and may even become reproducible. However, digital philology produces new blind spots: for instance, we philologists are challenged to understand the statistical procedures we are using, and which summarize and transform the masses of texts which we cannot apprehend directly anymore. Or, we need to understand the visualisation procedures which show trends and patterns in our “textual data”. My belief is that faced with such issues, we can bring core humanities scholars’ competencies to the task, such as the ability to deal with hypercomplexity, to create syntheses, or the capacity to ascribe meaning to phenomena while retaining a self-critical perspective on the process of doing so.

Third, and directly related to that last point, I would like to raise the issue of the competences specifically necessary to the digital philologists of the future. I defend the position that we as “digital philologists” do not need to become full-fledged programmers and/or statisticians (although of course, we would like to have those competencies). On the contrary, I think it is essential for us to bring our disciplinary training, strengths and specialized knowledge to a research process which is pushed forward by a small and disciplinarily diverse teams. It will be decisive, however, to better learn to communicate across disciplinary boundaries, and to make a step towards the others. For instance, if we as humanities scholars can learn to write precise pseudo-code, based on a passive knowledge of data formats and programming languages, our team members from information science will have a much easier job. And if the statisticians develop visualisations which make visible and intelligible to humanists what the essential features of an algorithm are, then we philologists will be very thankful. In any case, this leads me to sing the praise of humanities scholars’s core competencies again: they also include the ability to deal with alterity and to build communicative bridges.

Those apparent paradoxes are what makes digital philology so exiciting to me. We need hermeneutic competencies to make sense of empirically created data; even and especially in the formalised world of statistics and information science, there are vast spots of lacking intelligibilty and transparency; and the seemingly mechanic categorisations produced by stylometry raise issues concerning the historical and epistemological status of literary genres. In any case, the bottom line for me is that the digital is far from putting the humanities under pressure; rather, the digital can in surprising and new ways help us widen our horizon and sharpen our perception.