Computational Genre Analysis

Introduction

Genre is, like authorship or time period, one of a number of fundamental categories allowing authors and readers as well as literary scholars to endow the vast field of literary production with some internal structure. Genre is not specific to literature, of course: whether we consider painting, music or cinema, genre as an intermediary category situated between individual works and an entire artform is always a relevant category. Unlike theme or style, literary genre is not in itself a level of analysis; rather, different levels of analysis such as theme or style can be used to describe and distinguish genres.

The aim of this chapter is to introduce to the analysis of genre with quantitative, computational methods. First, the chapter will discuss what literary genres are and which aspects of genre may be studied quantitatively. Second, some issues relevant to building text collections for genre analysis are presented. Then, several example analyses are discussed, all based on the idea that computational genre analysis can be conducted from a contrastive perspective – that is, by comparing texts belonging to a genre of interest to texts from other, related genres.

What is literary genre and how may we study it?

What is literary genre?

Genre has been studied at least since Aristotle’s Poetics and is a mainstay of literary theory. Aristotle posited that all literature is imitative in nature, but that different types can be distinguished with regard to the manner of imitation they use and the subjects they imitate. For example, drama and epos are distinguished based on the manner of imitation: in drama, the characters speak for themselves, while in epic narrative, the author or narrator acts as an intermediary voice. The two dramatic subgenres of tragedy and comedy, in turn, are distinguished based on the type of subject they represent. In tragedy, the protagonists are superior in importance and virtue to the audience, while in comedy, they are inferior to the audience. For Wellek and Warren in their Theory of Literature, literary genres are described as “institutional imperatives,” that is as sets of rules and properties of certain texts established through convention across a community. Authors may accept genres in this sense and write new works that respect them; they may reject or reshape them with new works; or they may aim to create entirely new genres. At the same time, genres for Wellek and Warren are also “specifically literary types of organization or structure” which are susceptible to be studied in a descriptive manner (Wellek and Warren 1942, 235).

This chapter, as most quantitative enquiries into genre, likewise assumes that literary genre is not only a social and cognitive construct, but that there are indeed shared stylistic, thematic and structural characteristics of groups of texts, and that these characteristics can be identified, described, counted and compared. To what degree such groups of texts correspond to established, conventional literary genres is another matter and needs to be examined case by case.

On the most basic level, then, genres are groups of texts that share at least one, but usually several, common characteristics. These characteristics can relate to very different aspects of the texts, such as their themes (as in adventure novels or autobiographies), their protagonists (as in detective fiction), their setting (as in urban novels), their form (as in a sonnet or an epistolary novel), their audience (as in children’s books), their communicative purpose (like catharsis for tragedy) or their length (as in short stories). The characteristics shared by the texts belonging to a genre, it is assumed in computational genre analysis, are reflected in the fact that these texts also share textual features which a quantitative analysis can discover and describe.

Any given genre, then, can be described as a complex combination of traits. Detective fiction, for example, not only involves a detective as protagonist, but also is typically set in an urban environment and provides the readers with suspense or intellectual curiosity. Therefore, it usually makes sense to think of a given genre or subgenre as an abstract and ideal prototype, with individual works realizing a more or less complete set of traits corresponding to that prototype. A picaresque novel may respect the conventional traits of the genre but introduce one crucial novelty, such as letting the protagonist learn from their experience. Such a novel, then, is a less typical representative of the picaresque novel and at the same time, acquires one of the defining traits of the Bildungsroman or novel of education.

How may literary genre be studied with computational methods?

While research into authorship using computational methods has been conducted since the 1960s, computational analysis of literary genre is a somewhat more recent development. Earlier work has primarily been done in computational linguistics using non-literary texts or contrasting non-literary and literary texts, developing many methods of analysis relevant to the study of literary genre (for a textbook, see Biber and Conrad 2009). More recently, the field of Computational Literary Studies is asking questions about genre, and particularly subgenres of drama and the novel, and trying to find answers to them using large collections of digital literary texts and quantitative methods adapted from Computational Linguistics, Statistics and Computer Science. The encounter has been fruitful already and promises to be an exciting field of inquiry for a considerable time to come.

Literary genre analysis is sometimes interested in looking at differences between the three or four broad genre categories (narrative fiction, drama, poetry and literary non-fiction). However, differences between them are quite well-understood and relatively easy to detect. Interest is more frequently directed at the level of subgenres, such as distinguishing crime fiction from adventure novels, or comparing comedies, tragedies and histories. Quantitative genre analysis frequently focuses on one specific aspect of genre and, using descriptive statistics, identifies systematic differences between genres as well as the textual features indicative of these differences.

Crucially, this approach presupposes an agreement about the genre classes themselves and about the assignment of genre classes to each text under consideration. Other ways to analyze genre obviously exist, particularly methods from machine learning (see Alpaydin 2010). Supervised methods of classification serve to assign genre labels to texts of unknown genre, based on a labeled training corpus. Unsupervised methods do not use any predefined labels at all; rather, they assign texts to groups based on their similarity according to one or several types of textual features. Such approaches allow for doubt regarding the genre classes and have the potential to disrupt traditional genre labels. However, due to their complexity, these methods are not discussed here (see the chapter on „The Life Spans of Genre” in Underwood 2019 for a recent example and Calvo Tello 2021 for an in-depth exploration of this issue). Note also that the methods discussed here for contrastive genre analysis may, mutatis mutandum, be used for other aspects of literary texts, such as authorship, author gender, or time period. Finally, not all of the many aspects relevant to the study of literary genre can be discovered using quantitative methods.

Genre aspects and textual features

As a category fundamentally shaping texts, genre potentially affects any aspect of literary texts typically considered relevant in literary studies, whether they concern a text’s formal divisions, the way fictional time, space and events are organized, the way in which characters are described and how they relate to each other, a text’s themes and motives, or the stylistic or other formal features a text exhibits: “genres, like buildings, possess distinctive features at every possible scale of analysis: mortar, bricks, and architecture” (Allison et al. 2011, 8).

In quantitative genre analysis, it has turned out to be useful to distinguish three perspectives: the top-level genre category, which is used as a synthetic label for a given subgenre; the different aspects of genre, that is the various abstract characteristics used to define genres (on the levels of content, structure, character and style); and the textual features which make each aspect accessible to quantitative investigation. Dividing genre into its constituent aspects is instrumental in making formal analysis possible; at the same time, it opens up the possibility of reconceptualizing genres. Most defining aspects of genre are not, however, directly accessible to quantitative analysis. What is accessible are textual features of various types – sometimes called genre markers or cues – which can be related in a more or less direct way to specific aspects of genre (Kessler et al. 1997). Aspects of genre that boast readily identifiable textual features have been studied first, but with advances in Natural Language Processing, features indicative of more elusive aspects of genre will certainly come to be used very soon. In the meantime, consider this table of textual features or genre markers which have been studied up to now (Table 1):

Textual features	Content	Structure	Characters	Style
punctuation marks		+		+
word length		+		+
sentence length		+		+
content words	+
function words			+	+
part of speech			+	+
syntactic complexity		+		+
internal divisions		+
character names			+
character vs. narrator speech		+	+	+
topic models	+
vocabulary richness				+
textual complexity / richness				+

Table 1: Textual features or genre markers used in computational genre analysis. 

The following example analyses will focus on style and theme (via topics as well as function and content words). They revolve around the following questions: Which aspects are most relevant to a given genre distinction? Which textual features correlate with a given aspect of literary genre, and why? Is a given feature really more frequent in one genre than in the entire collection? Are observed differences really related to genre alone? What are the relations between characteristics that, according to experienced readers, describe or define a genre well and the features that differentiate two genres statistically?

The example collection

The analyses discussed here rely on a small set of ten novels, all written by British author Arthur Conan Doyle around the turn of the nineteenth century (Table 2). Four of them are clearly detective fiction (Sherlock Holmes novels, in fact), and detective fiction will be the main focus of the example analyses. Among the remaining six novels, which together will serve as our basis for comparison, two are clearly historical novels, two are sometimes considered horror novels, and two are sometimes called adventure or science-fiction novels.

Title	Publ.	Tokens	Subgenre	Detective?	Narration
A Study in Scarlet	1887	42k	detective	yes	homodiegetic
The Hound of Baskervilles	1902	57k	detective	yes	homodiegetic
The Sign of Four	1890	42k	detective	yes	homodiegetic
The Valley of Fear	1915	56k	detective	yes	homodiegetic
The Refugees	1893	119k	historical	no	heterodiegetic
The White Company	1891	148k	historical	no	heterodiegetic
The Mystery of Cloomber	1889	48k	horror	no	homodiegetic
The Doings of Raffles Haw	1891	37k	horror	no	homodiegetic
The Poison Belt	1913	29k	adventure	no	homodiegetic
The Lost World	1912	74k	adventure	no	homodiegetic

Table 2: The corpus of novels by Arthur Conan Doyle.

The novels in this collection display two phenomena that can frequently be encountered in genre analysis. First, there are rather clearly defined and easily recognized subgenres, such as detective fiction and historical novels, which have very distinctive plots, characters, themes, settings, vocabulary, and style. And there are less clearly-defined genres, with few or less clear-cut distinctive characteristics. The exact difference between an adventure and a horror novel, for example, is comparatively less clear-cut, although it may be argued that the intended effects on the readers are quite different. Second, novels can be more or less typical representatives of a given subgenre, sharing only some or almost all characteristics found in the abstract definition of a prototype. While A Study in Scarlet is a prototypical detective novel, The Poison Belt is an adventure novel that, in some respects, is not unlike a science-fiction novel.

Building text collections for genre analysis

The first issue to be addressed in genre analysis is that of building a text collection that will help isolate and discover the genre distinctions one is interested in (see Wynne 2004 for an introduction to corpus building). There are three basic ways a text collection can be built for genre analysis, depending on the project’s particular aims. The collection may have two groups of texts of uneven size, one (smaller one) with texts representing the target genre and one (larger one) representing a wide range of texts acting as a sample of representative writing of the same period. Alternatively, the collection could have two groups of similar size, one with texts representing the target genre and the other representing a comparison genre. Finally, the collection can have more than two groups of similar size, one containing texts representing the target genre and the others each containing texts representing one specific comparison genre. The Doyle collection can be construed as either the second or third case: the detective novels may be opposed to all non-detective novels; or the detective novels may be opposed to each set of the historical, horror or adventure novels.

Representativeness (in the sense of the collection being a statistically valid random sample from the complete literary production of the time or from a smaller, but relevant and complete subset) is usually very difficult to achieve for a number of reasons, not least among them the availability of metadata and texts. It usually makes more sense to focus on balancedness of the collection. This means the collection should purposefully reflect some of the major distributional facts one knows about the targeted genre and period. For example, if one were to build a collection of eighteenth-century plays and knew that about 20% of the plays written at the time had female authors, one could strive to include in the collection 20% of plays written by female authors. An alternative understanding of balancedness would consider that, for each subgenre contained in the collection, the proportion of plays written by female authors should be as similar as possible, even if that means it does not precisely reflect the historical reality.

This leads us to another aspect of collection building: that of the absence of undesired correlations between key variables. For example, if one is interested in the contrast between detective and adventure novels, one should avoid including a lot of detective novels by one author and a lot of adventure novels by another author. The contrasts one may find, may in that case reflect the contrasts between the two authors rather than between the two subgenres. In such a case, it is important to include in the collection examples for both subgenres written by several different authors. The same principle holds for other fundamental categories such as the year or decade of (first) publication or for narrative perspective. In any computational genre analysis, it is therefore necessary to collect and carefully check metadata – that is, information describing the texts – relating to the most relevant aspects of the texts apart from genre. At the very least, this includes authorship, date of publication, narrative perspective, an indication whether the text is in verse or prose and, possibly and controversially, author gender.

In the Doyle collection, for example, the detective novels are all narrated by Holmes’s sidekick Watson, so their narrative perspective is homodiegetic (the narrator is part of the narrated world, but he/she is not the protagonist). The historical novels are all narrated from a heterodiegetic perspective (where the narrator is situated outside the narrated world). Now, comparing the detective novels with the historical novels would generate misleading results because differences between the two groups might be caused by the difference in narrative perspective rather than by the genre difference in itself. For example, the first-person singular pronouns “I/me” are a lot less frequent in historical novels than in detective fiction; however, this is likely to be an effect of narrative perspective, not, say, due to the fact that historical fiction is less concerned with issues of identity than detective fiction. Therefore, it is preferable to contrast the detective novels either with the horror or adventure novels (which have the same narrative perspective as the detective novels), or with all other novels by Conan Doyle (most of which are written in the same narrative perspective).

Contrastive genre analysis

The basic assumption that motivates contrastive genre analysis is the following: a textual feature (such as a word, part-of-speech, topic, or punctuation mark) is important in one specific genre, or distinctive of that genre, not just by its pure (even relative) frequency. Rather, its distinctiveness, or “keyness” (Scott 1997), also depends on how frequent the feature is in the other genres in the collection. A word like “the” may be very frequent in the target genre, but it is also very frequent in all other genres, so that it is not distinctive of the target genre. What counts, rather, are the features that are overrepresented in the target genre relative to the comparison texts. In addition, it is not just the frequency of a feature that is important, but also its degree of dispersion, that is, how evenly distributed the feature is in the target and comparison texts (see Du et al. 2021). The more evenly-dispersed a feature is in the target texts, the more representative of these texts as a group it is likely to be. A number of statistical measures have been suggested for establishing these distinctive features; only a small selection of them will be discussed here.

Measures of distinctiveness (for words)

The ratio of relative frequencies, a very simple measure used for contrasting two groups of texts, is calculated as follows (for details, see Gries 2010). For each example of a given feature, derive its absolute frequency in the two groups of texts; calculate the relative frequency based on the overall feature count (length) of the groups; set the relative frequencies of the feature in one group against the frequencies in the other group; sort by descending ratio of relative frequency; examine the top and the bottom of that list. Converting the absolute frequencies to relative frequencies means taking into account the differing length of texts. The extreme values of the ratio between the features in the two sets correspond to the most distinctive features of each group of texts.

A measure that takes the degree of dispersion of word usage inside the groups into account is Zeta (see e.g. Craig and Kinney 2009). Here, texts are split into smaller segments and then, for each term, the proportion of segments containing that term at least once in the first group is set against the proportion of segments containing the term in the second group. Positive scores indicate that a term is distinctive of the first group, negative scores indicate that a term is distinctive of the second group. If we do this for the detective and adventure novels from the Doyle collection, focusing only on verbs, we obtain the following data (table 3), shown here alongside the relative frequencies of the verbs in the two genres (in descending rank).

highest relative frequencies		most extreme Zeta scores
detective	adventure	detective	adventure
be have say do come see know go take think	be have say do see come make take look get	concern print inform guard marry engage recognize inquire recommend teach	survive stroke project admit exhaust destine encounter boom cheer justify

Table 3: Verbs with the highest relative frequency compared with verbs with the highest Zeta value. 

The ten verbs with the highest relative frequency are all important verbs, but they are very general and, for the most part, not distinctive of or specific to either detective or adventure novels. Indeed, the list of verbs is very similar for both genres. The two lists of verbs by most extreme Zeta score, by contrast, are most distinctive of the two genres, and do not show any overlap at all. Some of them can immediately be related to the two genres in question, with “inform”, “recognize” and “inquire” designating cognitive acts associated with detective fiction, while “survive”, “exhaust” and “encounter” are actions that make sense in the context of adventure novels. Not all terms that are, statistically speaking, distinctive can be so easily interpreted. Also, this type of analysis could of course be conducted with other features, such as function words or punctuation marks.

Measures of distinctiveness (for topics)

We do not always want to limit ourselves to individual word tokens or other token-level features for genre analysis. Because genre is in part defined by thematic aspects, it can be interesting to use topic modeling in genre analysis as well, even though topics are not identical to themes (see Blei 2012). Here, a topic model with 60 topics was created using Mallet and topic scores for each of the four subgenre categories were calculated using Python, yielding a genre-topic-matrix.

A genre-topic-matrix is basically a table that shows the values for several topics in several genres. The values correspond to the median probability score for each combination of genre and topic. (The median is the most typical value among several values. The probability of a topic in a genre roughly expresses its ‘importance’.) Again, we need to offset the scores per genre by the scores in the collection. One simple but useful way of doing so, when using topic scores, is to apply a mean-normalization, obtained by subtracting the topic’s mean score in the collection from each individual score. The resulting data can usefully be visualized as a heatmap, with topics sorted by their decreasing variation across genres. The heatmap (figure 1) shows at a glance which topics are overrepresented (positive values) or underrepresented (negative values) in which subgenre.

We see the topics with their top three words on the vertical axis and the four subgenres on the horizontal axis, each cell reflecting the normalized score of one topic in one subgenre. A topic that is quite distinctive of detective fiction, unsurprisingly, is topic 41 (crime-detective-murder). Topic 46 (fact-point-reason) is also important in detective fiction, no doubt due to the special role of cognition and epistemology in this genre; however, this topic is also present in Doyle’s horror novels and even more important in his adventure novels. To some degree, then, and unlike Doyle’s historical novels (which show a notable absence of topic 46), these two subgenres (as practiced by Doyle) seem to share detective fiction’s interest in cognition and epistemology.

Conclusion

This chapter has aimed at justifying, explaining and illustrating a particular type of quantitative genre analysis based on contrasting normalized values for different types of features or textual cues. What becomes evident is that appropriate statistical calculations do indeed bring distinctive features of genre to the surface. However, the relation between statistically distinctive features, on the one hand, and findings that can be usefully interpreted from the point of literary studies, remains complex. As Allison et al. find:

Statistical findings […] made us realize that genres are icebergs: with a visible portion floating above the water, and a much larger part hidden below, and extending to unknown depths. Realizing that these depths exist; that they can be systematically explored; and that they may lead to a multi-dimensional reconceptualization of genre: such, we think, are solid findings of our research. (Allison et al. 25)

References

Allison, Sarah, Ryan Heuser, Matthew L. Jockers, Franco Moretti, and Michael Witmore. 2011. “Quantitative Formalism: An Experiment”. In: Literary Lab Pamphlets. Standford Literary Lab.

Alpaydin, Ethem. 2010. Introduction to Machine Learning. 2nd ed. Cambridge: MIT Press.

Biber, Douglas, and Susan Conrad. 2009. Register, Genre, and Style. Cambridge: Cambridge UP.

Blei, David M. 2012. “Probabilistic Topic Models”. Communications of the ACM 55.4:77–84.

Craig, Hugh, and Arthur F. Kinney. 2009. Shakespeare, Computers, and the Mystery of Authorship. Cambridge: Cambridge UP.

Calvo Tello, José. 2021. The Novel in the Spanish Silver Age: A Digital Analysis of Genre Using Machine Learning. Bielefeld: transcript.

Du, Keli, Julia Dudar, Cora Rok, and Christof Schöch. 2021. “Zeta & Eta: An Exploration and Evaluation of Two Dispersion-Based Measures of Distinctiveness.” Proceedings Computational Humanities Research 2021, https://ceur-ws.org/Vol-2989/. 

Gries, Stefan. 2010. “Useful Statistics for Corpus Linguistics”. In A Mosaic of Corpus Linguistics: Selected Approaches, edited by Aquilino Sánchez and Moisés Almela, 269–91. Frankfurt: Peter Lang.

Kessler, Brett, Geoffrey Numberg, and Hinrich Schütze. 1997. “Automatic Detection of Text Genre”. arXiv.org: cmp-lg. https://arxiv.org/abs/cmp-lg/9707002.

Scott, Mike. 1997. “PC Analysis of Key Words and Key Key Words”. System 25.2:233–45.

Underwood, Ted. 2019. Distant Horizons. Digital Evidence and Literary Change. Chicago: The University of Chicago University Press.

Wellek, René, and Austin Warren. 1942. Theory of Literature. New York: Harcourt Brace & Company.

Wynne, Martin, ed. 2004. Developing Linguistic Corpora: A Guide to Good Practice. Oxford: AHDS. http://www.ahds.ac.uk/creating/guides/linguistic-corpora/index.htm.

Note

This text has initially been prepared in 2015-2016, then revised in 2019 and 2021, for an edited volume on Digital Humanities for Literary Studies: Methods, Tools & Practices. Despite the best efforts of all involved, the volume never saw the day. For this reason, the text is now made available here. 

Citation suggestion

Christof Schöch (2022): “Computational Genre Analysis”. The Dragonfly’s Gaze. Marseille: OpenEdition. URL: https://dragonfly.hypotheses.org/1219, DOI (archive copy): 10.5281/zenodo.7553854. 

 

---

---