Teaching and Research, or: a Post-scriptum to ‘Topic Modeling Genre’

Wouldn’t it be nice if teaching and research could always be closely connected? And I don’t just mean students benefitting from their instructors being active researchers. I also mean it the other way around, with researchers benefitting from the work they are doing with the students. The true Einheit von Forschung und Lehre (i.e., the unity of research and teaching) that German academics like to trace back to Humboldt. While I do feel this unity is often the case, just as it sometimes definitely feels like it is totally not, the other day, I experienced a particularly striking example where it was the case. 

This semester I am teaching a class called “Programming 2: Machine Learning”, where we are learning to use Python to perform regression, clustering and classification on datasets relevant to the Digital Humanities. The other day, we were practicing logistic regression, as a sort of segway from regression to classification. It is really important to me that we use datasets relevant to the humanities for this, and sometimes good datasets are hard to come by. So the dataset I proposed we use came from a study I have done several years ago, using Topic Modeling on a corpus of French Drama (shameless plug follows here: see the article in DHQ that resulted from that study). The dataset documenting that study contains the probabilities of 60 different topics in about 5000 segments from the drama corpus I had used, with the dramas falling into three classes: tragedies, comedies and tragi-comedies. Now before you say: Ah, old hat, when are you finally going to move on, that is exactly my point. 

In any case, this dataset seemed to be just the right size and difficulty to ask the students to come up with a solution – using logistic regression as implemented in sklearn – for the following question: Which topics show the strongest association with, or are the best indicators for, tragedy and comedy, respectively? 

The students found out, like I had done when preparing the programming task, that the two topics in question where the following: on the one hand, topic 39_sang-mort-main (i.e., “blood, death, hand”, so a nicely explicit ‘violent death’ topic) with the highest coefficient for tragedy. And on the other, topic 36_bon-monsieur-beau (i.e., “good, gentleman, beautiful”, as an appropriately polite ‘well-mannered gentleman’ topic) with the highest coefficient for comedy. Those are precisely the two topics that also turned out to be the most distinctive of the two genres in the original study (see figure 8 in the DHQ study, if you’re interested). The only topics with a higher standard deviation across subgenres were two topics that were either over-represented only in tragi-comedy (which we didn’t take into account) or were equally under-represented in both tragedy and tragi-comedy (and came up in second place in our test). But the clearest contrast was in topics 39 and 36. So far, so good. 

However, I then did a few plots, among them a very simple scatterplot showing, for all of the tragedies and comedies in the dataset, the probabilities of topics 39 and 36, on the x and y-axis, respectively. And that is where things got interesting and I learned something new: 

Scatterplot of plays and the probabilities of two topics

Figure 1: Scatterplot of plays and the probabilities of two topics. 

What do we see? First of all, the blue squares are the tragedies, and the red dots are the comedies. The two groups have clearly distinct values in the two topics, but not in the way I had represented things in my mind before. If a topic is strongly associated with one group of texts, it should have high probabilities in the texts belonging to that group, right? Well, that’s what I thought, too, but I was wrong. 

What we see here instead is that topic 39, our ‘violent death’ topic, has a high range of possible probabilities in tragedy, from almost non-existent (on the bottom left of the plot) all the way up to around 0.65 (at the bottom right). So in some tragedies, it is important, while in many others, not so much. Inversely, topic 36, our ‘well-mannered gentleman’ topic, has a similarly high range of possible probabilities in comedies. This means that there are many tragedies in which the tragedy-topic is not very strongly-represented, just as there are many comedies where the comedy-topic is not very strongly represented. How, then, can these topics be good predictors of comedy and tragedy? 

Well, the other thing we see in the plot is that there is almost no tragedy that has any significant probability on the comedy topic, just as there is almost no comedy that has any significant probability on the tragedy topic. This means that the tragedy topic is a good predictor of tragedy because it is absent from the comedies, and inversely, the comedy topic is a good predictor of comedy because it is absent from the tragedies. And it makes sense: there can be tragedies with or without violent deaths being discussed explicitly and at length, but there cannot be a comedy that even approaches the topic. And inversely, there can be comedies with or without gentlemen being called good and beautiful, but there cannot be tragedies that have any characters of the sort (well, maybe a few). 

The regression plots we did confirm this, by the way (see below). 

Two regression plots for the tragedy and the comedy topic

Figure 2: Two regression plots for the tragedy and the comedy topic

Here, the absence of any high probabilities of the topic for one category nicely contrasts with the wide range of probabilities (but still with a focus on lower probabilities) of the topic for the other category. The regression line hesitates for the plays with low values in both topics but quickly moves away from the class for which the topic has no examples, even when the topic probabilities are still quite low for the other class. (Note that the vertical spread of the points is exclusively due to jitter added to make the density of the points more perceptible.) 

Beyond this particular insight on the ex-negativo distinctiveness, so to speak, of these two topics for the French tragedies and comedies, there are some more things I have learned thanks to teaching this dataset: First, this phenomenon may be more widespread than I had assumed. Second, it is always wise to look at your data from as many angles as possible, including using visualizations. And finally: sometimes, teaching and research do indeed go hand in hand!

 



Cite this blog post
Christof Schöch (2023, November 23). Teaching and Research, or: a Post-scriptum to ‘Topic Modeling Genre’ The Dragonfly's Gaze. Retrieved June 23, 2024, from https://doi.org/10.58079/nwfj

Christof Schöch

Christof Schöch is Professor of Digital Humanities at the University of Trier, Germany, and scientific Co-Director of the Trier Center for Digital Humanities. He is vice president of the Digital Humanities Association for the German-speaking area (DHd), President of the Alliance of Digital Humanities Organizations' (ADHO) Constituent Organization Board and co-editor of the Journal of Computational Literary Studies (JCLS). 

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Search OpenEdition Search

You will be redirected to OpenEdition Search