Category: Tools

3

Topic Modeling with MALLET: Hyperparameter Optimization

This is a short technical post about an interesting feature of Mallet which I have recently discovered or rather, whose (for me) unexpected effect on the topic models I have discovered: the parameter that controls the hyperparameter optimization interval in Mallet.[1] Yes, there are parameters, there are hyperparameters, and there are parameters controlling how hyperparameters are optimized. For a long...

0

Detecting Transpositions when Comparing Text Versions using CollateX

The aims of this post are to explain what collation is, why detecting transpositions is special, and how to accomplish it using CollateX. The example I will be using involves comparing two versions of the recent best-selling novel The Martian by Andy Weir, a novel whose publishing history Erik Ketzan and me are currently investigating. More specifically, I will illustrate...

Topic Clustering Dendrogram 0

What’s in my topic model? Or: Clustering topics by semantic similarity

For the last year or so, one my major interests has been topic modeling and applying it to a variety of French literary texts. I have been doing this as part of an exploration of the computational analysis of literary genres in our Würzburg junior researchers group. One of the interesting things to come out of all of this has...

0

What’s the simplest way of telling verse and prose apart?

Everyone knows that verse and prose are different. Well, there are two exeptions: the first is Monsieur Jourdain in Molière’s play Le bourgois gentilhomme, who only learns about that difference from his “maître de philosophie”. The other exception is the computer, for which both prose and verse are simply and equally strings of characters. Now, if you have nice TEI-encoded texts,...

1

How to Create Lemmatized (French) Text for Topic Modeling

It would not, some years ago, have occurred to me that anyone would want to reduce literary texts to the following pitiful state: “me avoir tu faire un rapport bien sincère / ne déguiser tu rien de ce que avoir dire mon père / tout mon sens à moi-même en être encor charmer : / il estimer Rodrigue autant que...

0

Extracting selected text from XML files using lxml in Python

As you can probably guess from the title, this post is a companion piece to another recent post dealing with the creation of simple TEI from HTML in Python (actually, using Python as a wrapper to several Regular Expressions). Once you have created such TEI files, have annotated them more richly by hand or by some other means, or were...

12

What the perfect repository for text analysis looks like (to me)

The longer I work with various collections of literary texts, available in various formats, and for use with various tools, the more I would like to have a nice repository which me and others could use to ingest, store, transform, update and extract text collections. So what would this repository look like? Basically, I’m describing a use case, and would...

0

Getting a closer look: More on visualizing results from stylometry with Gephi

This post is a follow-up post to my last brief post on the new version of the stylo script and its output for visualization with Gephi. Today, I just want to report on what Maciej told me about the technical details, and to show some more graphs resulting from my data. First things first, then. Here is how the data for...

1

Connecting our Tools, or: Stylometry and Network Analysis made easy

Do you like stylometry, and maybe have tried out the scripts for R provided by the Computational Stylistics Group? And do you like visualisation, and maybe have played around a bit with Gephi, the network visualisation tool? I suppose it’s rather likely that you have, as a reader of this blog, or at least that your interested in this kind...

1

Unsupervised relative distances, or supervised clear-cut decisions: more work (or play) with Eder/Rybicky’s stylometry script

The biggest Digital Humanities event of the year is over: DH2012 in Hamburg. And one of the major topics seemed to be, at least from my perspective, stylometry. Several panels and some separate papers were directly concerned with stylometry, during the main conference. I’ll have to postpone more thorough comment on the ones I heard and liked best to another...

0

Taking Eder/Rybicki’s Stylometry Script for a test drive

One of the things I planned to do with this blog is to document my experience with various tools for computational text analysis. This is a way of letting other people know what these tools can do, and what I am currently working on and maybe hear their (your!) thoughts on it; it is also a way of forcing myself...