Connecting our Tools, or: Stylometry and Network Analysis made easy

Do you like stylometry, and maybe have tried out the scripts for R provided by the Computational Stylistics Group? And do you like visualisation, and maybe have played around a bit with Gephi, the network visualisation tool? I suppose it’s rather likely that you have, as a reader of this blog, or at least that your interested in this kind of thing. Well, then you should be excited: Maciej Eder just published a new version of the stylo script that produces an output file you can import into Gephi, and visualize with a few clicks! This is just what we need right now, these little but essential things that connect tools and in this way create empowering tool-chains!

Fig. 1: Cluster Analysis Graph for 223 plays (License: CC-BY-SA 3.0)

So, how does that work? It is really easy to do, and this is how I did it. I downloaded the latest, brand new version of the script (0.4.9.1, in these guys’ modest counting) from the CSG’s page. First, I ran one of my usual stylometric analyses, using a rather large amount of documents (in this case, 223 comedies, tragedies and some other plays, all from the seventeenth century), with some basic parameters that usually work nicely. (In my case: 1200 most frequent words, Cluster Analysis, Eder’s Delta, no bells and whistles, no fine-tuning of any kind).
When you do this, you run into a visualization problem: Cluster Analysis Graphs naturally tend to get very stretched out when you have high numbers of documents and it gets really difficult to get an overview of the data (Fig. 1: Cluster Analysis Graph for 223 plays; part of the graph to the left has been cropped; the two lines connect further to the left). What this graph shows, however, is that there is a high-level divide between comedies and tragedies, with the comedies being divided into verse and prose plays. Only after that, there are author-related smaller clusters. There are some very clear author sub-clusters and some clusters which are really more of a mixed bag. This is a very rough picture with quite a bit of noise, typical of a quick stylometry run.
But besides this graph and some other visualisation options, you now have a very powerful addition! Along with the other output files, the stylo script now produces an output file called “temp.csv”.[1] This file, I assume, is derived from the distances matrix the script produces, and contains, for every pair of documents in your corpus,[2] a value corresponding to the distance between these two documents, with a value between 0 and 1. There are five columns in that file: a running number, source, target, weight (the “distance” value) and type (undirected in this case). This is the bare minimum that Gephi needs.
And this is the file you import into Gephi. To do so, start a new project, switch to the “Data Laboratory” view, and import the file in question via the “import spreadsheet” command. Make sure to choose the “Edge table” as the import target, set the “weighs” column to “float” (because these are values between 0 and 1) and import. Move back to the Overview window and check out the network. Before you really get a nice network, there is of course some work to do, like choosing an appropriate network calculation method (ForceAtlas2 worked fine for my data, with an increased gravity to pull things a little closer together), and some additional adjusting of labels and colors.

Fig. 2: View just of the nodes without edges, for 223 plays. (License: CC-BY-SA 3.0)

Before going any further with this (partitioning the data by genre and prose/verse would obviously be of interest to me), I just wanted to get a quick first look at my data, and here’s what I found (see Fig. 2: Network Analysis Graph of 223 plays, ForceAtlas2, nodes only):

• The three large groups found in the Cluster Analysis Graph do not seem to show up here, at least not in such an obvious way. Rather, there are quite a number of smaller groups in the center, all relatively closely connected to each other, as well as some isolated cluster with weak or no links to the rest of the nodes.
• These isolated groups are not as visible in the Cluster Analysis Graph as they are here. So what are they? The group at the left is made up of three smaller groups, with tragedies by Racine, Pradon and Campistron; the groups at the right are Quinault plays of various genres (at the top); tragedies by Du Ryer (middle), and tragedies by La Calprenède (bottom).
• The larger middle region is less clear-cut. The plays to the bottom tend to be comedies in prose, the middle part to the right tend to be comedies in verse, and the middle part to the left and the upper part tend to be tragedies in verse.

The static images have their limitations, so it is really best you look at a dynamic version. For those of you who want to do this, I provide a ZIP-archive with the output file from the stylo script as well as a gexf-file to open with Gephi. Although this is really just a preliminary exploration of this new feature,  I’d be interested to hear what this new way of visualizing cluster analysis data inspires you!

1. That could maybe be renamed to something like “network-data.csv”. [↩]
2. Actually, not every pair: some plays are assumed to have no connection, the way this is implemented right now; for 223 plays, I get 669 nodes, exactly three times the number of plays; I need to ask Maciej about the details. [↩]