method, a corpus linguistic method for extracting valence compounds for German
out of a very large corpora. Now, what are valence compounds? There's also a German
technical term in the literature of German linguistics called Wechsians compositor.
There are noun compounds that have a D verbal head and a nominal non-head. In
other words, I'll just give you an example, I think that will make it clear. The
German compound alto farre has as its head the second component of the compound
farre, and this is derived from the verb farren, and a direct object alto, which
is the object of the verb farren. So alto farre means car driver, and the first
part of the compound serves as the object in that compound. Now, it turns out
that not in all compounds that have farre is the second element, it's the first
element that's an object. So fern farre, long distance driver. There fern is some
kind of modifier, but it's not a direct object. Or formula one driver, formula eins
farre. Again, formula one is not an object. There are also cases of compounds like this,
of so-called rechtsians or valence compounds, where it's ambiguous whether the first part
of the compound is serving as the subject or the object. So in student interview,
Schiller befragung, Schiller could either be the people who are doing the questioning
or the people being questioned. So the task of trying to figure out what grammatical
role the first part of such a compound fulfills is not a trivial task. Now, why
bother about this problem, right? Why should we even worry about it? Well, for
those of you since we're at the Digital Humanities Conference who have read Mark
Twain, right, he thinks that the wonders of the German language all have to do with
compounds and that you couldn't possibly ever understand this language if you don't
know about compounds. But there is also less anecdotal evidence and less humoristic
evidence that shows that compounds are everywhere. Two cases in point. Marco
Baroni in a recent large study of large web corpora has guesstimated that about 40
percent of all nouns in German newspaper text are actually compounds. And a similar
finding has been made by Lothar Lemnitzer who has what he calls the word watch,
which goes through German newspaper texts on a weekly basis to see what new words
appear that haven't been published before. And surprise, surprise, almost exclusively
the new words are compounds, right? So it is worthwhile to look at this question. And
our leading question here was, can we simply apply digital humanities techniques, fairly
straightforward techniques from corpus linguistics? And in particular, our hypothesis was that
we should be able to reliably detect valence compounds where the first part of the compound,
the nominal part, is an object. Now a little bit about the data that we based our experiments on.
We used a very large newspaper corpus that we have available and that we've been processing
at in my research group for quite some time. It's called the TUP-DZ. Never mind this weird
acronym, it has to do with a family of tree banks that we've constructed. And this one stands for
partially parsed corpus for D, for German, and Z stands for Zeitung's text, right? So this is
actually the whole CD that the German daily newspaper Die Tagus Zeitung-Tatz has put out,
and we've parsed all of it, and it's a sizable corpus of 200 million words. Now for this
particular project, what we needed to do is to get a syntactic parser, and we used for this a
dependency parser called the MALT parser that was developed by Joachim Niebrer and his team. We
trained it actually on another German tree bank that we also have, in that case, automatically
annotated and trained this MALT parser on this newspaper text and then ran the 200 million word
corpus through the MALT parser. And we also obviously needed to figure out what the morphological
analysis of particular words are, in particular whether it's accusative, whether it has case
accusative, because that's one of the indicators, of course, in German for whether one of the
indicators, whether something is a direct object or not. So this little colorful representation
here is not some kind of weed, but this is the dependency analysis for a sentence like,
Dan Vöderdachnibant meer autofahren. For those of you familiar with, you know, dependency parsing,
you'll see that the meer more is actually wrongly attached here, it should be attached to fahren
rather than auto. So this is just to say that the output, since this is all automatically parsed,
is noisy, there are mistakes in here, and of course this also, in the long run, affects the
results. Now what we did then, and just want to make sure that it's kind of clear how we did the
data analysis and came up with the classification, what we did was we took a non-compound like
autofahrer, we broke it up into its components, and then we will look in the TUP corpus, in this
very large German corpus, the co-occurrences of these two words. So auto and fahren versus
fahren and some other object, say fahrad, but also the co-occurrence of auto versus some other verb,
or take that other verb and see what other objects it can occur with. So if you look at these four
cells of data, then you can use log-likelihood or mutual information to, as it were, compute the
strength of association between auto and fahren and figure out whether this is a very strong
association, which of course is a good indicator for the classification of auto as a direct object
or not. We also used a third measure, a simpler measure of relative frequency. This doesn't look,
this doesn't take into account all these bins that are shown on the slide, but rather what it looks
at is the relative frequency with which auto appears as an accusative case, direct object,
or in some other grammatical function with the same verb. So these are the three statistical
measures that we used and we wanted to evaluate how well they did with respect to each other.
Now for this purpose, we created a hand annotated corpus of 200 examples, half of which are actually
valence compounds, the other half are not, because not all German compounds are valence compounds,
you also have things like car insurance, for example, where it's not a drive nominal that
appears as the second, as the head constituent of this compound, and so those of course shouldn't
be classified either. And in order to have a good evaluation, we took half of the compounds to be
valence compounds, of course, among those not all will be direct object valence compounds and the
other ones simply being so-called ungoverned compounds. Okay, so that was the setup for the
evaluation. Now I'll briefly talk about the results in the last minute. So on the left,
you see the results for the log likelihood evaluation and what you actually see on the x-axis
here, both in the figure on the left and on the right, are association strength thresholds
that correspond to a quantile between 10% and 100% of the values observed for these measures.
And the y-axis then shows the corresponding effect that this thresholding has on precision
and recall for each measure. And so what you see is that the precision, it's probably a little hard
to, yeah, you can see it, the precision in the right figure here is best for, you know,
comparatively speaking to the log likelihood, it's best for mutual information. However,
recall is better with log likelihood and actually also with relative frequency. And here, of course,
it's a different thresholding and the precision is very good, but recall turns out to be pretty bad.
Okay, I think I have probably seven and a half seconds future work. What we have done so far
is simply look at core current strength, association strength. We haven't taken
semantic similarity into account at all. One of the things that I think is worthwhile doing,
and we're in a good position to actually do, is looking at if you have a candidate that is
by all these association measures, a very strong candidate for an object interpretation,
look at other nouns that are semantically very similar. And again, I'm giving you this
car driver example, right? So car driver, tank driver, bus driver, right? All of these
are vehicles, right? And they all have this object interpretation. So if you actually start
looking at so-called families of these semantically similar nouns, then that can give you
additional reliable evidence, but this is yet in the future. We are in a good position
to do this because we have developed a German wordnet of considerable size that will allow
us to look at not only syntactic, but also semantic similarity. Thank you for your attention.