Shallow Semantic Reasoning from an Incomplete Gold Standard for Learner Language

Levi King and Markus Dickinson

Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications

We investigate questions of how to reason about learner meaning in cases where the set of correct meanings is never entirely complete, specifically for the case of picture description tasks (PDTs). To operationalize this, we explore different models of representing and scoring non-native speaker (NNS) responses to a picture, including bags of dependencies, automatically determining the relevant parts of an image from a set of native speaker (NS) responses. In more exploratory work, we examine the variability in both NS and NNS responses, and how different system parameters correlate with the variability. In this way, we hope to provide insight for future system development, data collection, and investigations into learner language.


Electronically available file formats:


Bibtex entry:

@InProceedings{king:dickinson:16,
  author    = {King, Levi  and  Dickinson, Markus},
  title     = {Shallow Semantic Reasoning from an Incomplete Gold Standard
               for Learner Language},
  booktitle = {Proceedings of the 11th Workshop on Innovative Use of NLP 
               for Building Educational Applications},
  month     = {June},
  year      = {2016},
  address   = {San Diego, CA},
  pages     = {112--121},
  url       = {http://cl.indiana.edu/~md7/papers/king-dickinson16.html}
}