Evaluation Metrics for End-to-End Coreference Resolution
Commonly used coreference resolution evaluation metrics can only be applied to key mentions (i.e.already annotated mentions). We here propose two variants of the BCubed (Bagga and Baldwin, 1998) and CEAF (Luo, 2005) coreference resolution evaluation algorithms which can be applied to coreference resolution systems dealing with system mentions (i.e. automatically determined) mentions.
The algorithms and relevant analysis can be found in details in our Sigdial 2010 paper here.
Both BCubedsys and CEAFsys java classes are available for download, along with a couple of necessary parent classes. They should be used within the BART framework, whose repository is available here.
