{"id":33408,"date":"2019-03-25T16:45:06","date_gmt":"2019-03-25T15:45:06","guid":{"rendered":"http:\/\/www.h-its.org\/downloads\/evaluation-metrics-for-end-to-end-coreference-resolution\/"},"modified":"2019-05-23T10:56:46","modified_gmt":"2019-05-23T08:56:46","slug":"evaluation-metrics-for-end-to-end-coreference-resolution","status":"publish","type":"hits-software","link":"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/","title":{"rendered":"Evaluation Metrics for End-to-End Coreference Resolution"},"content":{"rendered":"\n<p>ommonly used coreference resolution evaluation metrics can only be&nbsp;applied to key mentions (i.e.already annotated mentions). We here&nbsp;propose two variants of the BCubed&nbsp;(Bagga and Baldwin, 1998)&nbsp;and CEAF (Luo, 2005) coreference resolution&nbsp;evaluation algorithms which can be used to evaluate coreference resolution&nbsp;systems dealing with system mentions (i.e. automatically determined mentions).The algorithms and relevant analysis can be found in details in our&nbsp;<a href=\"http:\/\/www.aclweb.org\/anthology\/W\/W10\/W10-4305.pdf\">SIGDIAL 2010 paper<\/a>.Both BCubedsys and CEAFsys java classes are available for download, along with a couple of necessary parent classes. They should be used within the BART framework, whose repository is available&nbsp;<a href=\"http:\/\/www.assembla.com\/wiki\/show\/bart-coref\">here<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Download<\/h2>\n\n\n\n<p>Click\u00a0<a href=\"https:\/\/cosyne.h-its.org\/nlpdl\/lea\/evalmetrics.zip\">here<\/a>\u00a0to download BCubedsys and CEAFsys.<\/p>\n\n\n\n<p>Publications related to evaluation metrics for end-to-end coreference resolution<br><\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>SIGDIAL\u201910 (<a href=\"http:\/\/www.aclweb.org\/anthology\/W\/W10\/W10-4305.pdf\">PDF<\/a>)<br>Cai, Jie; Strube, Michael (2010).<br>Evaluation metrics for end-to-end coreference resolution systems.<br>In<em>Proceedings of the SIGdial 2010 Conference: The 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue,<\/em>Tokyo, Japan, 24-25 September 2010, pages 28-36.<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>ommonly used coreference resolution evaluation metrics can only be&nbsp;applied to key mentions (i.e.already annotated mentions). We here&#038;&#8230;<\/p>\n","protected":false},"featured_media":0,"template":"","hits-research-group":[1302],"hits-software-category":[],"class_list":["post-33408","hits-software","type-hits-software","status-publish","hentry","hits-research-group-nlp-de"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Evaluation Metrics for End-to-End Coreference Resolution - HITS gGmbH<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Evaluation Metrics for End-to-End Coreference Resolution - HITS gGmbH\" \/>\n<meta property=\"og:description\" content=\"ommonly used coreference resolution evaluation metrics can only be&nbsp;applied to key mentions (i.e.already annotated mentions). We here&amp;...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/\" \/>\n<meta property=\"og:site_name\" content=\"HITS gGmbH\" \/>\n<meta property=\"article:modified_time\" content=\"2019-05-23T08:56:46+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data1\" content=\"1\u00a0Minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/software\\\/evaluation-metrics-for-end-to-end-coreference-resolution\\\/\",\"url\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/software\\\/evaluation-metrics-for-end-to-end-coreference-resolution\\\/\",\"name\":\"Evaluation Metrics for End-to-End Coreference Resolution - HITS gGmbH\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/#website\"},\"datePublished\":\"2019-03-25T15:45:06+00:00\",\"dateModified\":\"2019-05-23T08:56:46+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/software\\\/evaluation-metrics-for-end-to-end-coreference-resolution\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.h-its.org\\\/de\\\/software\\\/evaluation-metrics-for-end-to-end-coreference-resolution\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/software\\\/evaluation-metrics-for-end-to-end-coreference-resolution\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Software\",\"item\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/software\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Evaluation Metrics for End-to-End Coreference Resolution\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/#website\",\"url\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/\",\"name\":\"HITS gGmbH\",\"description\":\"Heidelberg Institute for Theoretical Studies\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.h-its.org\\\/de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Evaluation Metrics for End-to-End Coreference Resolution - HITS gGmbH","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/","og_locale":"de_DE","og_type":"article","og_title":"Evaluation Metrics for End-to-End Coreference Resolution - HITS gGmbH","og_description":"ommonly used coreference resolution evaluation metrics can only be&nbsp;applied to key mentions (i.e.already annotated mentions). We here&...","og_url":"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/","og_site_name":"HITS gGmbH","article_modified_time":"2019-05-23T08:56:46+00:00","twitter_card":"summary_large_image","twitter_misc":{"Gesch\u00e4tzte Lesezeit":"1\u00a0Minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/","url":"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/","name":"Evaluation Metrics for End-to-End Coreference Resolution - HITS gGmbH","isPartOf":{"@id":"https:\/\/www.h-its.org\/de\/#website"},"datePublished":"2019-03-25T15:45:06+00:00","dateModified":"2019-05-23T08:56:46+00:00","breadcrumb":{"@id":"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.h-its.org\/de\/software\/evaluation-metrics-for-end-to-end-coreference-resolution\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.h-its.org\/de\/"},{"@type":"ListItem","position":2,"name":"Software","item":"https:\/\/www.h-its.org\/de\/software\/"},{"@type":"ListItem","position":3,"name":"Evaluation Metrics for End-to-End Coreference Resolution"}]},{"@type":"WebSite","@id":"https:\/\/www.h-its.org\/de\/#website","url":"https:\/\/www.h-its.org\/de\/","name":"HITS gGmbH","description":"Heidelberg Institute for Theoretical Studies","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.h-its.org\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"}]}},"publishpress_future_action":{"enabled":false,"date":"2026-04-23 08:08:59","action":"change-status","newStatus":"draft","terms":[],"taxonomy":"hits-research-group","extraData":[]},"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/www.h-its.org\/de\/wp-json\/wp\/v2\/hits-software\/33408","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.h-its.org\/de\/wp-json\/wp\/v2\/hits-software"}],"about":[{"href":"https:\/\/www.h-its.org\/de\/wp-json\/wp\/v2\/types\/hits-software"}],"wp:attachment":[{"href":"https:\/\/www.h-its.org\/de\/wp-json\/wp\/v2\/media?parent=33408"}],"wp:term":[{"taxonomy":"hits-research-group","embeddable":true,"href":"https:\/\/www.h-its.org\/de\/wp-json\/wp\/v2\/hits-research-group?post=33408"},{"taxonomy":"hits-software-category","embeddable":true,"href":"https:\/\/www.h-its.org\/de\/wp-json\/wp\/v2\/hits-software-category?post=33408"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}