• angle-up angle-right angle-down angle-left close user menu open menu closed search globe bars phone store

    Automated Scoring and Natural Language Processing

    Natural Language Processing (NLP)

    At ETS, our researchers have extensive experience in Natural Language Processing (NLP) — a field that applies principles of linguistics and computer science to create computer applications that interact with human language.

    NLP technology is the basis for the automated scoring applications that we are developing to address the increasing demand for open-ended or constructed-response test questions, which elicit responses such as extended writing responses (e.g., essays), shorter written responses to subject-matter items, and spontaneous speech. In our research, we also seek ways to build NLP technology into:

    • classroom support tools that teachers can use to help maximize the time and resources they have available for instruction
    • assessment creation tools that can aid in the test development process

    Automated Scoring

    ETS has been at the forefront of research in automated scoring of open-ended items for over two decades, with a long list of significant, peer-reviewed research publications as evidence of our activity in the field. ETS scientists have published on automated scoring issues in the major journals of the educational measurement, computational linguistics and language testing fields. Their work has also resulted in over 50 U.S. patents related to NLP applications for assessment, significantly more than any other organization.

    The topic of automated constructed-response scoring has begun to receive substantial attention in the context of discussions related to assessment reform in the United States, as ETS measurement professionals and their colleagues in other assessment organizations noted in the recent report, Automated Scoring for the Assessment of Common Core Standards. This report highlights five key requirements to verify when considering the use of automated scoring systems:

    • Automated scores are consistent with the scores from expert human graders
    • The way automated scores are produced is understandable and substantively meaningful
    • Automated scores are fair
    • Automated scores have been validated against external measures in the same way as is done with human scoring
    • The impact of automated scoring on reported scores is understood

    ETS is committed to developing automated scoring systems to meet these conditions, and evaluating them accordingly. Responsible application of automated scoring requires evaluation of all five conditions; using agreement with human raters as the sole basis for assessing the performance of a scoring system can misrepresent the effects of introducing it into large-scale operational use.

    Our Research

    Learn more about our research in automated scoring and natural language processing related to topics such as these:

    Find More Articles

    View a comprehensive list of publications related to automated scoring and natural language processing.

  • 北京 快乐8 大星彩票七乐彩走势图百度 陕西十一选五前三组 老时时彩手机走势图 天津快乐10分彩票 凯时百家乐 七星彩走势图可下载 云南时时彩计划软件手机版 下载浙江快乐12走势图 内部透码信封图片 江苏7位数可以在外省对奖吗 七星彩走势图近260期 北京快三助手下载安装 上海时时乐注册 独来独往代表什么生肖