The Learning Curve: Applying Game Rating Systems to Data Labeling

Recorded Nov 19th @ 8PM Coordinated Universal TimeOnline

About The Webinar

In this tech talk, we show how game rating systems can be applied to NLP and Search relevance data labeling at Scale AI to track the performance of crowd-workers on our platform.

Agenda

  • The challenge of skill assessment
  • Overview of game rating systems
  • How game rating systems work
  • Analysis of the results, pros & cons, and areas for future improvement

Presented By

Logan Ford

Logan Ford

Logan Ford is an engineering lead on the NLP and Search Evaluation team where he works on applying data science to ensure quality on subjective labeling use-cases. He previously co-founded Klarity, a Y-Combinator startup using machine learning to analyze legal contracts, and received his B.S. and M.Eng. in Computer Science from MIT.

The Learning Curve: Applying Game Rating Systems to Data Labeling