They seek out hateful language in video titles. They decide whether to classify clips as "offensive" or "sensitive." They are Google's so-called "ads quality raters," temporary workers hired by outside agencies to render judgments machines still can't make all on their own. And right now, Google appears to need these humans' help—urgently.

YouTube, the Google-owned video giant, sells ads that accompany millions of the site's videos each day. Automated systems determine where those ads appear, and advertisers often don't know which specific videos their ads will show up next to. Recently, that uncertainty has turned into a big problem for Google. The company has come under scrutiny after multiple reports revealed that it had allowed ads to run against YouTube videos promoting hate and terrorism.