How do human raters improve online services? There is a lot of information here about the different kinds of jobs that a rater does for companies such as Appen Global, Lionbridge, and Leapforce. And over the past year or so, there are a lot more jobs than just your average rater job. There are even more types of jobs now than before. Everything from recording your voice for voice recognition applications, to social media evaluation. What isn’t generally talked about it the process and what you are actually providing as a service.

What it mostly comes down to is data. In reality, most of these jobs are simply data collection. Facebook, Instagram, and Google all require some type of data collection in order to create programming for their algorithms. Most of you who work for these companies probably don’t realize what it is that you are providing. In fact, your job is a lot more important than you think.

Google was always at the forefront of providing the best user experience for its searchers or users. And many people think that it is just their algorithm that does all the work. But no one really knew until recently when the Google Guidelines were leaked online that humans actually had something to do with it directly. As a rater, you are not actually rating the website, or giving the website you are looking at, a grade. Instead, you are using the guidelines written by Google engineers, to give the algorithm its data.

The data that you return for doing rating is actually fed to the algorithm where it judges and grades websites based on the real world interactions through raters. It is often thought that the job you are doing is actually affecting those pages and ads that you see. In a way, you are affecting them, but not directly. It is only by the data you collect that will affect websites, ads, and services in the long run once the algorithms are programmed using your data.

The same can be said for Facebook and what some of the social media evaluation jobs do. For instance, the Falcon Video Recommendations program had guidelines that were written so that when raters were looking at videos, the algorithm would know what to do concerning video’s and the content of those videos, whether they were helpful, if they should be flagged, and why they didn’t meet any quality expectation. Facebook ads also have an algorithm which is programmed by the engineers from the resulting data collected from raters. Instagram is no different. This company needs real people to rate and check ads so that its algorithm can be programmed. The videos were judged on how relevant and interesting they were, and why the videos were interesting or not. It is the information that was given on what made a good video good, or a bad video bad that was the most important data needed.

This becomes valuable real world data in which future guidelines and algorithms can be programmed. But once again, when the guidelines will change, the data needed will also change.

This is the main reason why you see some projects come and go, and why some are continual projects. Once one project has enough data, the algorithm is programmed and data collection stops until more data is needed or they make some changes in any general guidelines. Anytime changes are made in guidelines, raters or data collectors are once again needed to evaluate the current content being put online by advertisers, website builders, or even people who run Facebook pages and ads.

Some other types of projects that include the collection of data are voice recognition applications where you get a project that asks you to record your voice saying certain commands. Once enough voice data is collected, the product is programmed and tested. Some of these projects can be very interesting and I’ve seen a lot of them being offered by Appen Global as part of the talent pool network.

All these companies have one thing in common. They seek to improve their user experience. They can only improve the experience by having real world testers or raters look at the actual content, and rate the content based on the carefully crafted guidelines. This data definitely improves services and ads for the best user experience possible. Google, Facebook, and Instagram know that it would be impossible to create this experience without its social media and website raters.

How does Google use human raters in web search?