Bing Search Engine Rating Project (Side by Side Judge, Multimedia Judge) vs Google projects (Yukon/Internet Safety/Internet Assessor)

When getting into Search Engine Rating Projects, you will find that one company will produce ratings for one search engine, such as Google, while another company will offer jobs as a rater in Bing which is Microsoft’s Search Engine. Like Google, Microsoft also needs research and pages rated just like Google.

As far as the job is concerned, it is basically the same job except you are rating results in different search engines that turn up different results. The pay generally varies depending what company you work for with some reporting earnings in the $12 to $15 dollar per hour range.

Google projects (Yukon/Internet Safety/Internet Assessor)

Some of these Search Engine Rating Projects are changing and they are being called different things such as ‘internet analyst’ which puts a twist on the tasks you are given. Previously when working as a search engine rater, you were given results that were based on generic search results. However, as time went by, Google started asking raters for permission to use their personal data. Basically, if you were surfing for stereo’s or smartphones, and you started doing this job, you’d get back results based on your surfing habits as linked to your Gmail address that you use.

The job that Raterlabs offers is a Google Search Engine and Google Playstore project where the raters are now called ‘Internet Analyst’ where the tasks you get are based solely on your personal Google use. It’s a part-time position that currently looks to be for the USA region only, but this does change from time to time. The requirements and skills to get this job are nothing that most people don’t already have if they are good at searching or surfing the internet. With this job they want raters to possess a computer with high speed internet, and also an Android phone with version 4.1 or higher, or an iPhone version 4 or higher. The job most likely involves doing ratings on both the computer and the smartphone. Sometimes tasks are available on only one device or the other. So it’s good to have them both because you will likely get smartphone tasks. It is also noted that in the description that they want raters who use Google Play often. There is a possibility that tasks involving Google Play will be available. This job’s testing and qualification is no different from the previous exams for search engine related jobs. There are three parts to the exam leading us to believe it’s the same exact material for the regular rater. The job is basically the same after all, the only difference is now its personalized to you. It is not known what the pay is for this job as that information is confidential and you sign a non-disclosure agreement about not discussing your pay. But it’s likely in the $12 dollar per hour range if its Raterlabs.

Appen Global also offers Search Engine Rating Projects even though they also own Leapforce and likely are using them as a source of experience and also taking advantage of the workers who were already employed by Leapforce along with their experience. Appen calls their position Web Search Evaluator which likely has the same test. It is unknown if Appen supplies raters for both Google and Bing but it’s possible that they do both. All the requirements and testing are the same for Appen since they are providing raters for Googles Search Engine. But it is rumored that the pay for this position is much lower than you would get if you worked at Lionbridge or Raterlabs. This is because Appen is a much larger company with a much larger talent pool to take people from. This allows them to pay much lower wages. So in this case it’s actually better to get these types of jobs at other companies like Leapforce/Raterlabs, and Lionbridge. It’s interesting to hear that Appen is doing some interviews from Skype now.

Speaking of Lionbridge, we are getting to them next. This company seems to have a good reputation for paying people better than the other companies for the same type of Search Engine Rating Projects. They also offer opportunities for the Search Engine Rater but they likely offer rating jobs for both Google and Bing. Microsoft does use Lionbridge to hire human search evaluators, but Microsoft calls you ‘judges’ instead of raters. Lionbridge was known to pay raters $14 an hour for these positions although that wage has probably fluctuated over time.

Miscrosoft’s HRS project (Multimedia Judge, Side by Side Judge)

You’re probably wondering what the difference is in rating for Google or the Bing Search Engine Rating Projects. Bing has a project called Human Relevance System or HRS and it’s the system that they use to have human raters judge search results with. The HRS project is similar to the Quality Rater program that Google employs. However, it’s interesting to know that Microsoft’s version has been around since 2004 since MSN Search began generating its own results.

Not much has been written about Miscrosoft’s HRS project, but leaked copies of the guidelines are floating around that go over the process of rating or ‘judging’ websites. The difference between Googles rating guidelines and the HRS is that HRS uses what is called a rating matrix. This matrix combines the searcher intent with how well the returned document satisfies the searchers intent.

You can see the five rating options that judges can use from perfect to bad. This is the absolute heart of how you would rate returned search results. This might actually be a little easier to understand for some raters by judging just the intent and how well the intent is satisfied.

But there is one other aspect of the HRS system and the guidelines go over it in a pretty detailed manner. This is called Freshness and basically you will look at the content to see how up to date it is. Some information can be relevant but be a couple years old when some things have changed and the information isn’t updated or fresh enough to contain those changes. The Freshness system is based on three categories; Fresh Not Important, Very Likely Fresh, and Most Likely Fresh. The Fresh Not Important will be for things where the facts will never change no matter how fresh the article or the page is, while the other two will be based on the articles ability to receive fresh content.

When it comes to the difference between Google and Bing Search Engine Rating Projects as far as their rating systems, they aren’t really all that different from each other. But if you will be working as a search engine rater, and you are looking for stable tasks, you might want to try and get Google search rating jobs since there is more volume when it comes to this search engine over Bing. Still, both jobs are decent, but it really depends on who you are employed by. Appen seems to be one of those companies that will pay people as little money as possible to make more money themselves. This was their whole point of buying Leaforce; to squeeze out the competition. So this company will pay less. Lionbridge or Raterlabs will be the places to try and get these jobs.

When it comes down to Bing, it’s not a very big project in terms of the number of raters who are hired for this as compared to the raters that are hired for Google. That shouldn’t be a big surprise because we all know that Google has a much higher search volume and a lot more users. The judging platform for HRS is a bit outdated, not convenient to use, and it only works on Internet Explorer of course. Also, the test has a part in where you have to judge the search results for relevance of around 100 items and they give you very little time to do this. Many candidates often fail the test and are unable to pass the exam.

iSoftstone is another vendor offering Search Engine Rating Projects and is regarded as the main vendor for Microsoft Bing. They also have the 3 test qualification and that test seems to be the same no matter who you apply for. The first test goes over the guidelines, the second test is an English Grammar test, and the third exam tests your computer and internet skills. All work is done in the HRS platform and you have to complete a qualification for each task. There is a guideline for each one of the tasks, but you don’t need to worry because everything is explained in videos and emails. Tasks are categorized as hits or pages and for each hit or webpage that is shown you get 0.01 penny to 0.20 cents US. If you do tasks/hits quickly, then you can make easily $10-$15 dollars US per hour, and some people even more. You might be tempted to go through some of the tasks as quickly as possible because the more you complete, the more money you obviously make. However, sometimes during these tasks you are sometimes tested so you have to be very careful. Too many mistakes and they will remove you from the project and you won’t receive any more hits. ISoftstone is a bit easier to work for in that they don’t have a huge amount of work compared to Appen/Raterlabs.

Basically, what this whole thing boils down to is the individual. Some people might like working for Bing because the workload is lower, but you’re going to make less money because of this. If you need more stable work and higher than average work, then working on a project that involves Google will probably be a better alternative.

Do you work for any of these companies on any Search Engine Rating Projects? We would like it if you shared your experience here in the comments. And as always, if you have any questions, let me know by email or in the comments and I will answer them as soon as possible.