Leapforce and Lionbridge have a new unwelcome surprise for its contractors. In April, US raters received an e-mail from Leapforce, which states the following “Effective June 1, 2017, [Raters in the US] can work up to 26 hours in a calendar week (Sunday to Saturday).” All raters will be moved to a new platform called raterlab and will not be able to work on another projects at Leapforce, the same situation will happen with Lionbridge raters in USA.

The Phantoms of the Internet – Google Ad Raters

If you are visiting my website, then you probably have an idea of what a Google Rater is. But if you have no idea, this is your first time visiting, then grab a tea or a cold one, and lets take a journey behind the scenes, down the rabbit hole, choose the red pill, but understand that there is no turning back once you start. Many people think that Google is all about a robot running its secret recipe algorithms, or that the spiders (bots) that crawl webpages are the only things that are responsible for a web pages’ rankings. It isn’t true. And you might understand by now there is a human factor involved. Enter the Google Rater.

First of all, we need to understand that there weren’t always humans involved in the ranking of web pages. The changes came about when Googles advertising algorithms were no longer able to discern certain types of content, or mainly, keeping bad content out of places that it shouldn’t be. The purpose of the Google algorithm is responsible for placing ads in appropriate places or context, for example, someone searching for restaurants might want to see some ads for pizza, or sushi. Or someone searching for music would see some ads based on some famous artists. But something went wrong and in the UK, something disturbing happened that caused Google some big problems. Uk Government ads for the Home Office, Royal Navy, and Transport for London ended up being paired with Youtube videos that featured a certain Steven Anderson who is a Holocaust-denying pastor, and who actually praised and enthusiastically commended the guy that killed 49 people in the Florida nightclub Pulse. And that is not all, according to the UK Government, its taxpayer-sponsored ads ended up running alongside videos from so called ‘rape apologists’ and some speeches from David Duke the white supremacist.

So what happened? Well you guessed it. Google’s ad business suddenly lost a lot of business as big European ad agencies had to cut ties with Google, and to top it off, Verizon and AT&T cut all their video ad buys. When you lose a big chunk of money suddenly, you have got to do something to fix the situation, and fast. Google stepped up and assured its clients that it would make sure that no ads ran alongside with bad, upsetting, or offensive content. Google came out and said it was going to unleash an army of 10,000 humans to rate ads, that would work around the clock to make sure that their algorithms don’t return bad results, results that are unhelpful, offensive, or even horrific like the previously mentioned stuff.

So who are these raters behind the scenes? They are carefully trained and tested people that can spend 40 hours a week checking and rating tasks, and use a special website system called Raterhub, which is exclusively owned by Google. The tasks vary, and new tasks or ‘experimental’ tasks are available on a continued basis. Every day thousands of tasks are completed by the army of raters that produce invaluable data about how Googles always changing algorithms are performing. Raters produce significant results to several Google and Android projects from your regular search feature, to voice recognition, and photo tasks in Google/Android.

Not many people realize just how much the rater contribution adds to the smooth functioning machine of Google search. And no one knows who these people are. Not even Google engineers who take the resulting data have any idea. But it seems now that some raters want this to change. The main reason is that recently, an email went out to all raters telling them that their hours would be cut in half, and that is was partly due to Googles staffing policies.

And although Google brags about its army of raters, the raters aren’t even Google employees. Instead these raters are contracted out by different firms. Now raters believe that Google has reaped significant benefits from their rating results without ensuring their jobs are stable and secure. So who are these companies?

Leapforce

There are a handful of companies that supply raters for Googles Raterhub evaluation on a contractual basis. Leapforce, located in Pleasanton, California is one of the biggest. Their offices lie just outside Silicon Valley. But it doesn’t matter where their offices are located, because no raters work from the office. All raters work remotely from their home and there is a chat room in the backend where raters can ask each other questions and meetings are held in this chat room weekly. The meetings are unpaid, but they are optional and you don’t need to attend. Sometimes the meetings can help if you have questions about certain tasks that are seemingly hard to rate for whatever reasons. Managers never go by their names and also raters all have a nickname. Not even in Raterhub do other raters know who each other is. Ghosts…doing important tasks, hidden behind the scenes, and providing invaluable data while remaining anonymous even to each other. You can go years without knowing anyone at all, the real names of managers, or the people you work with.

All the work is task based somewhat like MTurk. To get a task you log into Raterhub and you will see usually 2-3 buttons that say Acquire if available and the name of the task which most now say experimental for some reason or another. Acquire if available means what it says. If there are a lot of raters working, you might not have any tasks to do. Some days there are plenty of tasks available and some days you might work 12 minutes and be given the message ‘Sorry there are no tasks available…. please check back later’. This can be frustrating, but if you figure out the times when tasks are loaded and are right there to get work, you can be OK. You would think that this job might seem like you could work any hours you want. But basically you are at mercy of the Raterhub system and its tasks availability.

Typical tasks can take anywhere from 30 seconds like the voice or speech recognition tasks, and others can take 15 minutes where a rater is looking at a webpage and judging the contents and relevancy. The time a rater can bill is really limited because the tasks have a time limit and in Raterhub there is a timer showing you how much time you are allowed, and how much time you are taking. You can take 5 minutes to do a 30 second task. If you try to bill for extra minutes compared to your tasks, then your invoice will likely get kicked back to you. But this isn’t an only problem, sometimes Raterhub can be really slow and can take a lot of time to load. If you have a 5-minute task, but it takes 2 minutes to load, you’ve only got 3 minutes to complete the task because you can only bill 5 minutes. Sometimes the loading of the task can eat up all your time. Raterhub can be slow and it’s a common occurrence.

what about the tasks themselves?

Tasks can vary, and even some new tasks will pop up with the term ‘Experimental’ which means they are new tasks rolled out by the engineers. The raters job is to evaluate if the search result is useful to the searcher or not. Audio recognition is becoming more prevalent on Android, and raters sometimes must check and listen to the audio to see if its been transcribed correctly. And according to some raters some tasks can feel a bit creepy. The tasks are related to personalization services, which requires raters to first give Google access to their emails, photos, chats, and even allow Google to record their own search behavior and other Google services that the rater uses. Google then turns the personal data into tasks that allow the rater to give them feedback on how well their algorithm is working. As you can imagine, this can and has created some discomfort giving Google access to their personal accounts.

A rater’s job isn’t easy. Before a rater can even think about starting work, each rater must pass a 3-part exam that is extremely difficult. You are given a 160 page guidelines that you need to more than study, you need to absorb these guidelines, you need to eat, drink, and sleep with them under your pillow. If you do not, you will fail the exams. In fact, the exams are so difficult to pass, that you are even told in the first email with the guidelines that the majority of people who take the exam fail it. And they stress over and over how important it is to know these guidelines. In fact, it’s a good idea to even copy and print them out because you will be going back to them often, and even if you pass the exam, you will want to reference them. So how hard is the exam to pass? One rater had said they referred nine people to the job, and every one of them failed the exam. It is worthwhile to mention that if you do pass the exam, they give you a second chance to pass it. There are three parts. After you pass each part starting with the first, you are sent an email saying you passed, and only then are given access to the second part. If you fail the exam, no matter which part, you must take the entire exam over again.

For those who pass the exam, the training isn’t over as one might expect since Google is ever changing in its algorithms. At the beginning of each month there are evaluation tasks that have to be taken. The right answers for these tasks are already recorded, so if you fail these tasks, you then become limited in what tasks are available until your score improves.

Sometimes a bot will spot check a raters work which can cause problems because it can automatically lock you out of the tasks area. Raters call this being ‘botted’ and the bot can be buggy and occasionally lock you out for no reason. Once this happens, its often difficult to get a hold of a management person to fix the problem which results in a loss of work because it can take days to get a reply.

Despite all these frustrations, raters generally like their jobs and the pay can reach up to $17.40 per hour for those who achieve excellent scores. Regular raters start out with and get $13.50 per hour which is still above the min wage for most parts of the USA at the current moment. The work isn’t fun by all means. But you do have the sense that you are doing something meaningful and helpful.

But now the big problem is there is no more full-time work. On April 3^rd thousands of US Leapforce raters received an email from ‘The Leapforce Team’ saying that effective June 1, 2017, each rater can only work 26 hours in a calendar week which is Sunday to Saturday. With that being said, 20 percent of the US raters just took a giant pay cut. Almost half their pay now gone for full time raters. Many raters were greeted by LFAdmin who for the first time told everyone that he was indeed the CEO Daren Jackson and said this was not a decision that they were in control of. It was Googles decision. He added the change was due to risk mitigation and some other regulations but didn’t specify. In the e-mail, however, the Leapforce Team claimed the change was driven by “circumstances that are somewhat out of our control, but will ensure Leapforce is compliant with federal and state regulations including the FLSA [Fair Labor Standards Act] and the ACA [Affordable Care Act].

Its interesting to note however that key provisions of FLSA and ACA don’t apply to contract workers, which is what Leapforce raters are. If raters are independent contractors, Leapforce has no ACA obligations. This makes the changes and loss of hours a confusing subject. A manager in the main rater chat room had told that the pay cut had come from Google themselves. Can they change this? It’s a chance less than zero when Google makes a decision based on this.

What really upset a lot of raters was that this sudden reduction in their hours came a couple of weeks after a Google engineer named Paul Haahr had raved in the media about how terrifically helpful raters are to the company. “We’ve only been able to improve ranking as much as we have over the years because we have this really strong rater program that gives us real feedback on what we’re doing”. So why would they cut the raters hours nearly in half if they were doing such a good job for the company?

One of the mysteries for raters at Leapforce now is, ‘who do I actually work for, Google? Or Leapforce’. Daren Jackson actually left Google and created Leapforce in 2008. Jackson had been working on the project EWOQ which was the precursor to Raterhub. Rumor has it that Google actually had a rater tool as early as 2004.

Google purchased Raterhub in 2012, and at that time, all raters were coming from companies such as Leapforce, Lionbridge, Appen, and ZeroChaos.

Google’s policies have a direct effect on Leapforce raters’ employment status, and its been known that Leapforce CEO has started to reclassify its US Raters from independent contractors to actual employees. It appears as though a change in Google contracts is behind the switch:

“They had a contract that was up for bid and it stipulated that they wanted to work with employee-based workforces. It wasn’t Google’s attorneys [who forced us to do this], but the process was they wanted vendors to work with an employee-based workforce. To continue with the contract, to work on that project, we’re doing [the employee] conversion.”

So is this a joint employment situation? The problem in the Leapforce situation is that the company isn’t depriving employees of benefits they already had, since the raters have been contract workers. But Google does provide its employees with benefits. If a court found that these raters were joint employees of Google, the search company might find itself in a tough spot.

Ghosts in the machine

Many of these raters actually believe they are working for Google and not Leapforce, and simply contracted by these 3^rd party companies. But raters at other staffing firms such as Appen, ZeroChaos, and others face similar problems. In fact, ZeroChaos has put is rater system on hold until all of this is worked out. An email went out on May 22^nd asking for patience as the program has been put on hold. It didn’t say why, it simply said it was on hold. And its absolutely certain that they are facing the same issues and triyng to figure out the best course on how to proceed.

The Leapforce raters’ current situation appears to be the result of Google insisting that its staffing agencies work with employees rather than independent contractors. Unfortunately, the change has had unintended consequences. In a month, Leapforce raters will be employees, but without full-time work and benefits.

This doesn’t please any of the raters, who had a range of opinions about what would be a good outcome to the situation. Many raters would like to join Google and work directly for the company, and who wouldn’t Google boasts and brags that it likes to take care of its employees, but if you are an independent contractor, you are basically a phantom or a ghost. An unknown force behind the curtain so to speak.

It will be an interesting thing to see what happens now. It is certain that the Google Rater is a very important part of its algorithms, and its not likely that a robot is going to take away the important aspect of a human making sure that things don’t go wrong, like your child accidently seeing some bad websites or ads that aren’t relevant to a search. No robot could likely take away this type of job. In any event, we will keep you updated on the situation.

Please comment below if you liked this story and share.