Arrow & Lionbridge Ads Evaluator projects workflow
Want to know what the common issues new raters face? After you have passed either the Lionbridge Ads or the Arrow ads exams, you will start getting access to different tasks. There are some things to keep in mind here. You should understand that if you are in a different market that the USA market, the training tasks are going to be in English. And as a side note, it’s going to be a good idea to have someone that is fluent in English ready in case you don’t understand something. Once you have passed all the initial training tasks, you will start to see tasks appear in the language, or region, that you applied for. There is a high chance that you will still see some English tasks come through. In fact, 70% of all tasks will be in English language.
RECORDING AND INVOICING WORKING TIME Appen Arrow and Lionbridge Ads Assessor project
Time management is one of the most important things you need to keep track of when you are doing your tasks. It’s recommended to track your time manually including the amount of tasks that you have completed. There are some free apps out there you can download for your computer, or you can use your timer app on any smartphone. I keep a small notepad and a pen handy and just cross down a line on paper for each task. Depending on how organized you are, it might take you some time to get used to this. Times sheets are reviewed every weeks starting on Monday and ending Sunday. Make sure you are entering the actual time it takes you from getting the task, to submitting it.
One of the tough things that a rater has when they first start is getting the quality up high and improving the quality of your rated tasks and keeping them there. That is likely because of the overwhelming amount of information to swallow at the beginning. The guidelines that you get need to become your bible so to speak. You should have them open always or easily accessible when you are rating so you can check some of the examples if you are confused about something. If you have some extra time to study the guide while not rating, then it’s a good idea to do this before rating. Don’t spend too much time going over the guide while rating. You are only allotted a certain amount of time for each task. So you can’t spend 15-20 minutes on one task and then bill for that if the task should only take you 5 minutes. Bookmark the examples in the guide for quick reference. Your tasks are reviewed and you will get some feedback concerning your rated tasks and where you can improve. You need to be prepared at all times so that when your quality is personally examined, you get good scores. All of this is very important because the client has a strict process that must be followed. By not following the process, you won’t supply the valuable data that is needed. Read the guidelines even after you passed the exam, and read them again and always refer to them when rating. Even the most experienced raters refer to the guide for help.
Appen Arrow blind test and Lionbridge Ads Assessor quality review tasks
Blind sets as they are called, are rating tasks that have the correct rating answers already correctly completed as they should be as per the guide. You will have no idea when these blinds sets come, or be able to identify them, but generally the blind quality tasks are uploaded to the system throughout the month, and you are graded on these to see if you are following the guidelines correctly, and applying the correct ratings. This is why it’s important to follow the guidelines at all times. In fact, it might be even better to not think about the blind sets at all since you will have no idea when you are doing them. We just thought it would be important for you to know that they are there. In total during a month working cycle there are will be about 150 blind test tasks of different types of tasks, raters are expected to obtain at least 70% of correct answers in the pool of blind test task, raters with lower quality score will be removed from the program.
Reasons for losing your service as an Ads Quality Rater
It can happen to anyone at any time. You could end up getting a short email from the company you work for at any time saying that your contract will not be renewed or that your service is no longer required on the project. We hope this doesn’t happen to you, but if it does it could be from some of the following issues.
Quality of service is really the big one here as the client needs each rater to strictly follow the guidelines and the processes of doing each rating. There are levels of quality and you should be sure to try and get the highest quality rating possible. Also its important to remember that just because you are at an acceptable level doesn’t guarantee job security. If there are a lot of raters for a project and a group of raters are providing much higher quality, then the ones at the bottom are likely to go first in the event of a downsizing or, the client just needs the best quality possible. It is advisable to keep striving for perfection when it comes to rating.
Participation in the program means that you must put in a required amount of hours or ‘minimum hours required’ which for most of these programs is 19-20 hours per week. Some programs require you to work at least one weekend day as well. If you miss a day or two on some projects, it will be OK but be careful because if you don’t participate on a regular basis then you end up losing access and are removed from the system. Regular participation in the projects also guarantees that you see a larger portion of different types of ads, and will certainly help you build up your quality levels.
IP address conflict/Internet Network sharing
In some cases people may share their Internet network, while working as search engine evaluator make sure that you have your personal unique IP address and no one uses the same IP to go online for rating project.Once you are in the Project, you may change your IP address many times, it will not affect your project status. Make sure that you do your project tasks in a private place as it stated in the contract.
Multi-accounts
Be informed that it is prohibited to have multiple accounts, without special arrangement. The customer can find duplicate accounts very fast (even if you are in different Google projects which administrated by different vendors), and as a result, you will lose both accounts.
No tasks are available, all task options are not available anymore except the QR code
This is a sign that system automatically cut your account from receiving tasks. Unfortunately this is an indication that your account has been deactivated due to unacceptable quality, such raters will be removed from the program shortly.
Lionbridge (Ads rating project) vs Appen (Arrow)
Productivity
In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. In this respect, Lionbridge is very loyal, and most invoices are approved without any issues. Of course, raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.
Appen has it is own internal productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters. In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RGP metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen(Leapforce) has again invented their own metrics to pay less to its raters.
Working hours and flexibility
At Lionbridge, there is no minimum requirement for working hours. However, the maximum normally is 20 hours and in some cases, the max might be increased.
At Appen there is a strict requirement to work a required number of hours as indicated in the consulting agreement (15-20 hours), not meeting this requirement may lead to removal from the project.
Payments
Lionbridge normally pays via direct deposit for US raters or wire transfer for raters in other countries. Non-US raters will receive their first payment only on the second month of work this due to their specific payment cycle set up.
Appen(Leapfroce) pays on a monthly basis via the Payoneer payment system, normally on 14th of every month.
Payoneer As a Payment Method for Appen, Raterlabs, Lionbridge and Utest – How to Get Rewarded with 25$?
Payments rates
In most markets, the Lionbridge payment hourly rate is 15-30% higher than what Appen pays for the same work. The customer sets the basic pay rate which is the same for all vendors (Lionbridge, Appen), and part of this rate goes to the vendor for project administration services and the other part is paid to raters. As some of you may know this project was previously administrated by Zerochaos and at that time the payment rate was 2 times higher than it is now. After Google outsourced this project to Appen and Lionbridge the payment rate was significantly reduced including US market.
More info about payment rates is available here:
Projects and Vendors Compatibility: How to Work on Several Projects and Do Not Violate Vendors Conditions? How Much Can You Earn With Lionbridge and Appen When Combining Several Projects?
General Support
The project itself is fully automated and was designed in a way that raters should manage their training and education, and work with minimum support. Those who fail are automatically removed from the project due to low quality etc. In general terms Appen provides better support by means of a monthly practice test and detailed feedback on those practice tasks, Lionbridge does not provide this type of support.
Optional Agreements
It might be beneficial to know that there is an optional agreement that you can sign that gives the vendor your personal browsing history that personalized tasks are generated by. Basically what that means is that you would get tasks to rate that would be based on your own personal browsing habits. A lot of people don’t want to do this for fear of sharing personal data with an employer. However, your data still remains private and the vendor will not see what websites you are visiting. It does add a little more personalization to your tasks though. This history is associated with the Google Account that you are signed into at the time of browsing, like if you were logged into Chrome and browsing the internet. It’s important to understand that Google has access to this data anyway, but it’s an optional thing whether you want your own personal history used for generating tasks.
The Final Word
If you can get in on this project, it can be very interesting while you are contributing to search engine or ad algorithms. Your direct contributions help to train specially written algorithms that help ads and search engine results become more relevant to the user. So it’s a very important job, and one that you should take seriously. However, this job is not for everyone. Unless you find looking at ads and webpages 4 hours a day, you might get bored very quickly. But for some this is the perfect part-time job because as long as you meet your productivity, deliver high accuracy and quality, then you could be on the project for as long as you want. As with all other Appen projects, you may even be invited to work on other projects too.
Appen Arrow invoice correction required due to low RHP
what is mystical RHP and is this practice legal?
In this project, most of the tasks do not have any recommended time limits. Raters have to manage their own productivity and their own time. The customer does not indicate average estimated time because the priority is quality in rating the tasks rather than productivity, or how many tasks you rate. Raters have to keep in mind that the productivity and time invoiced should be within reasonable metrics. It is OK to spend more time in the beginning and increase your productivity when becoming a more experienced rater.
Appen has productivity metrics called RPH. It is assumed these metrics are based on the average raters’ performance in the project which is calculated as the ratio between the number of tasks and the time spent within an invoice period completed by all raters.
In my view this metric is not useful, because in many cases raters get different types of tasks which require a different amount of time to complete, therefore the RPH metric is not accurate at all. This can lead to a situation where if your productivity doesn’t fall within their RPH metrics, your invoice will be returned for correction and they will ask you to align your RHP to average even though you spent a reasonable amount of time for completing more time-consuming tasks. The customer (Google) understands this issue. That is why they do not indicate the average estimated time. But it would seem that Appen invented their own metrics to pay less to its raters and make more money by invoicing actual time spent on doing task by all raters to Google. Good business, in fact. In this respect Lionbridge plays a fair game and most of raters get their time sheets approved without any issue.
The example of the email is bellow:
*************************************************************************************************
Dear Search Engine Evaluator,
Our records show a significant discrepancy between the hours you’ve written in your invoice and the amount of work completed. You over-billed more than xx hours for the week of x/x/201X – X/X/201X.
Please check our Chat feature, review our project information page and Intl FAQ.
Do not write your missed hours in your invoice time card as the entry will be considered billable.
In the meantime, these hours will not be approved until clarification has been received.
Please let us know if there’s ever an issue outside your control that is impacting your overall productivity such as tool errors.
The productivity rate/ rate per hour (RPH) is calculated by looking at the average of team who worked on a particular task. When you have a low RPH it negatively impacts the RPH average of the entire team and project. It is our expectation that raters meet the average rate per hour on a consistent basis. Listed below are some tips to try if you have not done so already, to hopefully push your rate per hour to the average:
• Time Management: Refrain from multi-tasking while working on EWOQ. Watching videos or listening to music not associated to a task may impact your focus and performance.
• Timing Devices:
In order to get a better feel for how fast or slow you are working, you might consider using a desktop timer, Toggl, or stopwatch. Always exit EWOQ when taking a break, no matter how short. Do not count the time spent on breaks towards your billable hours.
*************************************************************************************************
A company can’t just suddenly take away your pay when you put in the hours and did the required tasks to the best of your ability and as per your contract. There are some laws that protect freelancers and independent contractors, and if you are doing the work, then you should be paid. But for freelancers and independent contractors, your options are limited depending on what country you live in.
What raters can do in this situation?
-
-
- Before making any invoice correction, ask Appen for detailed clarification about RHP: what is the RPH ratio for all raters; what is your own RHP ratio, what is recommended RPH ratio, what is average recommended time for completion each type of tasks, ask them to provide list of average estimated time for completion for each type of tasks, ask why customer does not indicate recommended time for completion for many types of tasks. Be ready that they will not provide any info, because there is no official RHP approved by customer (Google).
- Point out that fact there is neither any policy regarding RHP can be found on Appen connect project platform nor customer (Google) does not provide any recommended time for most of tasks.
- Notify Google about the issue by sending email to the ads rating project admin (Google Ads Eval: ads_eval@google.com) to make them aware that this is some kind of issue that really needs to be investigated. The message should be the following: actual time spent on doing live tasks was not approved due to RHP, what is the RHP ratio for all raters; what is your own RHP ratio, what is recommended RHP ratio, what is average recommended time for completion each type of tasks, ask them to provide list of average estimated time for completion for each type of tasks, ask why Google does not indicate recommended time for completion for many types of task. FINALLY ask them – is it legal that Appen cuts hours based on mystical RHP which is not trasparant?
-
Perhaps if the right people receive a large response on the issue, someone will take a look and fix the problem. But as of right now things aren’t looking too good. For those of you who have lost wages, hang in there and hopefully, someone will get this straightened out.
If you have questions or comments then leave them below.
Hello Tim
After two months into working with LB, my account has been blocked and I got an email explains that my quality has been poor(around sixty). So I went on to check my feedback in the portal. There are 6 tasks that were reviewed and given feedback to.
However, about half of the feedback are wrong. I think the auditors failed to check accuracy of the news, clicking further into the links. So I sent an email explaining the possible mistakes of them and indicate which tasks that I can agree with. After one week that my account has been blocked, I got another email instructs me to complete additional learning path but nothing related to my questions.
Is this audit final and not disputable? Should I continuing asking for explanation or just accept this?
What if this quality assessment stands, how much more errors can I make?
NB. I have been always met the required hours, but I missed about 1-2 hours from maximum the week before the blockage.
Please help. Thanks you.
Hello,
I think you posted your message under wrong article/project? Are you talking about Yukon/Google Search Results rating project? I assume you are talking about the search engine evaluation project…You will find info related to your question in this post:https://passleapforceexam.com/2016/01/10/common-mistakesof-new-raters/
I have been working on ads rating project for a year, never had a task shortage before but since last and I’m facing a task shortage Tuesday. However I still got the 3 options marked on my EOWQ. Is anyone experiencing the same issue with shortage of tasks? I am from USA.
Hello,
As far as I know all markets are experiencing the same issue (shortage of tasks in Ads rating/Aztec projects).
I have an assumption that it has to do with updating raterhub tool. Vendors do not provide any info regarding this issue, the only response I got from them is to check raterhub for tasks on a regular basis. I hope the volume of tasks will increase in the near future.
Hello,
I got such email from Appen:
Our records show a significant discrepancy between the hours you’ve submitted and estimated hours based on RPH as shown below.
Week
Estimated Hours Based on RPH
Total Rater Hours Submitted
04/02 – 04/08
19
29
When you have a low RPH it negatively impacts the RPH average of the entire roster and project. It is the project requirement raters meet the average rate per hour on a consistent basis. The average rate per hour varies according to each task and should be made available to you in the judging interface or guidelines. In the meantime, these hours will not be approved until clarification has been received. Please amend your hours if needed.
*************************************************************************************************************************
Why do they send this email and how do they calculate RHP? I had lost of time consuming tasks (like video evaluation) and I am sure that I entered correct and reasonable time…moreover there is no any indication for the recommended time for many tasks…Do you think Appen cuts my hours to pay me less on purpose?
Hey Tim,
I have a question regarding the Lionbridge payment.
“Non-US raters will receive their first payment only on the second month”. What about the next payments after the first one ? Will they be again after 60 days or switched to 30? Thanks.
Hello,
After the first payment, the following up payment will be paid on a monthly basis, you may find payment schedule on your project portal.
Worked for Lionbridge for almost 5 months. Never was paid a dime. Finally resigned. The last month they said I was not in line with the hours/minutes for each project. I removed some time although I kept diligent records of time spent on each task. Was not paid again so that was it for me.
Hello,
Lionbridge pays all approved timesheets, could you explain your issue in more details? What project did you work in and what was the reason your timesheet was not approved?
Hi Tim! Thanks for the website: your work is a lifesaver!
I recently completed the LF Yukon evaluation and when the exam due date came I received no email or any contact from them.
At around the same time as the exam was due (same day, some minutes after), I got an email from Appen inviting me to join/qualify for Aztec. I know LF belongs to Appen now. Was this invitation their way of letting me know I had failed the exam? I had applied for positions at Appen so long ago, I had forgotten about it.
Normally having two job opportunities on the table wouldn’t bother me at all but you wrote Yukon and Aztec are incompatible and that is the source of my worries.
I have information that some of candidates have not received exam results on time I assume this is due to transition of the qualification process from Leapforce administration to Appen. I recommend you to write to agent support and ask for an exam retake.
Hi Tim! Thanks for the reply.
I got the results later and.. I passed! Unfortunately I got an email stating they had no use for me for the time being, so there’s that. I suppose they don’t call you after sending that, do they?
Any idea why I might have received that email?
Cheers
Hello, try to apply for the the same or similar role at anther vendor (Lionbridge).
Hello,
Today I have got an email from Lionbridge where they ask me to cut 40 hrs from my July timesheet/invoice. I am really confused about this request as I always keep track of time I spend on doing live tasks in Ads Rating project, my average monthly productivity is 2 minutes. Does anyone experience the same issue? I also noticed that the number of tasks I completed each week in July does not correspond to the number of tasks I entered into the timesheet, is it a system glitch? I remember the had the same issue in may that time they sent out emails that they sent out incorrect data, and now the same issue???
Hi everyone here
I have the same issue again and this does not make me happy at all.
I am still having problems with my timesheet and this is getting bigger and more worrying.
Since the May timesheet incident, I’ve been very careful logging time; I log time for ‘live’ tasks only, I disregard time spent on reading task guidelines, I make sure not to let any tasks expire, etc. Yet I am still getting emails from Lionbridge – regularly – warning me that I have overstated my work time.
Fyi, I had to remove 6.5 hours from my May timesheet, 12 hours from my June timesheet. Few days ago, I was told I had overstated by 20 hours for the first week of this month (I was given permission to work up to 30 hours due to large volume of tasks). And the list goes on..
There is no explanation from Lionbridge as to why this is happening and it’s driving me crazy.
Do you have any idea what’s going on? Do you think perhaps I’m spending too much time on each task?
If I can’t solve this problem, I might quit and apply to Appen..its too stressful.
Thanks
Hello,
Thank you for your comments. I am informed about this issue with massive disapproval of time sheets in Ads rating project as I am getting lots of personal emails from raters from different markets… What is in common is the fact that the correction data they send (weekly number of tasks/hours) do not correspond with weekly number of tasks/hours entered by raters into their timesheet. Based on this information I may assume that they have a system glitch and my recommendation is to compare your timesheet data vs their records for discrepancy (in particular the number of tasks completed weekly vs. lionbridge records). Once you do that, write email to Lionbridge about the issue. I think if they get lots of emails about the issue they will finally review all their records and fix their stupid system.
THEY ARE REALLY BECOMING SCAMMERS. I tried to make a post before, but it was deleted. Here is my message:
I’m a web search evaluator from Lionbridge and in the past 2 months I’ve been receiving emails from the homeworker supporter team asking for us raters to update the timesheet accordingly the time registered in their system and remove the hours as stated. The problem here is that, most of us raters have contact with each other and we know exactly the time we spent rating tasks and we all are receiving the same email asking to uptade the timesheet. Most of us are honest about the time we spend and record rating live tasks and it’s pretty IMPOSSIBLE for us to have ENORMOUS amounts of discrepancies in the timesheet. We worked an enormous amount of times and they are asking us to remove almost half of the hours we worked throughout the month. We don’t even record the time we spend reading the guidelines or tests, we register the time we spend rating live tasks only. I’m almost sure they are being dishonest this time. Emailing them won’t resolve, they will always email tag back never admitting they are wrong. Why did they delete the timer and rating history to keep track of our tasks? They’ve been doing the same thing to so many people lately and the only explanation I can find is that they are trying to deceive or steal us, or maybe their systems are glitched or bugged, but they don’t want to fix it in order to profit on our backs.
I assume there is a glitch in the system (raterhub), what we can do in this situation to write to Lionbridge about the issue and submit the issue report/complain directly to Google rating team via raterhub platform->report a problem option
I’ve been working with Lionbridge for almost 1 year and the same thing happened to me, Lionbridge has asked me to take 68 hours from 06/25/2018 to 08/05/2018 and I’m outraged! I sent 3 emails to them and they didn´t respond to me until now. I don´t know how to proceed. I found this another link where several people are complaining about the same issue in comments section, check it out:
https://www.reddit.com/r/WorkOnline/comments/9b0n2n/lionbridge_scammers/
I think that we need to discuss in both websites and take an action together…
Hi guy I sent email directly to the project admin team at Google via raterhub, I hope they will take actions on the issue.
Hello,
I would like to infom you about the issue related to the invoicing process at Lionbridge, it looks like that they have a system glitch in terms of recording time/the number of tasks completed by raters. In my case they asked me to cut hours and number of tasks completed, thier data is wrong, because the number of tasks and hours donot even closely correspond to actual number of tasks and hours I completed and entereted into the timesheet.
Based on the thread on Reddit I assume this issue is massive and take place across all markets.https://www.reddit.com/r/WorkOnline/comments/9b0n2n/lionbridge_scammers/
Please take actions on this unprofessional approch which undermine project reputation.
I think this issue started from May when they had a issue with gmail logon on ratehub, then many raters reported that they could not login the platform and they could see someone’s gmail on their page, I assume that this cased the issue in SQL database and as a result the system records a wrong rater time/the number of tasks, and it can be clearly seen from the fact of huge discrepancies…I wander why Lionbridge still cannot/or does not want to see this problem. I recommend to connect the project admin team directly, i have their gmail from the old version of raterhub:ads_eval@google.com
I’ve emailed EWOQ now [ads_eval@google.com], for two reasons: because of this timesheet issue, and because I’m not getting into EWOQ to work, and an automated message has been directed to my email where they say we should not contact them in case of issues in the timesheet, as I show below in the excerpt of this email:
Contact your contract administrator when you need an individualized response, or a specific action taken on your account.
As a contract worker, your contract administrator manages and is responsible for all work, timesheet, and billing issues. Contact them if you have questions about:
timesheets
billing
time off/leaves of absence
any other contract-related question.
So, I believe that our only option be connect the project admin team directly through this email:
The reason I recommend to send email directly to project admin is to make them aware of the massive issue.
Update on the issue: many raters who indicated discrepancies in Lionbridge emails with request to cut hours, got all their invoices approved! Lionbridge acknowledged that the issue was on their side.
Hello Dear Allen,
I failed the Appen Aztec Ads Assessment Project Task, something got wrong and trouble about QR code etc. and appen regret me to inform this project.
Do you know that Appen give a second chance me for retaking exam?
sincerely
Hello,
First of all, I would like to congratulate and thank you for your work with tips and information on projects and crowdsourcing companies; His training is good, but in practice, a lot of things go dark. So I would very much like you to help me to clear some doubts.
1 – Recently, aiming to level up I started to do the tasks of the simulator. At first I was not worried about the number of errors – and I was wrong because the simulators and quizzes are based on situations, places, and American research, which makes it difficult to understand who is from another country – And I noticed that parallel to this I started to receive feedbacks, in the beginning within expectations, but then I was below expectations.
The question is: Does the quality review and feedback they send us by email are based on simulators or the real tasks of the Rather Hub ??
1.1 – Can I not do the tasks of the simulators to avoid being poorly evaluated?
2 – Are Invoices divided into 30 days of work or month?
2.1 – The conversion of the currency is made by the quotation of which day?
3 – How is the calculation of the number of hours that Rater works? Is the total number of hours worked and divided by the number of days added, or is the number of exact hours worked in the day observed?
4 – Should I note the time worked manually or is there an allowed app that facilitates this count?
Thank you very much
Hello,
Answers to your question you will find here:https://passleapforceexam.com/2016/01/10/common-mistakesof-new-raters/
New raters do live tasks immediately after getting access to the task portal. It is highly advised that you hold off on these if you do not want to be restricted for quality in the next month. Typically tasks which are available for new raters are quality tasks on which raters are accessed for their quality. Instead, new agents should focus on doing simulation tasks and self-education. Time spent doing this will really pay off, and you wont be restricted for quality in the future. Also, by doing a certain number of simulation tasks correctly you will upgrade your status in the system, and will be allowed to work more hours.
During your participation in the project your service will be evaluated on quality by means of quality review tasks.In the beginning of each month, a certain number of tasks are uploaded into the system. These tasks are quality performance tasks up on which your quality is measured each month,in the end of the month you will get your quality score feedback.
If your ratings often falls out of general range of ratings provided by another raters in the project, most likely your account will be flagged by the system automatically and your work will be checked for quality manually.
Hi Tim,
Thanks for your blog, we are very helpful to those of us who work on these projects. I am working for Appen in the Aztec Ads project and it has only been a month and they are already telling me that I do not reach the RPH average, I have asked them to guide me regarding this metric indicating an approximate RPH and they do not tell me, they are limited to To say that when I get experience I will improve my RPH. I have been told that they will pay me all the time, but I am afraid that they will give me a leave of absence from the project. Could this be grounds for dismissal?
Thanks for your help,
Hello,
RPH is Appen metric the aim of which is to pay less to its raters and to take the difference as profit (they invoice Google actual time worked by rater), additionally I have information from Google project administrator regarding this metric, in fact there is no official time metric for most of tasks, but in general the average time spent on each tasks should be about 2 minutes.
I confirm that those raters who do not meet internal Appen RPH will be removed from the project. That is the reason why I recommend to apply for Ads Rating project at Lionbridge and avoid Appen.
Hi Tim,
I have a question. When you say, time metric for most of tasks are about 2mins, do you see Ad Creative + Landing Page evaluation as Two independant two tasks or one task consist of two pages? Clarification is greatly appreciated, Thank you!
Hello,
For this type of tasks (Search Ads Evalution), customer has an average estimated time which is 2 minutes to evaluate both Ad Creative and Landing Page.
Has anyone been experiencing the “No task available” for the past week in december? I work at Lionbridge
Hi
Thank you very much for your post.
– I think you answered LB or LF doesn’t pay for simulation tasks, but
what about Appen?
– Does Appen pay for quality tasks for new raters?
– All the webpages I rated on the qualification exam were written in
English. Will I rate webpages written in my local language?
– Are people who will review my quality tasks fluent both in English
and my local language?
Hello,
Training and doing simulation tasks are not paid.
Quality review tasks are normal tasks, you will not know if tasks is a quality review tasks for this reason they are called blind test tasks.
Most of tasks will be in language of your country/locale, all instructions are in English.
Hi Tim,
Is it true that lionbridge beginners get limited tasks to do in the first month of working… even after completing the Quiz given upon entry?
I say this because in these first two days of working… I have had access to around 2 hours of tasks per day but was expecting more. There seems to a limit right around the 2 hour mark and then I am automatically given the “No tasks available message”.
Is this normal and did this happen in your own experience?
Thanks, Josh
In general, Appen and Lionbridge guarantee at least 2 hours of work per day but task availability may depend on project locale.
There might be some days when tasks are available for more than 4 hours. Some locales, like the US project provides raters with 24 hrs/7 days tasks availability.
For the rest market task availability may be limited. You should understand that the system of task distribution is based on a ‘first come first served’ basis and you need to be ready to catch your portion of tasks when they are uploaded into the system. In order to identify time slots when task availability is the highest please look at task completion graph, usually tasks are uploaded every day into the system at the same time slots.
Lionbridge does not provide raters with a task availability instrument, so raters have to identify time slots when tasks are uploaded into the system by themselves.
Appen also has a system that restricts access to tasks for new raters, and those who are undergoing their quality review. To lift this restriction, you have to complete a certain number of simulation tasks or wait till the next quality review, if your quality score is good, the restriction will be lifted automatically.
Official clarification from Lionbridge regarding your question: As per your welcome to the program email please note the following:
As a new Rater on the program you will initially have access to a variety of task types. This is important as it will help you become more familiar with the core concepts of rating and help build up your overall experience and knowledge of the program. The volume of these tasks will initially be low as you continue to build experience on the program. Once we feel that you have a good overall understanding of the program and of the various task types you will see a switch in the tasks that you receive. From this time onwards, you will be required to focus on evaluating more video type tasks from an internet safety perspective and you should also see an increase in the number of tasks available to you.
Following from this there may be times when you have no access to tasks. Please follow the onscreen instructions. Please check back at the date and time specified for available tasks.
Please note that this process is no connected to your quality score.
We would encourage you to continue to login regularly and complete the tasks available to you. Tasks availability will improve as you complete tasks and the program progresses.
We will contact you directly to advise when this transition process has been completed.
Hi Tims,
May I know the process in Ads Assessor? because in the mail and the EWOQ slightly different, in mail there are no RC Code scan exam, but inside the EWOQ there is RC Code scan exam,
Here my main question, after finishing all tasks listed in the mail, I try to Acquire the next task, but after I finish the survey about RC Code Scan exam the system said No More Task Available, and on top page there is notification to wait my contractor to give result. Does that mean I failed in the exam?
Thanks Yuki
After completing the exam, the following message will appear:No More Task Available, your contractor will get in touch with you soon.
You should wait for email with exam results info. Normally results are sent on exam due date +1-3 days.
Hello,
Hello,
After being assigned to a Client Final Assessment for the Arrow project and reading the necessary guidelines, I have started with my first task which is a User Interest Evaluation. I have been rating and submitting incidents for a while now however one thing has made me baffled and I couldn’t find any suggestions to it in the guidelines provided. Should I only rate the incident in front of me or is it always necessary to fill the comment section for each incident? I would like to have your advices on this. It has been 4 hours and coming up with a comment as to why I have given the rate for the each incident proves to be a bit tedious. Should I keep going like that or just use the comment section when I feel it is necessary?
Kind Regards,
Burak
In most cases no comments are required for all types of tasks in the exam.
Hi,
I know at Lionbridge they only pay you at the second month
But are you paid only 1 month or for the 2 months you worked ?
Hi,
I know that Lionridge pays you at the second month
But do they pay you for 1 month or the 2 months you worked ?
Hello,
Yes, all approved invoices will be paid, the first payment is done on the second month of work (payment for the first month of work). For detailed payment cycle info please refer to Lionbridge support portal.
Hello,
I would like to know once I’ve finished my daily 4 hrs on Lionbridge, what to click to exit from rater hub? Just Sign out? I’m afraid that if I press sign out, I’ll lost all my tasks.
And have you any advice to record my time and task? any good tool for PC?
Thank you in advance
Kind regards, Tina
Hello,
You do not need to sign out in order to stop your rating, instead you should return to the the main page of the raterhub.
Time is counted from when you click on the ‘Acquire a New task’ button to when you click on the ‘Submit’ button for each task.
Hello,
I would like to know once I’ve finished my daily 4hrs on LB, what to do to exit from rater hub? Just Sign Out? Cause I’m afraid that if I’ll press Sign Out button I’ll lost all my worked tasks. And have you any advice to count my time and task? Any good tool for PC?
Thank you in advance
Kind regards, Tina
Hello,
Could you specify your project?
I can’t access the exam it keeps saying unauthorized access please sign in but when I click sign in nothing happens. I tried searching rater hub support but got this.
happy to found this page, i want to ask about the task. did you experience the task in other language. i am work for ads quality program, the agreement with LB is task in native language + English, but in my first two day, i got Germany task. I ask to LB and Ads Eval, but no answer. since that email about my question, i got no new task till today (from monday). is the LB cut me off?
Hi,
Sometimes it happens that raters get tasks of another locale/language in particular this case is common with Youtube tasks.
If you are unable to rate such tasks, simply use appropriate flag (foreign language).
Hai Tim,
Thanks for your article.
I’ve been working as ads evaluation rater for 2 month.
But today i got this message in my EWOQ System:
“Task access suspended. Your employer will notify you of next steps”.
What the meaning of this message?
Did i got terminated?
I’m asking via email to Lionbridge but no response.
Thanks tim
Hello, most likely the system automatically cut access to tasks due to not meeting a required level of quality.
Hi Aidah,
Got the same message 2 days ago, got an email today. I need to re-qualify and take the test again.
Good luck
A few raters reported the same case, when the vendor asked disqualified due to low quality raters to go through re-qualification. This is in fact new procedure and might be used only in markets where they can’t hire enough raters.
Try to ask your vendor for a chance to re-enter or re-qualify for the project.
Hello Tom,
Thank you for always useful information.
I worked on a Arrow project, but I was removed from the project in one shot due to unacceptable quality. Can I be re-certified to this Arrow project in the future? Or is it no longer possible?
Hello,
It is possible.
1. You may apply for the same role as ads assessor at Lionbridge.
2. You make new registration with Appen and cross fingers for arrow project to appear on your project list.
3. If you need support with blind test to make sure that your quality is score is always high, contact me directly via email or Facebook.