Get all the information you need to find and apply for these legitimate work at home jobs, and get all the resources you need for becoming a successful rater
If you are looking for a list of projects out there from companies like Lionbridge and Appen, we have them. As you may know Appen acquired Leapforce in 2017, since then the company has decided to close it is own project management platform known as Appen Global and move all projects to Appen connect. The group of projects known as Falcon (mainly Facebook-related projects) now is no longer actual, these projects also have been moved to Appen connect platform.
Taking into account the fact that all projects have code names and little description is provided, many candidates might be lost in terms what project to go for.
Bellow you will find a list of most popular projects and links to more info about each project:
Provide relevance and accuracy ratings for one of the world's largest and most popular mapping applications.
The job has an ABOVE average volume of work.
Maps Evaluation Project
Provide relevance and accuracy ratings for one of the world's largest and most popular mapping applications.
The job has an ABOVE average volume of work.
Apple iTunes Music Search Results Evaluation Project
Improving the relevance of search results for music-related content.
Raters must have access to an iOS device capable of running iOS 12.x. and higher. The job has an average volume of work.
Search Relevance task focused on improving the relevance of search results, based on the most likely intent of user searches. Raters must have access to an iOS device capable of running iOS 12.1.4. The job has an average volume of work.
Apple Store Search Relevance project focused on improving the relevance of search results based on user search intent.
Raters must have access to an iOS device capable of running iOS 12.1.4
Similar to Echo
Sharp
Apple Search Relevance task focused on improving the relevance of search results within different media categories.
Raters must have access to an iOS device capable of running iOS 12.1.4
Similar to Echo
Index
Apple App Store Search Results Evaluation Project
Search Relevance task focused on improving the relevance of search results based on user search intent.
Raters must have access to an iOS device capable of running iOS 12.1.4. The job has an average volume of work.
Similar to Echo
Slickrock
This project focuses on Facebook Stories and rating these personalized recommended stories.
Apple Safari Search Result Evaluation project. The goal of the project is to understand the user intention as represented by the query and determine how well or how completely the result satisfies the user's need.
Post categorization on Facebook. You have to decide what topic that post fits under. It is basically a categorizing system job which aims to put the posts that are made into the correct categories of broad topics.
Help improve ads quality for one of the world’s largest image sharing sites.
Required: An active Facebook and Instagram account – The Instagram account must be connected to your Facebook account. You also need a smartphone with Android or iOS. The job has an Above Average work volume.
There is a Facebook project called Tahoe that uses a team that checks events that come up in Facebook results. The main mission of this program is to bring people together through real-world activities by showing real events.
Facebook Falcon Projects. In this task, you will be asked to research and evaluate claims stated in various pieces of content (e.g., articles, links, photos, or videos). 20 hrs per week, work is done via Facebook rating tool.
The aim of this project is to help the client evaluate and improve the quality of the Ads that they present to their users.
Requirements are a personal computer running Windows 8 or better and a desire to look at different ads.
The aim of this project is to help the client evaluate and improve the quality of the Ads that they present to their users.
Requirements are a personal computer running Windows 8 or better and a desire to look at different ads.
Average volume of work.
No info
Tigris
The project consists of rating tasks focusing on improving the relevancy of advertisement results from a major social media platform. Consultants will rate advertisements to help the client understand which advertisements (also known as ads) people want to see, and why. The daily task of the consultant consists of rating a batch of ads in a client-owned rating tool.
FB project?
Carmel
Utterance annotation and evaluation.Microsoft Bing project aimed to learn search engine to recognize voice and human speech.
No info
*group of projects known as Falcon (mainly Facebook related projects) now is no longer actual, these projects also have been moved to Appen connect platform. Newly added projects such as Mohalwk, Prospect MI, Zeya, Copper and Alkali are most likely related to small Facebook testing projects. If you have been participating in one of the above listed projects, feel free to share your experience with us via leave a comment option!
Lionbridge
Lionbridge project code name
Description
Reference for more info
Internet Assessor, Personalized Internet Assessor, Rater
Looking to pass aPPEN Yukon project/lIONBRIDGE iNTERNET Assessor & Internet Safety Evaluator Exams? THEN YOU HAVE COME TO THE RIGHT PLACE FOR all THE INFORMATION THAT YOU NEED
(Updated Jan 2020)
Before you can start working on various tasks, you must pass the qualification exam. The qualification exam you will take covers two types of rating tasks which will make up 70% of all tasks in the project: Needs Met and Page Quality.
Once your candidature is approved for certification, you will get an exam email with detailed instructions. Within seven days you have to study 160 pages of the general guideline which consists of rating rules and concepts. The guide explains you how to do the rating of search results and assessment of quality of web pages. You should follow these rules and not use your personal judgment. What they want is for you to represent a typical user in your country. Before you proceed with the exam you need to study the guideline at least twice paying particular attention to practical task examples. Keep in mind that it’s 160 pages long. As I always advise, don’t wait till the last minute.
Note: At the stage of examination, Lionbridge provides candidates with very helpful exam preparation support. You will get access as an exam guest to the Lionbridge learning portal where you can listen to webinars and even try to do a few practical tasks before proceeding with actual exam.
Important update: Lionbridge is very selective in terms of recruiting new raters, up on revising exam results they will check your activity on their exam preparation portal, with this in mind, I strongly recommend you go through all the webinars on exam preparation portal before proceeding with the exam itself.
There are three parts in the exam: you can proceed with the next part only if you pass the previous one. Once you complete any part of exam, within 3-10 minutes you will receive an email with information about results.
Google updates the way the exam is done from time to time. The main feature that has been changed is that you must rate pages from your smartphone (send to device tasks). If you do not have smartphone, they say you cannot apply for this job. This is due to the fact that the number of mobile users is constantly growing and the company wants to assess websites from the perspective of on-the-go mobile users.
Note: There is a recommended schedule that will give you plenty of time so that you’re not rushing to complete the exam at the last minute:
Day 1 – Study the Guidelines – Make sure to setup your smartphone so that you are ready to take the final portion of the exam.
Day 2 – Start the Theory part of the exam making sure you are taking time to CAREFULLY read each question. Some of them can be tricky and that’s meant to make sure you fully understand the concepts.
Day 3 – Go back and study the guidelines once again starting with the Page Quality section.
Day 3/4 You can start the second part of the exam which is the Page Quality section if you feel comfortable on day 3. You may want to study this entire day, and you’ve got plenty of time if you decide this to take the exam on day 4.
Day 5 – Read the Needs Met section of the guidelines again and make sure to take your time. You’ve still got two more days to finish. And as it has been mentioned before, have your guidelines open in case you need to refer back to them.
Days 5/6/7 – Get started on the last part of the exam ‘Needs Met Rating’
Internet Assessor and Yukon projectexam part 1 (theory)
In this part of the exam you will be assessed on how well you understand the theoretical part of rating concepts, as well as on your ability to interpret questions and search for information in the general guidelines. It is an open book exam, so you can use the general guidelines to find answers to the test questions. This part has 32 question sections; each section has 4 parts, so in fact you have 132 true/false questions to answer. Read questions carefully, some of questions can be tricky in terms of their wording. You can consult with the guideline book while passing the exam and it’s strongly recommended to use this opportunity.
Note: Do not leave incomplete tasks more than for 1 hour, otherwise they will expire and you will fail the exam automatically. If you need to take a break, complete tasks and press “Submit and Stop rating”.
In most cases an hour would be enough to complete the theoretical part of the exam. The following message will be shown to you once you have submitted the last page of questions:
No tasks are currently available. Please try again later.
Within a few minutes a result notification email will come to your gmail, and in the case you are successful with part 1, the next part will be available in a few minutes minutes.
Lionbridge Internet Assessor and Appen Yukon projectexam part 2 (Page Quality)
In this part of the exam you will be asked to assess 7 web pages for their quality. My advice for this part is to check all quality aspects, and fill in the online form very carefully so you can give a correct overall quality score (from lowest to highest) to a page. You should expect 2-3 low quality web pages, 2 high quality web pages and 2-3 medium quality web pages. However this is not a rule, sometimes candidates get 4-5 pages of high or medium quality pages and only 1 or 2 of low quality pages to rate. Keep this in mind. In order to give correct ratings, examine web pages for ETA (Expertise, Authority and Trustworthiness) criteria very carefully.
Also, keep in mind that in this part of exam you need to give a PQ rating mainly to a web page and not to a website, although some quality aspects of a website have to be taken into consideration when assigning overall PQ rating, in particular in case of YMYL (Your Money Or Your Life) content which Google takes very serious.
Do not forget to write useful comments where required, otherwise the system may show an error.
You don’t need to finish this part at once. If you want, you can take a rest. The tasks are still available when you sign in. Again, if you pass this part, the company will send you an e-mail saying that you can proceed with part 3.
Note: The url/web page may not open on your smartphone, in this case do not mark these as “Did Not Load”. During the exam you should not expect to give a website this flag. If you do so, your answer will be incorrect. To solve this issue try to open link using incognito mode on your browser or/and give rating using other sources of information (website reputation, etc).
Page Quality Task
Internet Assessor and Yukon projectexam part 3 (needs met)
This is probably the most difficult part of the exam, and many candidates fail the exam at this stage. There are 7 tasks, and each task contains 8-12 result blocks. In each task you will be given a certain query, and a list of search results blocks with links where you have to give a usefulness rating to search results using a scale from “Fails to Met” to “Fully Met”.
Internet Assessor and Yukon projectexam Passing score
In order to pass this part you have to get an overall score of at least 80%.
YUKON EXAM PART 3 (NEEDS MET) SEND TO DEVICE – OPENING LINKS ON MOBILE & MULTIPLE Accounts LOGIN ISSUE
Note: while doing Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3, keep in mind that all links have to be opened on your phone because Google analyses web pages from the point of view of mobile users. If you do not have a smartphone, you may use any android emulator for this purpose, in some cases it is even more convenient than using a smartphone for opening links.In order to open the link on your mobile phone, click the link on the result block on your desktop and then go to www.raterhub.com/url on your mobile browser (Google Chrome) to visit the link. You must be logged in your gmail account in order to have access to raterhub platform on your phone. The same rule works if you use android emulator.
Here is an official Leapforce (now Appen) video instruction on how to open links on mobile. Please note that this video was created by Leapforce and uploaded on Youtube by someone.
Note: Some candidates when accessing the link (www.raterhub.com/url ), got informed that Gmail account was unauthorized to visit the link. Most likely reason for this error is the fact of being logged on multiple Gmail accounts on a device. The solution for this problem is to go to android setting -> users & accounts -> add project associated Gmail account, then restart your device. This setting should fix the issue.
Lionbridge Internet Assessor and Appen Yukon projectexam part 3 (Needs Met) Expert Tips and recommendations
Additionally I would like to give general tips on how to proceed with Yukon (Appen) and Internet Assessor (Lionbridge) exam part 3:
1. Make sure that you fully understand the query and user intent. If you do not fully understand user intent, make additional research on the Internet.
2. When you are evaluating landing pages in the exam, don’t place too much importance on the Page Quality slider. The focus needs to be on Needs Met Ratings. Do not worry about using flags in the exam such as the ‘Upsetting-Offensive Content’ flags.
3. One of the most important things is to absolutely know when to use Fully Meets. Fully Meets ratings are very rare compared to Highly Meets ratings. Avoid being too generous on landing pages. If you have any doubts at all, you should not be rating a page ‘Fully Meets’.
4. Determine what type of query (user intent) is:
o Single entity (business) query with local or non local or both intent/to visit in person/navigate to a website
o Business chain query or query which is looking for business within one category (e.g. car repair service near me)
o Simple know query (where the user requests a specific piece of information, do not assign the Fully Meets rating to any particular result, the correct answer must be present in an SCRB).Examples of simple know queries:paris weather, delta airlines customer service number, current time in Amsterdam)
o Information query seeking a long answer/piece of information (what cases stress)
Types of SCRBs (Special Content Result Blocks): ny weather (simple know query), how old is universe (simple know query), does god exist (long answer seeking), berlin airport (single entity), burger king amsterdam (chain), coffee shops amsterdam (categorical).
o URL/website query
5. Consider the following aspects while giving ratings to result blocks: user location, for local intent queries far away results are useless or slightly relevant. This aspect is very important for the following types of queries: tire shops near me, where to eat near me, Walmart near me, etc. When you get a task with this type of query do use in built map for researching user location/view port and results.
6. Make sure you give correct ratings to special content result blocks for single entity and business category queries. In case of business chain/category query, the result block with a list of businesses is always better than just result for a single business within one category.
7. Consider expertise and trustworthiness of search results for financial, legal, and health related queries; low expertise and trustworthiness automatically reduce needs met rating of the results.
6. Think of dominant, possible common, and minor interpretations for the query and take this aspect into account while giving ratings to a result block. Minor interpretation results should get a low usefulness (Needs Met) rating.
8. Take into account how well the web page/website is optimized for mobile users.
9. Once you have analyzed and considered all listed above aspects, you should give a rating to result blocks one by one.
Note: In some cases there might be some results (links) which are not accessible from outside U.S. (usually websites of well known retailers such as Target, Walmart, etc. They ban access from foreign IPs to prevent hacker attacks). Appen, Raterlabs or Lionbridge do not take into account this factor and therefore rating such results as fails to met may lead to overall fail.
To make sure you do not miss anything before proceeding with part 3, follow pre-exam instructions to pass this part. There will be a link to a video. Watch it and follow the instructions.
Examination Needs Met Task
Note: In part 3 of the exam (Needs Met), most of results blocks have Needs Met and E-A-T (previously PQ) sliders. Main focus should be on providing correct Needs Met rating, which in some cases might be affected by E-T-A (PQ) rating when it comes to YMYL queries and results. Leapforce/Google re-named PQ to E-T-A in part 3 to make it clear that the most important aspects in PQ are expertise, trustworthiness and authority. In general it is similar to PQ concept. While giving E-T-A rating you should evaluate MC under these 3 criteria. In most cases you may do it fast at a first glance approach, but be sure to provide good evaluation for E-T-A when you have YMYL query.
Note: If the user location map does not show user location correctly, and you have a blank page, hit the black arrow button. The task will be closed, but you will see “Incomplete tasks” on the evaluation platform. Click continue and come back to your task with the updated user location field. Remember that it is very important to take into account the user location while giving ratings to block results. Also keep in mind that there is no need to finish the task in one sitting. You can take a break if you need to.
Once you complete 7 tasks you will get this message:
No tasks are currently available. Please try again later.
Yukon (Appen) exam results and Internet Assessor (Lionbridge) exam results
You will get overall exam results on the exam due date from Appen and exam due date plus one day if your vendor is Lionbridge. Appen will ask you to sign a few electronic forms and in 2-4 days you will be given access to live tasks. At Lionbridge you will have to complete educational course with no pass/no fail quiz before you are granted full access to the tasks.
US YUKON Raterlabs and Lionbridge raters
Once you pass the exam you will then go through the employment process which might take up to 3 weeks. You will be asked to complete an online version of the IRS Form 8850 (Pre-Screening Notice and Certification Request for the Work Opportunity Credit), and a tax credit questionnaire from a vendor payroll provider. After you have completed these two forms, you will be presented with a formal employment offer letter. After acknowledging your offer, you will still need to complete the required activities in a timely manner. Try not to postpone the completion of these important steps: Providing bank details for ACH payments on Paycom website. Providing your I9 Employee Section 1 and I9 Employment Eligibility Documents (section 2) must be completed by specified deadline. It is very important that candidates complete all required steps as soon as possible to avoid termination of your employment.
NEW!!! Welcome to Search Engine Evaluation Exam (Yukon, Internet Assessor) Simulation Tool!
What is The Exam Simulation Tool?
I have created a special tool that will help anyone who is getting ready or considering taking the exam. This tool was designed to assist candidates with exam preparation with the main focus on practical rating tasks that include Page Quality, and Search Results Satisfaction ratings. The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met) tasks of all the major query types. (If you’re not sure what a query type is, that is the keyword that a user searches Google for). These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.
How can this tool can help you pass the exam?
The way the tool is designed will show you exactly your problem area and what you need to focus on. It helps you identify your level of understanding of the rating concepts, and your ability to apply them by practicing first, and then identifying your weak points. You will receive your overall score (pass or fail), detailed feedback on your mistakes and know exactly what you need to go back and study in the guide. Of course we need to mention that there are no real actual exam tasks because that isn’t going to help you understand anyway. Each one of these is hand designed from the most typical exam tasks, and is also designed to be the best simulation possible. If you pass this, you will be ready to pass the actual exam.
How do I get my hands on the exam simulation tool?
If you’re still skeptical, we have a free demo version which you can try FREE right now! It contains a simplified Needs Met and one Page Quality task so you can see how it works. Once again, the demo is 100% free and remain completely anonymous.
The tool has 5 Page Quality tasks, and 5 Search Results Satisfaction (Needs Met) of all the major query types. These include; simple-know, local search, information search, website/navigation, visit-in-person/business which cover basically all rating concepts/tasks types and skills which will be tested during a real exam.
If you are not confident about going through the Appen Raterlabs or Lionbridge qualification exam, you may always get my expert support and advice. For detailed information contact me here.
Payoneer Payment method for Appen
Leapforce/Appen/Raterlabs candidate? If you get a job at one of these places, you’re going to need Payoneer account because that is the only payment option available for these companies. Simply sign up for Payoneer under my referral link and you’ll get the simulation tool subscription fee back as Payoneer subscription 25$ bonus once your account is loaded with 1000$! Normally it takes 1-2 months to make 1000$ for EU/USA/Canada raters and 2-4 months for raters from another countries.
Leapforce/Appen/Raterlabs candidate? If you get a job at one of these places, you’re going to need Payoneer account because that is the only payment option available for these companies. Simply sign up for Payoneer under my referral link and you’ll get the simulation tool subscription fee back as Payoneer subscription 25$ bonus once your account is loaded with 1000$! Normally it takes 1-2 months to make 1000$ for EU/USA/Canada raters and 2-4 months for raters from another countries.