Maps TryRating Assessment Quick Start Guide by Peroptyx

Quick Start Guide Overview

Maps TryRating Assessment
Quick Start Guide
(20 minute read)
Peroptyx Co
Welcome to the TryRating Maps Assessment Quick Start Guide.
This guide provides you with important insights needed to successfully
complete the Maps Assessment.
This guide presents the most common mistakes made when evaluating
maps search results using the TryRating assessment tool. These mistakes
can be avoided by reviewing the examples and explanations outlined
below. In addition, you should refer to the Maps Search Guidelines and
perform an appropriate amount of real-world research when preparing
your answers.
Familiarising yourself with the examples below can help ensure you
achieve a passing grade of 30 out of 40. Please review each common
mistake carefully and refer to each section of the guidelines mentioned
below.
Please don't hesitate to reach out to our learning support team at
learn@peroptyx.ai should you need any additional support.V3.2

Pre-Assessment Guide

Pre-Assessment Quick Start Guide
Peroptyx

Purpose of Maps Evaluation

The Purpose of Maps Evaluation
Maps search results need to be useful to the user (including you!)
When a user searches on a map (a map query) they are looking for results that are most
relevant to the intent of their query, and require that the result data be accurate.

When Search Results Are Not Useful

Search results are not useful when they:

• Fail to satisfy the user's intent, or
• Fail to provide sufficient map data for the result.

Failing to Satisfy User Intent

Search results can fail to satisfy the user's intent when they:

• Bear no relation to the map query,
• Are in an inappropriate location (relative to the map query),
  o
  Locations are not judged relative to the user's location
  o Locations are judged relative to the user's location when the user is
  intentionally looking at different part of the map
• Ignore more relevant or more suitably located results.

Insufficient Map Data

Search results can fail to provide sufficient map data for the result when:

• The data is insufficient for the user to verify that the result satisfies their intent,
• The data is insufficient for the user to adequately locate the result,
  o
  Data is in an inappropriate language for the user query
• The data is less accurate than otherwise possible.
  o The address is missing/includes a component,
  o A business category is misclassified,
  o
  The pin marking the location is in an inaccurate location.

Assessment Skills

You will be assessed on your ability to use the TryRating Tool to correctly rate the
relevance and accuracy of search results.
This requires three distinct skills which will be assessed.
Tal

• Appropriate Judgement: to ensure that the relevance of the result satisfies the
  user's intent, requiring contextual knowledge and decision making.
• Attention to Detail: to ensure that small inaccuracies are caught, and the data is
  entered in a suitable fashion for the map hosts.
• Adequate Research: to ensure the relevance and accuracy of results relative to real-
  world information (e.g. from certified, trustworthy or authorised resources)

In order to help you prepare to pass the Map Assessment, provide useful rating data, and to appropriately
acquire and perform these three skills, we have created this Quick Start guide, a TryRating Tips video plus
access to the Maps Search Guidelines themselves which are available on this page of your Connect profile.
To see the TryRating tool in action please watch our TryRating tips video
2Peroptyx

Maps Search Guidelines

The Maps Search Guidelines provide detailed terminology, explanations, and examples of
the general structure of how to rate relevance of results and accuracy of data. They also
provide specific, key criteria for difficult to judge edge cases and boundary decisions.
If in Doubt, Search the Guidelines.

TryRating Tool Interface

The TryRating Tool
Map Assessment Tasks are provided through an interface that looks similar to the image
below. All key terms and definitions can be found by searching the guidelines.
They contain the key data concerning the:

• Query:
  o query, text, language, and locale,
  o
  the user location,
  o
  the viewport (what the user's screen is displaying), the viewport age (STALE or
  FRESH)
• Result:
  o
  Result name, address, classification
  Location of result, distance to user, distance to viewport

As well as the

• Rating response form:
  o Language issue, and closure/non-existence issue
  o Relevance drop-down (NAVIGATIONAL, EXCELLENT, GOOD, ACCEPTABLE, BAD)
  Pop-up Checkbox for user intent and distance/prominence issue
  o Data Accuracy drop-downs, Name, Address, Pin
  " Pop-up Checkbox for address component issue
  o Comment box explaining demotions/data inaccuracies.

Ashburn
Lowes Island
Wheaton
White Oak
Belt Standard
Broadlands
Potomac
Chevy Chase
Bethesda
Silver Spring
Greenbelt
Brambleton
National Park
7
Cabin John
Chillum
50
450
Lanh
Washington
.
Reston
McLean
50
International
South Riding .
Airport (LAD)
Tysons
Landover
Query Locale:
(The country and language combination this query is for)
Query:
(The term the user entered into the search bar)
15
Fairfax
796
Suitland
Centreville
Users Location
Manassa
User Location
(The latitude, longitute co-ordinates for the user's location)
leld Park
Fairfax Station
.
Rose Hil
Burke
1
Clinton
Ros
imore
Manassas
1
Woodlawn
Chelf
Bristow
Result Name
Mount Vernon
Fort Washington
Nokesvile
234
C
Piscataway Park
Address:
(Street name, locality -
Refer to GLS for more address components)
Independent Hill
Bryans Road
Waldorf
Classification
( (If included), will be shown beneath the result name/title.)
Indian Head
Bensvile
Distance to User
(Distance of this result location from the user)
Prince William
Forest Park
224
Pomfret
Triangle
La Plata
95
Quantico
301
Lat, Lng
(Result coordinates on the map)
Doncaster
6
Dentsvile
Aquia Harbour
Nanjemoy
N
Business/POI is closed or does not exist
17
Stafford
Maps
Legal
- +
« Back to landing page
PEROPTYX
Center User
Center Viewport
Vienna
267
Chantilly
Oakto
Oaks
495
Viewport age:
(How long ago the user moved the viewport. Fresh/Stale)
National
Oxon Hill
Gainesville
Clifton
.
Springfield
.
Linton Hall
Friendly
Viewport Location
(The viewport is the area in the purple box the user was
looking at when making the query)
1. Result Name
Brandywine
Woodbridge
ale City
White Plains
Cherry Hill
Bryantov
Distance to Viewport
(Distance of this result location from viewport centre)
Bel Alton
·
Goldvein
Result name/title is in unexpected language or script
4
Relevance //(will be evaluated for each individual result.)
To see the TryRating tool in action please watch our TryRating tips video
3
t
Greenbelt
Sterling
Great Falls
Rock Creek Park
Dulles
Washington
Alexandria
28
Fort Hunt
o

Common Rating Mistakes

Pre-Assessment Quick Start Guide
V3.2V3.2
Pre-Assessment Quick Start Guide
Peroptyx
13 common rating mistakes.
Many applicants have failed to rate the following scenarios correctly and have failed the
assessment as a result.
To ensure you have the highest chance of passing the assessment, we encourage you to
read and familiarise yourself with the following map rating concepts.
The mistakes applicants make fall into these general categories:

• Location intent: applicants incorrectly judge the location intent of the user.
• Chain queries: applicants apply general relevance criteria to chains, instead of
  specific criteria as outlined in the guidelines.
• Attention to detail: applicants make small detail errors due to lack of familiarity
  with the guidelines, speed, tiredness, or external factors.
• TryRating interface: the final common mistake is due to lack of familiarity with the
  structure of the TryRating user interface itself, leading to confused responses.

+
If in Doubt, Search the Guidelines.
ridential
To see the TryRating tool in action please watch our TryRating tips video
4Peroptyx

Location Intent: Fresh Viewport

Pre-Assessment Quick Start Guide
V3.2
#1: Location Intent, User in Fresh Viewport. When the user is inside the 'fresh'
(i.e., refreshed) viewport the location intent becomes the user location unless there is an
explicit location intent for the query.
Please refer to and familiarise yourself with section 2.3.2 (Implicit Location) and section 5.6.
(Many Possible Results) of the Maps Search Evaluation Guidelines (or 'Maps Guidelines')
Many applicants fail to use the user location as the location intent in this scenario.
A common mistake would be to rate every result inside the viewport as excellent, or
`demote' results further away from the user (but still inside the fresh viewport) to 'bad'.
Here is an example from the Maps Guidelines on how to rate a scenario like that correctly.

Starbucks Rating Example

Query and Screenshot
Result
Rating and Explanation
Relevance
Excellent
1 Starbucks
865 Market
Street
Rate Excellent for Starbucks locations that are in
close proximity to the user.
[starbucks]
Starbucks
170 O'Farrell St
Rate Good for locations that are a bit farther
away from the closest relevant locations to the
user.
-
4
Starbucks
580 California St
3 Starbucks
264 Kearny St
Rate Acceptable for relevant locations that are
even farther away from the user but still inside
the viewport.
.
3
Starbucks
264 Kearny St
C
2
Starbucks
170 O'Farrell St
@ Starbucks
580 California
St
Starbucks
140 Mason Street
When there are many locations close to the user,
rate Bad for locations that are significantly farther
t away and outside the viewport.
1
Starbucks
865 Market Street
Relevance
Excellent
5 Starbucks
140 Mason
Street
Rate Excellent for Starbucks locations that are in
close proximity to the user, even if they are
outside the viewport.
Relevance
Good (Distance/Prominence)
User and fresh viewport in San
Francisco, CA
PFRAIS
C
Relevance
Acceptable (Distance/
Prominence)
Relevance
Bad (Distance/Prominence)
5
To see the TryRating tool in action please watch our TryRating tips video
5

Location Intent: Partial Addresses

V3.2
Pre-Assessment Quick Start Guide
Peroptyx
#2: Location Intent, Partial Addresses. For queries where the location intent is not
explicitly stated in the query, you must infer (or estimate) the location intent from the user's
viewport and the user's location.
One difficulty in evaluating address queries has to do with partial addresses, which could
refer to multiple locations.
Generally, for partial addresses, users are looking for the location closest to their position or
within/ near the fresh viewport.
. Results that are farther away from an ideal result but can potentially satisfy the
user's intent can be rated as high as 'Good' (Distance/Prominence).
. Results are demoted based on distance.
. Results are demoted based on the density of potential results in a given area.
. Results that are too far away will be rated 'Bad' (Distance/Prominence).
ACTION: Please refer to section 10.2. (Non-Specific Address) of the Maps Guidelines.
Many applicants fail to do real-world research hence missing that there might be closer
results to the user or fresh viewport.
If in Doubt, Search the Guidelines.
Ifidential
To see the TryRating tool in action please watch our TryRating tips video
6