Toward Blind Travel Support through Verbal Route Directions: A Path Inference Algorithm for Inferring New Route Descriptions from Existing Route Descriptions

22 The Open Rehabilitation Journal, 2012, 5, 22-40

Open Access
Toward Blind Travel Support through Verbal Route Directions: A Path
Inference Algorithm for Inferring New Route Descriptions from Existing
Route Descriptions
Vladimir Kulyukin*,1 and John Nicholson2

1
Department of Computer Science, Utah State University, USA
2
Department of Computer Science, Austin Peay University, USA

Abstract: The work presented in this article continues our investigation of such assisted navigation solutions where the
main emphasis is placed not on sensor sets or sensor fusion algorithms but on the ability of the travelers to interpret and
contextualize verbal route directions en route. This work contributes to our investigation of the research hypothesis that
we have formulated and partially validated in our previous studies: if a route is verbally described in sufficient and
appropriate amount of detail, independent VI travelers can use their O&M and problem solving skills to successfully
follow the route without any wearable sensors or sensors embedded in the environment.
In this investigation, we temporarily put aside the issue of how VI and blind travelers successfully interpret route
directions en route and tackle the question of how those route directions can be created, generated, and maintained by
online communities. In particular, we focus on the automation of path inference and present an algorithm that may be used
as part of the background computation of VGI sites to find new paths in the previous route directions written by online
community members, generate new route descriptions from them, and post them for subsequent community editing.
Keywords: Independent blind travel, path inference, informat.

1. INTRODUCTION Golledge, Klatzky, and Loomis [14] demonstrate that
sighted individuals are able to “process data in a continuous,
Commercial and research systems have been developed
integrative, and gestalt-like manner.” Sighted travelers
to increase travel independence for visually impaired (VI)
localize on routes through large-scale geographic
and blind travelers. Various technologies, including GPS [1,
observations. For example, while traveling an outdoor route,
2], Wi-Fi localization [3, 4], and infrared beacons [5], have
a sighted traveler notices that the next sidewalk intersection
been proposed. Cost and quality of service have been is approximately fifty feet away from his current location
adequately discussed in the literature and found to play a
and that he is walking towards the main clock tower on
significant role in the slow or low adoption rates of these
campus, which happens to be hundreds of feet away.
technologies [6-10]. Another contributing factor is the trust-
me-you-are-here approach: assisted navigation systems take The same researchers [14] argue that, unlike sighted
readings from their sensor sets, localize those readings on travelers, travelers with visual impairments, especially those
some maps, and instruct travelers on where to move next to with complete vision loss, “actively search the environment
reach destinations. in a piecemeal manner.” For example, a cane user with
complete vision loss must physically encounter the same
Unfortunately, sensor readings can be noisy, irrelevant,
sidewalk intersection to actively sense it with the cane and
or absent. Garmin (www.garmin.com) reports that its GPS
may not be aware of his orientation in relation to the clock
units are accurate to within fifteen meters. Place Lab [11], a
tower. Consequently, erroneous or irrelevant assistance may
Wi-Fi indoor solution, achieves a median location error of be more harmful than helpful to this traveler, because it takes
fifteen to twenty meters. Since navigation can be viewed as a
the traveler’s cognitive energy away from critical decision
time series that unfolds as the traveler moves through an
making.
environment, sensor signal latency may render automated
decisions irrelevant when they are based on the sensor One sensor that many trust-me-you-are-here solutions
readings that are no longer representative of the traveler’s attempt to replace or tend to downplay is the traveler's brain.
actual location [12]. Such automated decisions, although Many VI and blind people receive extensive O&M training.
they purport to help the traveler, may in reality hinder the During training, individuals learn valuable skills that enable
traveler’s situational awareness [13]. them to safely and successfully navigate many indoor and
outdoor environments independently [13, 14]. They learn to
perform actions such as following sidewalks, detecting
*Address correspondence to this author at the Department of Computer
obstacles and landmarks, and crossing streets. They also
Science, Utah State University, Logan, UT, USA; Tel: 435.787.8163; learn how to remain oriented inside buildings or on
Fax: 435.797.3265; E-mails: vladimir.kulyukin@usu.edu, sidewalks and streets. Many individuals subsequently
vladimir.kulyukin@aggiemail.usu.edu improve their O&M skills through independent traveling and

1874-9437/12 2012 Bentham Open

Toward Blind Travel Support through Verbal Route Directions The Open Rehabilitation Journal, 2012, Volume 5 23

acquire new wayfinding skills specific to their life on turn locations, one-way or two-way streets, street
experiences. intersections, and stop signs or traffic lights.
The work presented in this article continues our research Another limitation of current map services is lack of
of such assisted navigation solutions where the main information for indoor environments. To the best of our
emphasis is placed not on sensor sets or sensor fusion knowledge, one exception is the site
algorithms but on the ability of the travelers to interpret and www.clicandgomaps.com that offers a narrative mapping
contextualize verbal route directions received en route. This service to render public facilities such as schools, airports,
work contributes to our investigation of the research and hotels accessible to blind and deafblind travelers through
hypothesis formulated and partially validated in our previous route descriptions written by professional O&M instructors.
studies [9, 15-17]: if a route is verbally described in Another promising approach is http://redpin.org/, an open
sufficient and appropriate amount of detail, independent VI source indoor positioning system that seeks to omit the time-
travelers can use their O&M and problem solving skills to consuming training phase and rely on travelers to train the
successfully follow the route without any wearable sensors multi-sensor (GSM, Bluetooth, and Wi-Fi) system while
or sensors embedded in the environment. using it in specific environments.
In this article, we investigate the question of how these 2.1. Volunteered Geographic Information
route directions can be automatically generated from existing
route directions. In particular, we focus on the automation of Volunteered geographic information (VGI) [19] is a
path inference and present an algorithm that may be used as bottom-up volunteer movement that sprang up to address
part of the background computation of VGI sites to find new shortcomings of commercial mapping services discussed in
paths in the previous route directions written by online the previous section. VGI communities seek to provide free
community members, generate new route descriptions from community-based GIS tools, data, and websites. VGI sites
them, and post them for subsequent community editing. encourage volunteers to provide GIS data. Many volunteers
upload small pieces of appropriate information. A VGI
Our article is organized as follows. In Section 2, we website’s back-end software, over a period of time,
review current Web map services and our previous research assembles these pieces into cohesive data sets. Although the
on community route information sharing. Section 3 discusses data quality may be inferior to that provided by professional
automated landmark autotagging in written route services, VGI sites often contain unique data not available
descriptions. While landmark autotagging is not the focus of elsewhere. For example, volunteers who live in a particular
this article, it is an important step in the automated route area are more familiar with that area than professional GIS
analysis done by the path inference algorithm presented in cartographers located elsewhere.
this article. Section 4 gives a detailed path inference
example. Section 5 shows how route descriptions with Many VGI sites provide the same services as
tagged landmarks are converted into a directed weighted professional sites. OpenStreetMap [20] is a VGI site for
graph. Section 6 presents a detailed description of the path creating large map data sets. Such data sets are similar to
inference algorithm. In Section 7, we give examples of the those of commercial sites such as Google Maps in that both
paths that were inferred by our algorithm and the use vector map data representations. Google, because it uses
corresponding route descriptions that were generated from commercial sources for its map data, offers satellite images
the inferred paths. Section 8 places our research in the unavailable in OpenStreetMap. However, OpenStreetMap
context of related work in natural language processing (NLP) features sidewalks and lesser known buildings. Wikimapia
and information extraction (IE). (http://wikimapia.org) is a VGI site that enhances Google
Maps with dynamic annotations. Trailpeak
2. MAP SERVICES (http://www.trailpeak.com) is a VGI site where users add,
edit, view, and download free-text trail descriptions, trail
Passini and Proulx [18] show that VI travelers make
directions, reviews, and GPS waypoints related to trails for
more decisions en route and use more information than
hiking, mountain biking, and kayaking in Canada and the
sighted travelers. The Internet abounds in map-oriented sites
U.S.
and services, e.g., Google Maps (maps.google.com),
MapQuest (http://www.mapquest.com/), and Yahoo! Maps A common strength of the VGI sites is the local user
(maps.yahoo.com). While these websites are useful to knowledge of GIS data. VGI site volunteers, because of their
sighted travelers, VI travelers are generally unable to use local experience and knowledge, create data sources that can
these sites, because much of the information is presented be as credible as traditional sources. VI travelers also possess
visually. credible sources of route knowledge for the areas where they
Google Maps has addressed this problem to a degree. live, work, or travel. Their expertise is a combination of
familiarity with routes and a deep understanding of VI
When using the mobile version of Google’s route direction
travelers’ cognitive and O&M skills. If this knowledge could
service at http://www.google.com/m/directions, the user is
be shared, it may be possible for VI travelers unfamiliar with
presented with a simpler, text-based user interface that works
the area to use this knowledge for traveling unfamiliar
with screen readers and allows users to receive text-only
routes.
route directions. Unfortunately, these directions are not yet
adequate for the needs of VI travelers. For example, when a 2.2. Community Route Information Sharing
warning “Use caution - This route may be missing sidewalks
or pedestrian paths” is issued, the segments with the missing Our previous research on independent blind travel offers
features are not textually identified. No information is given some evidence that, given sufficient context-sensitive verbal
assistance, independent VI and blind travelers can navigate

24 The Open Rehabilitation Journal, 2012, Volume 5 Kulyukin and Nicholson

complex and dynamic environments such as supermarkets The second data structure is a directed graph that encodes
[15, 16] and college campuses [21]. There exists research the current set of NL route descriptions. Route descriptions
evidence that VI travelers successfully share route and statements have unique IDs. Each route description has
descriptions with each other via spoken language [22], start and end locations and a NL description that guides from
especially via smartphones. the start location to the end location. The start and end
locations reference landmarks in the hierarchy. The
We have previously proposed a Community Route
description is stored as a list of route statements. A route
Information Sharing (CRIS) framework for VGI sites to
statement is a sentence from the user’s route description.
capture and share the local expertise of VI travelers [17].
Users may tag route statements with landmarks. For
CRIS is intended to provide user-adjusted levels of
example, one user may add a route description with the
information for independent VI travelers to follow routes in
indoor and outdoor environments without external sensors. following route statement: “Turn right when you detect the
intersection with the main hall.” Another user may tag this
External sensors, when available, can still be put to use. The
route statement with the tag “WATER FOUNTAIN.”
simplest implementation of a CRIS site is a Wiki where
users add, create, and edit data, resulting in large and 3. LANDMARK AUTOTAGGING
dynamically managed repositories of knowledge.
One set of techniques used to extract information from
Since many VI travelers perceive the world in terms of unstructured text into structured formats is Information
routes rather than layouts [14], CRIS knowledge bases will Extraction (IE) [24, 25]. Since IE has been researched for
likely consist of large collections of route descriptions. The over a decade, there are now IE frameworks that allow
route descriptions cover routes that are entirely indoors, researchers to develop IE applications and tools for specific
entirely outdoors, or a mixture of both. As the collection of domains. In this section, only a brief description is given of
route descriptions for a given area grows, the volunteer how IE is used for landmark autotagging in our system. A
community creates and subsequently maintains route-based detailed presentation and performance analysis are given in
maps. All descriptions are written or, as the quality of online [23].
speech recognition improves, spoken in natural language
(NL) and converted into free text. Information extraction Landmark autotagging falls in the category of named-
(IE) is used to extract landmarks so that route descriptions entity recognition (NER) [45]. NER modules identify text
are subsequently transformed into directed graphs. The chunks that refer to specific categories such as names of
graphs are searched for new paths. The new paths are used to persons, organizations, and locations. NER can be done with
generate new NL route descriptions that are added to the rule-based and statistical models [26, 27]. We did not use
database for subsequent collaborative human editing. statistical NER, because it requires large annotated corpora.
Our rules were developed manually on a small corpus of
A CRIS environment is represented as two data indoor and outdoor route descriptions written by VI travelers
structures: a landmark taxonomy and a directed graph of in an online survey [23]. The survey solicited two route
route descriptions. A landmark is any location, object or descriptions from each respondent. The respondents were
sensation in a route description that can be detected through first asked to describe an outdoor route from the entrance of
travelers’ sensorimotor means. Landmark can be rooms, one building to the entrance of another building and then
doors, hallway intersections, buildings, sidewalks, streets, asked to describe an indoor route from one room in a
smells (e.g., “when you smell bread baking”), and textures building to another room in the same building. The
(e.g., “where you feel the carpet change to tile”). Landmarks instructions required that respondents describe real-world
are assigned unique integer IDs and may have alternative routes with which they were familiar. Respondents were also
linguistic references. For example, the landmark “ROOM instructed to write the route descriptions as if they were
414” representing room 414 in the Old Main building on the describing the route to a fellow traveler with the same visual
Utah State University (USU) campus could have the impairments and the same traveling experience and skills.
alternative names “COMPUTER SCIENCE FRONT
OFFICE,” “CS FRONT OFFICE,” or “CS DEPARTMENT We received 52 responses, providing 104 English
HEAD’S OFFICE.” Landmarks in the taxonomy are language route descriptions: 52 indoor route descriptions and
organized in terms of the standard part-of relationship. Users 52 outdoor route descriptions. Two thirds of the route
can add and remove landmarks or change a landmark’s place descriptions were placed in the training set for the purpose of
in the hierarchy. analysis and IE rule development. One third was placed in an
evaluation set to test the effectiveness of the IE rules. The
Landmarks can be viewed as tags or metadata that give outdoor routes contained approximately twice as many
meaning to other data. Flickr (www.flickr.com) and sentences and words as the indoor route descriptions: 18.5 vs
YouTube (www.youtube.com) have user-created tags for 9.9 and 277.3 vs 148.7, respectively. The average number of
image and video content, e.g., “ORIGAMI.” One problem words per sentence was consistent across all descriptions:
with the Flickr and the YouTube tags is their inconsistent 14.98 for indoor vs 14.99 for outdoor. The number of
naming structure and their flat namespaces. The tag landmarks mentioned in outdoor route descriptions was
“DOORS,” for example, could be associated with a photo approximately double that number in indoor descriptions:
containing multiple wooden doors in a hall as well as a photo 37.1 vs 19.5. The number of landmarks remained consistent
of the members of the classic rock band. For more on a per sentence basis with 1.96 per indoor sentence vs 2.00
information on our landmark hierarchy interested readers per outdoor sentence.
may refer to [17, 23].
If our corpus had been larger, serious consideration could
have been given to machine learning techniques for


automated rule creation [28]. However, since there were only The first part of R-B can be combined with the second part
104 route descriptions, we decided to use the General of R-A to form the new route R-C from Old Main to Ray B.
Architecture for Text Engineering (GATE) system from the West shown Fig. (3). Path inference is the process of
University of Sheffield [24, 25] for defining landmark inferring new, previously unknown routes from sets of
autotagging rules. GATE’s IE component is ANNIE (A previously known routes. R-C is inferred from the known
Nearly-New IE system). ANNIE uses a Java-based pattern routes R-A and R-B due to the landmark (L-B) shared by R-
matching language (JAPE) [24] for writing patterns to A and R-B. Path inference is a time saving utility, because
extract matching information from text. JAPE is based on the the manual writing of route descriptions takes time, and an
Common Pattern Specification Language (CPSL) [29]. area such as a university campus requires many route
Interested readers are referred to [25] for details on GATE, descriptions for the coverage to support independent blind
ANNIE, and JAPE. travel.
ANNIE consists of several processing resources (PR): 1) R-A Description: Exit the Animal Science building
Tokenizer; 2) Gazetteer; 3) Sentence Splitter; 4) Part-of- doors on the south side. Walk straight until you find the
speech (POS) Tagger; 5) Named Entity Transducer; and 6) sidewalk entrance to the Quad’s sidewalk. Pass the main
Orthomatcher. The Tokenizer splits a text into basic tokens sidewalk intersection. Walk south until you detect a road and
such as numbers, words, and punctuation. The tokenized text then carefully cross the street. Continue to walk south until
is passed to the Gazetteer to find well-known entities, e.g., you find the doors to the Ray B. West building.
the names of all the employees in a company. The text is
R-B Description: Exit Old Main walking east. You will
then passed to the Sentence Splitter for sentence location.
walk through the Quad, passing the intersection. Keep
The POS Tagger assigns parts of speech to each token in
walking straight until you run into grass and then turn left,
each located sentence and passes the text to the NE
walking north. Walk until you detect the bike racks on your
Transducer to run JAPE rules for identifying named entities. right and then turn right. Walk east until you find the stairs
Finally, the Orthomatcher is run to find co-references.
leading to the entrance to the distance learning center.
In our system, landmark autotagging is treated as in the
NE subtask. Known entities are the landmarks that have been 5. TRANSFORMING TAGGED DESCRIPTIONS INTO
DIGRAPHS
identified through the process of tagging or are included in
the landmark hierarchy. Finding known entities is done in Path inference starts after landmarks are labeled in route
the Gazetteer. A file is created listing each possible string for statements though autotagging. Tagged route descriptions
a given entity type. Unknown entities, including unknown are transformed into a digraph. Given a starting landmark
landmarks, are entities that cannot be derived from look-up and an end landmark, the digraph is searched for a path
lists. These also include words that are misspelled. If “Old between them. If a path is found, a route description is built
Main” is in the Gazetteer look-up list but a user spells it as and given to the community for collaborative editing. If no
“Old Mian,” then the user’s text is not annotated. path is found, its absence may signify that more route
Autotagging unknown entities was done by extending the descriptions may be needed.
JAPE rules for the NE Transducer. The NE Transducer relies
Digraphs consist of statement nodes and landmark nodes.
on rules written in the JAPE language. Matches can be
Digraphs are used because landmarks and statements in a
performed by specifying specific strings, previously assigned
description of a route from A to B may not be identical to the
annotations, or tests on annotation attributes. Although
landmarks from in a description of a route from B to A. For
sufficient in many cases, the JAPE syntax cannot handle
complex annotation manipulations. For such more complex example, when walking down a hall from A to B, a traveler
may trail the right hand wall and encounter a water cooler
manipulations, arbitrary Java code can be written in a
and a soda machine. When walking from B to A and trailing
specific macro. For example, when processing the string
the wall on the other side of the hall, the traveler may
“John Angus Nicholson” for names, ANNIE’s Gazetteer
encounter only three office doors.
creates three annotations: “John Angus Nicholson,” “Angus,”
and “Nicholson.” This default behavior is modified to return Each route statement is represented by a statement node.
the longest single match for these cases, i.e., “John Angus When a statement node SN1 is connected to another
Nicholson.” statement node SN2 in the graph by a precedence edge, it
means that the action described in the statement that
4. PATH INFERENCE EXAMPLE corresponds to SN1 must be executed before the action
A VI traveler may travel some routes without realizing described in the statement that corresponds to SN2. If a route
that they share common landmarks. Suppose a VI USU statement is associated with a set of landmarks, each
student who is familiar with the campus describes a route to landmark becomes a landmark node. A statement node and
a new VI student. The route description describes a route (R- landmark nodes are connected via association edges. A
A in Fig. 1) that begins at the Animal Science building (L-A) precedence edge can also connect a landmark node and a
and ends at the Ray B. West building (L-C). The route statement node. This redundancy is useful in handling
passes through the Quad, a large grassy area with two collaborative route editions. For example, if SN1 precedes
sidewalks (L-B ). At a later date, the new student is given SN2 and SN1 is deleted, the landmarks associated with SN1
another route description (R-B in Fig. 2). This route starts at are not automatically deleted as they may be associated with
the Old Main building (L-D) and ends at the Distance other statement nodes. These landmarks still precede SN2 in
Learning Center (L-E) in the Eccles Conference Center. the sense that the traveler will experience them before
Both R-A and R-B pass through the Quad’s center at L-B. performing the action specified in SN2.


Fig. (1). Example Route R-A.

Fig. (2). Example route R-B.

Fig. (3). Route R-C inferred from R-A and R-B.


Route ID: R-3

Start Landmark: Landmark(name=“QUICK STOP”, id=“L-500”)
End Landmark: Landmark(name=“THE HUB”, id=“L-789”)
Statement List
1. Statement
id: “S-3-1”
text: “You are standing with your back to the south entrance
to the Quick Stop.”
landmarks: Landmark(name=“SOUTH ENTRANCE”,
id=“L-501”)
2. Statement
id: “S-3-2”
text: “Turn left so you are walking east.”
landmarks: NIL
3. Statement
id: “S-3-3”
text: “ATM machines which make distinctive sounds, and
the campus post office and mailbox.”
landmarks:
Landmark(name=“ATM”, id=“L-550”),
Landmark(name=“POST OFFICE”, id=“L-551”),
Landmark(name=“MAILBOXES”, id=“L-552”)
4. Statement
id: “S-3-4”
...
Fig. (4). Partial tagged route description.

Consider the following route description written by a VI each of which has a unique ID. The loop at line 17 connects
USU undergraduate student. It describes an indoor route a route statement to all its landmark tags. If the route
from the Quick Stop, a small convenience store, to the Hub, statement does not have landmark tags, it is connected to the
one of the main areas on campus for buying meals. The route next route statement or end landmark if it is the last route
occurs entirely in the Taggart Student Center. The beginning statement. The check at line 19 ensures that an edge from the
of the tagged route description is shown in Fig. (4). The first route statement to the start landmark is not created. The
resulting digraph of this tagged description is shown in start landmark is set as the first node for a route in line 5 and
Fig. (5). The figure contains a pentagon node (R-3) is always connected to the route’s first statement in the for-
representing the start of the route and a connection from that loop at line 10. The final check at line 30 ensures that the
node to the route’s start landmark (L-500). The final digraph end landmark is added. If it is added, this signifies that the
also contains edge weights (not displayed in Fig. 5) end landmark is one of the landmarks in the set of tags for
discussed in Section 6.1. the route’s final route statement node. If the end landmark is
not one of the final route statement’s tags, this check adds a
You are standing with your back to the south connection between the statement node and the end
entrance to the Quick Stop. Turn left so you landmark.
are walking east. On your left you will pass the
ATM machines which make distinctive sounds, The function BUILD_GRAPH() uses the function
and the campus post office and mailbox. You CONNECT() (see in Fig. 7) to create the weighted edges.
will pass the entrance to the financial aid Whenever there is a precedence edge that connects the start
office on your right and several bulletin landmark of a route to the first route statement, the edge’s
boards. Continue walking east and passing weight is set to START_ROUTE_COST. This weight
offices, the barber shop, and the copy center signifies the start of the route and is be higher than the other
as you walk down this long hall. Towards the weights. Fig. (8) shows the diagraph created by
eastern end of the building, you will come to a BUILD_GRAPH() and CONNECT() from the tagged
wide open area on your left. Turn left and walk description in Fig. (5).
a little north. Pass Taco Time on your left, and 6. PATH INFERENCE
look for a small opening on your lift. This
opening will have a cashier counter on your The constructed digraph is used to find a path from one
right. Turn left and enter the world of the Hub. landmark node to another landmark node. If a path is found,
You will find a wide variety of food stations the statement nodes along the path are combined into a new
around a semicircle. route description. Fig. (9) shows a digraph constructed from
Tables 1 and 2. Table 1 gives the landmark IDs and names;
The algorithm that transforms a set of tagged route Table 2 associates each route statement with its route ID and
descriptions to a digraph is shown in Fig. (6). The graph’s tagged landmarks. Table 2 is the output of the auto-tagging
set of vertices, V, contains statement and landmark nodes, algorithm. Route description R-1 describes the route from


Fig. (5). Partial transformation of tagged description into digraph.

Fig. (6). Algorithm for transforming tagged route description into digraph.

Old Main (L-1) to the Distance Learning Center (L-2), and exists in the database from Old Main to Ray B. West, path
route description R-2 describes the route from Animal inference finds the path in the digraph from L-1 to L-4 via
Science (L-3) to Ray B. West (L-4). If no route description L-99, as shown in Fig. (3).


Fig. (7). Algorithm for creating weighted edges in digraph.

Fig. (8). Partial result of transforming tagged description into digraph.

To build a new route description, the statements along • S-3-1 (cloned from S-1-1): Exit Old Main walking
the path are joined to create a new route statement sequence. east.
The original route statements are cloned, including their tags,
• S-3-2 (cloned from S-1-2): You will walk through the
and the new description and its route statements are given
Quad, passing the intersection.
new IDs. Statement cloning is done so that the new route
description can be edited without affecting the original route • S-3-3 (cloned from S-2-4): Walk south until you
descriptions. In the above example, the new route detect a road and then carefully cross the street.
description’s route statement sequence: • S-3-4 (cloned from S-2-5): Continue to walk south
Table 1. Example Landmark IDs for R-A and R-B until you find the doors to the Ray B. West building.
The newly constructed route description requires the traveler
ID Landmark Name to make a right turn at the Quad’s sidewalk intersection.
However, statement S-3-2 does not mention a turn, but
L-1 Old Main instead appears to instruct a traveler to continue walking
L-2 Distance Learning Center straight with the phrase “passing the intersection.” This is an
example of action inconsistency when actions from two
L-3 Animal Science different route descriptions result in inconsistency when the
L-4 Ray B. West statements are joined in a new route description.
L-99 Quad Sidewalk Intersection The path inference algorithm uses several heuristics for
handling action inconsistencies described in the next Section.


Table 2. Example Route Statements and their IDs for Descriptions A and B

Route ID Statement ID Statement Tags

A (R-2) S-2-1 Exit the Animal Science building doors on the south side. L-3
A(R-2) S-2-2 Walk straight until you find the sidewalk entrance to the Quad’s sidewalk.
A(R-2) S-2-3 Pass the main sidewalk intersection. L-99
A(R-2) S-2-4 Walk south until you detect a road and then carefully cross the street.
A(R-2) S-2-5 Continue to walk south until you find the doors to the Ray B. West building. L-4
B(R-1) S-1-1 Exit Old Main walking east. L-1
B(R-1) S-1-2 You will walk through the Quad, passing the intersection. L-99
B(R-1) S-1-3 Keep walking straight until you run into grass and then turn left, walking north.
B(R-1) S-1-4 Walk until you detect the bike racks on your right and then turn right.
B(R-1) S-1-5 Walk east until you find the stairs leading to the entrance to the distance learning center. L-2

Fig. (9). Digraph transformation of descriptions IDs from Tables 1 and 2.

Since these heuristics do not guarantee action consistency, it • S-3-2: You will walk through the Quad.
is assumed that all new route descriptions constructed by the
• S-3-3: Turn right at the sidewalk intersection.
system will be passed to the community for subsequent
collaborative editing to ensure route safety. Even if a • S-3-4: Walk south until you detect a road and then
stronger AI approach is taken to automate route editing, carefully cross the street.
subsequent crowdsourcing via collaborative editing will • S-3-5: Continue to walk south until you find the doors
likely be required. In the example above, knowledgeable to the Ray B. West building.
community users may edit the route description by deleting
the phrase “passing the intersection” and adding a new route 6.1. Inference Heuristics
statement “turn right at the sidewalk intersection” (S-3-3),
Choosing a path from which to build a new route
which results in the following edited route description: description requires that choices be made regarding which
• S-3-1: Exit Old Main walking east. path should be preferred over others. Three heuristics have


been designed to aid the system in choosing paths. Two of routes R-2 and R-4 that consists of route statements S-2-1, S-
the heuristics address action inconsistency and the third uses 2-2, S-2-3, and S-4-1. The upper path has two potential
the final length of the path as a metric. locations for action inconsistencies: the transition from S-1-1
to S-3-1 and the transition from S-3-1 to S-4-1. The lower
The first action inconsistency heuristic, H-1, chooses
paths put together from fewer route descriptions. The second path has only one potential action inconsistency - the
transition from S-2-3 to S-4-1. H-1 chooses the lower path,
heuristic, H-2, prefers to join route descriptions at their start
because it uses paths from two routes, R-2 and R-4, not three
and ending landmarks rather than at middle landmarks. Since
as the upper path.
action inconsistencies occur when a route statement from
one route description is followed by a route statement from H-2 selects complete route descriptions over partial ones.
another route description, H-1 minimizes the number of Fig. (11) demonstrates how H-2 is applied. The digraph in
changes from one route to another when searching for a path Fig. (11) has four routes R-11, R-12, R-13, and R-14.
in the digraph. Fig. (10) provides an example of how H-1 Suppose that a new route description from L-3 to L-429 is
works. The digraph in Fig. (10) has two possible paths for requested. There are two candidate paths. This first path
generating new routes from L-182 to L-428: the upper path starts with the first statement of R-12, S-12-1, and at L-99
from routes R-1, R-3, and R-4 that consist of route switches to R-11’s second statement S-11-2 and ends with S-
statements S-1-1, S-3-1, and S-4-1 and the lower path from 11-3. This route description, when constructed, results in a

Fig. (10). Deciding the path from L-182 to L-428.

Fig. (11). Deciding the path from L-3 to L-429.


three statement description (S-12-1, S-11-2, and S-11-3), The third heuristic, H-3, prefers shorter route
with one possible action inconsistency at L-99. The second descriptions over longer ones, because longer route
candidate path passes through L-823 using routes R-13 and descriptions are harder to remember. H-3 is used to build
R-14 that results in another three statement description S-13- new route descriptions in situations where H-1 and H-2 do
1, S-13-2, and S-14-1. This second route description also has not apply. Fig. (12) shows how H-3 is applied. The digraph
one possible action inconsistency at L-823. Both paths is built from four routes: R-23, R-24, R-25, and R-26. A new
generate route descriptions that have three route statements route description is requested from L-172 to L-311. There
and one possible action inconsistency. The difference are two candidate paths. The first one uses R-24 and R-25 to
between the two is where the action inconsistency occurs. In generate a new route description from three statements: S-
the first path, action inconsistency occurs in the middle of a 24-1, S-24-2, S-25-1. The second candidate path consists of
new route description that begins the first statement of R-12 R-23 and R-26 with two statements: S-23-1, S-26-1. Both
(S-12-1), but at L-99 switches to the second statement of R- descriptions are created from two routes and have one action
11 (S-11-2). Thus, the route change occurs in the middle of inconsistency each. Since H-1 and H-2, H-3 is applied. The
R-11 rather than at the end. On the other hand, in the second heuristic selects the second path (S-23-1, S-26-1), because it
candidate path, the switch from R-13 to R-14 occurs at L- contains fewer route statements: two instead of three.
823, i.e., at the end of R-13 and at the beginning of R-14.
The three heuristics assign different weights to edges in
Thus, the second path is chosen. the digraph. There are four weights that can be assigned to

Fig. (12). H-3 Application.

Fig. (13). W-RC assignment.


an edge: TO_LANDMARK_COST (W-TL), TO_STATE 8-7, it needs to determine the cost of the transition from S-8-
MENT_COST (W-TS), START_ROUTE_COST (W-SR), 7 to S-8-8 and S-11-6 via L-41. Since S-8-7 and S-8-8 are in
ROUTE_CHANGE_COST (W-RC). Edges are assigned the same route, the transition cost from S-8-7 to S-8-8 is set
these weights by CONNECT() shown in Fig. (7). W-TL is to W-TS. The transition from S-8-7 to S-11-6 via L-41
assigned to association edges, i.e., edges from statement creates a potential action inconsistency, because S-8-7 and S-
nodes to landmark nodes. Landmark nodes and their 11-6 are from different routes. Therefore, the weight of the
association edges indicate that two or more routes share a edge connecting S-8-7 to S-11-6 is set to W-RC. Similarly,
landmark so that the traveler is not affected by the presence transitioning from S-11-5 to S-11-6 retains the value W-TS
of the landmark because the landmark already exists. In our since both nodes are in the same route description. The
system, the value of W-TL is 0. This value is arbitrary and transition from S-11-5 to S-8-8 is set to W-RC, because S-
can be set to another real number so long as it is smaller than 11-5 and S-8-8 are from different routes.
the other weights. W-TS represents the cost of following a
The function MODIFY_DIGRAPH() in Fig. (16)
route statement, and is set to 1. W-SR is assigned to
modifies the digraph as shown in Figs. (13-15). The
precedence edges that connect a route’s starting landmark
modification ensures that there is higher cost for joining
and to the route’s first statement. Since new paths should be
route descriptions constructed from different routes, as
built using as few original routes as possible, W-SR can be
prescribed by H-2. In the modified digraph, mG, only two
set to any value greater 1. When a route description is landmark nodes, startLandmark and endLandmark, are
requested, the starting landmark of the requested route may
retained. It should be noted that, since mG is a clone of G, all
not be a starting landmark of any existing route. Therefore at
landmarks are still available for subsequent processing in G.
runtime, the weights of the edges leading from the requested
The edges of G are checked and, if they start or end at
route’s starting landmark are set to W-SR. W-RC, used in H-
startLandmark or endLandmark (lines 8 and 12), or connect
2, is different from the other costs in that it is determined by
two statements (line 10), then the original edge and its
the direction through which a node is reached. The value for original cost are retained in mG. Otherwise, as ensured at
W-RC is set to a value greater than W-SR.
line 14, the end node of the edge is a landmark different
Fig. (13) demonstrates a situation that requires W-RC to from startLandmark and endLandmark. The nodes to which
be calculated. Fig. (14) shows the same graph with the this landmark is connected and their route IDs are checked to
weights assigned by CONNECT() in the first pass through see if they are different from the current vertex v. If they
the graph in Fig. (13). Fig. (15) shows how the weighted have the same route IDs, signifying two route statements in
graph in Fig. (14) is modified during the transition cost the same route description, the new edge uses the original
analysis. The graph in Fig. (15) is a clone of the graph in Fig. cost of the edge from the landmark node to the second
(14). Therefore, the landmark L-41 removed from the cloned statement node. If two statement nodes come from different
graph is still retained in the original graph for subsequent route descriptions, the new edge is assigned the weight of
access. Specifically, when the search algorithm processes S- W-RC.

Fig. (14). Default weights set by CONNECT().


Fig. (15). Graph in Fig. (14) after Transition Cost Analysis.

Fig. (16). MODIFY_GRAPH applies H-2 heuristic to remove landmark nodes and adjust weights.


Fig. (17). Algorithm for inferring new route descriptions.

6.2. Path Inference Algorithm 7. EXAMPLES OF INFERRED ROUTE DESCRIPT-
The path inference algorithm consists of three basic IONS
steps. First, the digraph is cloned to handle H-2 This section contains several examples of inferred route
modifications as described in the previous section. Second, descriptions. The original route descriptions on which the
the modified digraph is processed to find the shortest path inferences were made were taken from our previous study of
between the start and end landmark nodes. Third, if a path is the self- sufficiency of verbal route directions for blind
found, a new route description is generated from it. The navigation with prior exposure in which we manually wrote
algorithm is shown in Fig. (17). The function a set of route directions for two routes on the USU campus,
INFER_PATH() takes as inputs the digraph G constructed machine-converted them into audio files, and placed these
by BUILD_GRAPH() (see Fig. 6) from the set of known files on a Nokia E70 cellular phone [21]. Three completely
route descriptions and two landmarks for a route description. blind USU students were then recruited to navigate the
INFER_PATH() is called after the original set of route routes using nothing but the instructions provided on the cell
descriptions has been searched and a route between phone. All three participants successfully navigated the
startLandmark and endLandmark has been found. The routes. Interested readers are referred to [21] for the details
modified graph constructed in line 1 is passed to the Dijkstra of navigation experiments. For the path inference examples
algorithm that finds the least cost path from startLandmark in this section, the original routes were tagged with the
to all nodes in mG. The last step in the process, landmarks from the USU landmark hierarchy by our
BUILD_DESCRIPTION() in Fig. (18), generates a route autotagging module described in Section 3. The inferred
description using the path found by Dijkstra's algorithm. route descriptions were built from the lowest cost paths in
Lines 12 and 13 ensure that the first and last route statements the digraph.
are tagged with the appropriate landmarks. The new
description is generated from the path from startLandmark 7.1. Inferred Route Description 1
and endLandmark. The texts of the statements from different This example demonstrates the concatenation of two
routes are not modified. If no path is found, then no source route descriptions. The first route description covers a
description is generated as evident from line 16. route, R-01, from the Animal Science building to the east

Fig. (18). Function BUILD_DESCRIPTION for building new route descriptions.


entrance to the Old Main on the USU campus. ground.
ROUTE_ID: R-01 6. Walk straight until you detect a wall.
START LANDMARK: Animal Science 7. Turn left, and look for stairs on your right.
END LANDMARK: Old Main East Entrance 8. Go up the stairs to the entrance. [Tag: Old Main East
STATEMENTS: Entrance]
1. Exit Animal Science at the south entrance. [Tag: 9. Enter the building from the east entrance. [Tag: Old
Animal Science] Main East Entrance]
2. Walk forward until you detect the entrance to the quad. 10. Walk forward until you find a second set of doors.
[Tag: Quad] 11. Go through them and walk down the long hall.
3. Keep walking south until you detect the sidewalk 12. Keep walking straight.
intersection in the center of the quad. [Tag: Center of the 13. You will eventually run in a wall at a t intersection.
Quad] 14. You should be able to detect some glass doors.
4. Turn right, walking west. 15. These are the doors to the President's Office. [Tag:
5. Keep walking straight until you detect bricks on the President's Office]
ground.
6. Walk straight until you detect a wall. This route description does not contain any action
7. Turn left, and look for stairs on your right. inconsistencies, because the phrases “entrance” and “east
8. Go up the stairs to the entrance. [Tag: Old Main East entrance” in statements 8 and 9, respectively, refer to the
Entrance] same landmark - Old Main East Entrance. The statements
where the two routes are joined, statements 8 and 9, form a
logical action sequence. Therefore, this description requires
The second source route description describes a route, R- little, if any, editing from the community during subsequent
02, from the entrance to Old Main to the entrance to the collaborative editing.
President's Office which is located inside the Old Main
building. 7.2. Inferred Route Description 2
ROUTE_ID: R-02 This example demonstrates how action inconsistencies
START LANDMARK: Old Main East Entrance arise in combining route descriptions. As with the previous
END LANDMARK: President's Office example, there are two source route descriptions. The first
STATEMENTS: description describes a route R-04 from Room 405, also
1. Enter the building from the east entrance. [Tag: Old known as the CSATL Lab, to Room 414, also known as the
Main East Entrance] Computer Science front office.
2. Walk forward until you find a second set of doors. ROUTE_ID: R-04
3. Go through them and walk down the long hall. START LANDMARK: Room 405
4. Keep walking straight. END LANDMARK: Room 414
5. You will eventually run in a wall at a t intersection. STATEMENTS:
6. You should be able to detect some glass doors. 1. Exit the CSATL Lab. [Tag: Room 405]
7. These are the doors to the President's Office. [Tag: 2. Turn left and walk down the hall, going east.
President's Office] 3. If you are trailing the right wall, you will pass an
entrance to a computer lab. [Tag: Room 406]
4. After the alcove, you will detect another hall on your
The path inference algorithm is asked to find a route right.
description for a route from Animal Science to the 5. Keep going straight.
President's Office. Since R-01 ends at the landmark “Old
6. You will pass a glass case on the right wall where they
Main East Entrance” and R-02 begins at the same landmark,
put announcements.
the algorithm concatenates these two routes into a new route
description for a new route R-03. 7. After the case is a large flat-screen TV hanging on the
wall.
ROUTE_ID: R-03 8. Immediately after the TV, you will find the door to the
START LANDMARK: Animal Science Computer Science front office on your right. [Tag: Room
END LANDMARK: President's Office 414]
STATEMENTS:
1. Exit Animal Science at the south entrance. [Tag:
Animal Science] The second source route description describes a route R-
2. Walk forward until you detect the entrance to the quad. 05 from the CS front office to the restrooms.
[Tag: Quad]
3. Keep walking south until you detect the sidewalk
intersection in the center of the quad. [Tag: Center of the ROUTE_ID: R-05
Quad] START LANDMARK: Room 414
4. Turn right, walking west. END LANDMARK: 4th floor restrooms
5. Keep walking straight until you detect bricks on the STATEMENTS:
1. When you exit the office, you are facing north. [Tag:


Room 414] the route from the Quick Stop, a small convenience store, to
2. Keep walking straight. the campus post office.
3. Immediately across from the door is the entrance to a ROUTE_ID: R-07
small hall. START LANDMARK: Quick Stop
4. Walk down this hall. END LANDMARK: Post Office
5. When you detect a T intersection, turn left. STATEMENTS:
6. The men's bathroom is on the left and the women's 1. You are standing with your back to the south entrance
restroom is on the right in a small alcove. [Tag: 4th floor to the Quick Stop. [Tag: Quick Stop]
restrooms]
2. Turn left so you are walking east.
3. On your left you will pass the ATM Machines which
The path inference algorithm constructs the following make distinctive sounds. [Tag: ATM Machines]
description for a route R-06. 4. On the left wall you will find a shelf or counter sticking
out from the wall.
ROUTE_ID: R-06 5. This is the counter to the Post Office. [Tag: Post
START LANDMARK: Room 405 Office]
END LANDMARK: 4th floor restrooms
STATEMENTS:
1. Exit the CSATL Lab. [Tag: Room 405] The second route description, R-08, describes how to get
2. Turn left and walk down the hall, going east. from the post office to Cafe Ibis, a small coffee shop inside
3. If you are trailing the right wall, you will pass an the TSC.
entrance to a computer lab. [Tag: Room 406] ROUTE_ID: R-08
4. After the alcove, you will detect another hall on your START LANDMARK: Post Office
right. END LANDMARK: Cafe Ibis
5. Keep going straight. STATEMENTS:
6. You will pass a glass case on the right wall where they 1. Turn to face east and start walking down the hall.
put announcements. 2. Continue walking east and passing the barber shop, and
7. After the case is a large flat-screen TV hanging on the the copy center as you walk down this long hall.
wall. 3. Towards the eastern end of the building, you will come
8. Immediately after the TV, you will find the door to the to a wide open area on your left.
Computer Science front office on your right. [Tag: Room 4. Turn left and walk a little north.
414] 5. Cafe Ibis is immediately on your left. [Tag: Cafe Ibis]
9. When you exit the office, you are facing north. [Tag:
Room 414]
10. Keep walking straight. The third route description, R-09, covers the route from
11. Immediately across from the door is the entrance to a Cafe Ibis to the patio outside the TSC, which has tables and
small hall. chairs where people can sit and relax.
12. Walk down this hall.
13. When you detect a T intersection, turn left. ROUTE_ID: R-09
14. The men's bathroom is on the left and the women's START LANDMARK: Cafe Ibis
restroom is on the right in a small alcove. [Tag: 4th floor END LANDMARK: TSC Outdoor Patio
restrooms] STATEMENTS:
1. After you get your coffee at Cafe Ibis, turn to face the
large hall, that is south. [Tag: Cafe Ibis]
This route contains an action inconsistency at lines 8 and 2. Walk across until you hit the hall's south wall which is
9 where the two routes were joined. Statement 9 from R-05 all windows.
describes an initial action of leaving the office. However, in 3. Turn left and walk east until you find doors on your
the context of the new route description, entering the front right.
office is completely unnecessary, because it is not a 4. Exit the door and go through a second door.
destination but a waypoint where the traveler must make a 5. Walk forward watching for the big concrete pillars.
turn. Therefore, during subsequent collaborative editing 6. This is the TSC Outdoor Patio and there tables where
statement 9 will likely be changed by the community. you can sit and relax. [Tag: TSC Outdoor Patio]
7.3. Inferred Route Description 3
This example shows how the number of routes is reduced At this point, a route description is requested from the
when a new route description is generated. All routes in this Quick Stop (the start landmark of R-07) to the TSC Outdoor
example come from the first floor of the Taggart Student Patio (the end landmark of R-09). The result description is a
Center (TSC), a large building on the USU campus housing concatenation of the three above routes. The path consists of
shops, restaurants, student services, and administrative a description from the Quick Stop to the Post Office and
offices. The first source route description, R-07, describes from Cafe Ibis to the TSC Outdoor Patio.


ROUTE_ID: R-10 conveying space to the extent that space is schematized
START LANDMARK: Quick Stop similarly in language and cognition. To gain evidence to
END LANDMARK: TSC Outdoor Patio support their conjecture, they analyzed schematization of
STATEMENTS: routes conveyed in sketch maps or directions. Allen [36]
1. You are standing with your back to the south entrance presented an experimental investigation of several principle-
to the Quick Stop. [Tag: Quick Stop] based practices in communicating route knowledge. His
2. Turn left so you are walking east. results showed that following route directions is facilitated
3. On your left you will pass the ATM Machines which by directions presented in correct temporal-spatial order
where information is concentrated in statements describing
make distinctive sounds. [Tag: ATM Machines]
choice points. Kuipers [37] presented the Spatial Semantic
4. On the left wall you will find a shelf or counter sticking
Hierarchy (SSH), a formal model of knowledge of large-
out from the wall.
scale space. The SSH consisted of multiple representations
5. This is the counter to the Post Office. [Tag: Post that interact with each other. The SSH can represent states of
Office] partial knowledge and enable human and robotic agents to
6. Turn to face east and start walking down the hall. handle navigation uncertainty.
7. Continue walking east and passing the barber shop, and
the copy center as you walk down this long hall. The problem of path inference draws on a large body of
8. Towards the eastern end of the building, you will come research on formal inference models in AI. Many formal
to a wide open area on your left. inference models rely on a combination of semantic
networks and first-order predicate calculus (FOPC), which
9. Turn left and walk a little north.
resulted in the development of semantic network languages
10. Cafe Ibis is immediately on your left. [Tag: Cafe Ibis]
such as KRL [38] and KL-ONE [39]. It was subsequently
11. After you get your coffee at Cafe Ibis, turn to face the
concluded that many such efforts were restricted variants of
large hall, that is south. [Tag: Cafe Ibis] FOPC. Russell and Norvig [40] give an in-depth historical
12. Walk across until you hit the hall's south wall which analysis of these representational efforts.
is all windows.
13. Turn left and walk east until you find doors on your As the amount of electronic data began to increase
right. rapidly after the advent of the Internet in the early 1990’s,
14. Exit the door and go through a second door. many NLP researchers and practitioners discovered that
15. Walk forward watching out for the big concrete knowledge-intensive approaches do not scale as well as
approaches based on statistical analysis, probability theory
pillars.
and shallow knowledge [41]. Shallow knowledge was a
16. This is the TSC Outdoor Patio and there tables where
basic IE principle. The Fast Reading, Understanding and
you can sit and relax. [Tag: TSC Outdoor Patio]
Memory (FRUMP) [42] system was one of the first IE
systems that put to use the power of shallow knowledge. The
system used simple rules, called sketchy scripts, to extract
8. DISCUSSION
information from newswire stories, e.g. car accidents.
Our present investigation contributes to the large body of Research and development on IE was supported through a
research on verbal route directions for blind and VI travelers. series of Message Understanding Conferences (MUCs) that
Gaunet [30] proposed several verbal guidance rules for focused on various domains such as joint ventures and
localized wayfinding aids that can be used by blind microelectronics and satellite launch reports [45]. The
pedestrians in urban areas. Tuti et al. [31] investigated the objective of this conference is to automate mundane tasks
relationship between lack of visual experience and the ability performed by human analysts reading large quantities of
to create survey-type spatial inferential representations. Their electronic texts. The proliferation of web documents
investigation concluded that visual experience may not be underscored the need for IE solutions that are low cost,
necessary for the creation of such representations. Franklin flexibility, and easy adaptation to new domains, which, as
[32] argued that, in the absence of perceptual experience, some researchers have argued (http://en.wikipedia.org/wiki/
language is used as a means of constructing and conveying Information_extraction), the MUC failed to address.
cognitive maps. She proposed several mechanisms for the The path inference algorithm presented in this article can
linguistic production of cognitive maps such as selection of also be categorized as a shallow knowledge algorithm. Our
important communication elements, temporal structuring of findings should be interpreted with caution, because we have
the elements, reference frame and perspective selection, and not yet tested our path inference algorithm on a large corpus
verbal regularization of spatial relations. Couclelis [33] of route directions (due to its unavailability) from a VGI site.
proposed a model of the cognitive mechanism underlie Nonetheless, our findings may serve as a sound basis for
verbal direction giving. The fundamental hypothesis was that future research and development efforts in automated path
that both cognitive maps and direction giving discourses are inference. JAPE rules are used to autotag landmarks in free-
generated by one unifying mental model, which is itself text route descriptions so that route descriptions can be
organized by more primitive kinesthetic image schemas and subsequently transformed into directed graphs. The
basic-level categories. Dennis [34] presented a cognitive generated graphs are searched for new paths not originally
approach to spatial discourse production and distinguishes present in the original route descriptions.
two types of product categories: action prescriptions and
landmark references. Tversky and Lee [35] studied how Our approach differs from AI intensive approaches (e.g.,
space structures language and propose the Schematization [39]) or IE approaches of many MUC systems [41, 45] in
Similarity Conjecture which states language is successful in that our objective is not to automate inference or knowledge


extraction. Since our long-term goal is to enhance travel [8] Langheinrich M. RFID and Privacy. In: Petkovic M, Jonker W,
independence for VI and blind travelers, travel safety on Eds. Security, privacy and trust in modern data management.
Secaucus, NJ, USA: Springer-Verlag 2009; pp. 433-50.
route is the top concern. Consequently, the results of [9] Nicholson J, Kulyukin V, Coster D. ShopTalk: Independent blind
autotagging and path inference are assumed to undergo shopping through verbal route directions and barcode scans. Open
Wikipedia-style collaborative human editing. There is a Rehabil J 2009; 2: 11-23.
growing body of research that shows that crowdsourcing [10] McQuistion L. Rehabilitation engineering: ergonomics for one.
Ergon Des 1993; 1(1): 9-10.
provides a reasonable alternative to automated annotation [11] LaMarca A, Chawathe Y, Consolvo S, et al. Placelab: device
tools (e.g., [43, 44]). It is argued that automated annotation positioning using radio beacons in the wild. In: Gellersen HW,
tools are expensive, because they require a great deal of Want R, Schmidt A, Eds. Third International Conference on
skilled labor both in development and maintenance and are Pervasive Computing (PERVASIVE 2005). 2005 May 8-13;
still error-prone. It is further argued that crowdsourcing Munich, Germany: Springer 2005; pp. 116-33.
[12] Golledge RG. Geography and the disabled: A survey with special
annotation tasks on commercial services such as Amazon reference to vision impaired and blind populations. Trans Inst Br
Mechanical Turk (https://www.mturk.com) can be used to Geogr 1993; 18(1): 63-85.
reliably annotate large bodies of electronic data. [13] Golledge RG. Human wayfinding and cognitive maps. In: Golledge
RG, Eds. Wayfinding behavior: cognitive mapping and other
Two potential problems with using commercial spatial processes. Baltimore, Maryland: John Hopkins University
annotation services for annotating landmarks in route Press 1999; pp. 5-45.
directions for VI and blind travelers are volume traffic and [14] Golledge RG, Klatzky RL, Loomis JM. Cognitive mapping and
wayfinding by adults without vision. In: Portugali J, Ed. The
payment. Volume traffic, an important condition for speed construction of cognitive maps. Dordrecht, Netherlands: Kluwer
and reliability, may not be feasible for route auto-tagging for Academic Publishers 1996; pp. 215-46.
VI and blind travelers due to their small numbers. Payment [15] Nicholson J, Kulyukin V. ShopTalk: independent blind shopping =
may be problematic for the same reason. Volunteered verbal route directions + barcode scans. Proceedings of 30th
geographic information (VGI) [19, 20] may present at least Annual Conference of the Rehabilitation Engineering and Assistive
Technology Society of North America (RESNA 2007); 2007 Jun
partial solutions to both problems. Since VGI is a bottom-up 15-19; Phoenix, AZ. Arlington, Virginia: RESNA Press 2007.
volunteer movement organized around specific web sites, [16] Nicholson J, Kulyukin V, Marston J. Building route-based maps for
local communities may be more interested and motivated to the visually impaired from natural language route descriptions.
enhance travel accessibility of specific geographic areas for Proceedings of the 24th International Cartographic Conference
(ICC 2009); 2009 Nov 15-21; Santiago, Chile: International
VI and blind travelers. Our path inference algorithm, Cartographic Association 2009.
presented in this article, is designed to be part of a VGI [17] Nicholson J, Kulyukin V. CRISS: A collaborative route
website’s back-end software that helps the website’s information sharing system for visually impaired travelers. Cruz-
community process and assemble route directions. Cunha MM, Oliveira EF, Tavares AJ, Ferreira LG, Eds. Handbook
of Research on Social Dimensions of Semantic Technologies and
ACKNOWLEDGEMENT Web Services (Volume II). Hershey, PA, USA: IGI Global 2009;
pp. 720-41.
Declared none. [18] Passini R, Proulx G. Wayfinding without vision: An experiment
with congenitally totally blind people. Environ Behav 1998; 20(2):
CONFLICT OF INTEREST 227-52.
[19] Goodchild MF. Citizens as Voluntary Sensors: Spatial Data
The authors confirm that this article content has no Infrastructure in the World of Web 2.0. Int J Spat Data
conflicts of interest. Infrastructures Res 2007; 2: 24-32.
[20] Haklay M, Weber P. OpenStreetMap: user-generated street maps.
REFERENCES IEEE Pervasive Comput 2008; 7(4): 12-8.
[21] Kutiyanawala A, Kulyukin V, Bryce D. On self-sufficiency of
[1] Sendero Group, LLC. Accessible GPS Products [Online]. 2012 verbal route directions for blind navigation with prior exposure.
June 30, [cited 2012]. Available from: http://www.senderogroup. Proceedings of the 25th Annual International Technology and
com/products/GPS/allgps.htm Persons with Disabilities Conference (CSUN 2010); 2010 Mar 22-
[2] Kulyukin V. On Overcoming longitudinal and latitudinal drift in 27; San Diego, CA: California State University, Northridge Center
GPS-based localization outdoors. In: Anson D, Ed. 28th Annual on Disabilities 2010.
Conference of the Rehabilitation Engineering and Assistive [22] Gaunet F, Briffault X. Exploring the functional specifications of a
Technology Society of North America (RESNA 2005). 2005 June localized wayfinding verbal aid for blind pedestrians: simple and
23-27; Atlanta, Georgia. Arlington, Virginia: RESNA Press 2005. structured urban areas. Int J Hum Comput Interact 2005; 20(3):
[3] Cheng YC, Chawathe Y, LaMarca A, Krumm J. Accuracy 267-14.
characterization for metropolitan-scale Wi-Fi localization. In: Shin [23] Nicholson J. Generation and analysis of verbal route directions for
KG, Kotz D, Noble BD, Eds. 3rd international conference on blind navigation. PhD thesis. Logan, UT, USA: Utah State
Mobile systems, applications, and services (MobiSys '05); 2005 University 2010.
June 6-8; Seattle, Washington. New York, New York: ACM 2005; [24] Cunningham H, Maynard D, Bontcheva K, Tablan V. GATE: A
pp. 233-45. framework and graphical development environment for robust NLP
[4] Seshadri V, Zaruba GV, Huber M. A Bayesian sampling approach tools and applications. Proceedings of the 40th Anniversary
to in-door localization of wireless devices using received signal Meeting of the Association for Computational Linguistics
strength indication. Third IEEE International Conference on
(ACL’02); 2002 Jul 7-12; Philadelphia, PA; Stroudsburg, PA,
Pervasive Computing and Communications (PerCom 2005); 2005 USA: Association for Computational Linguistics 2002; pp. 168-75.
March 8-12; Kauai Island, Hawaii. Los Alamitos, California: IEEE
[25] Cunningham H, Maynard D, Bontcheva K, et al. The GATE User
Computer Society, 2005; pp. 75-84. Guide [Internet]. Sheffield, England: University of Sheffield,
[5] Crandall WF, Brabyn JA, Bentzen B, Myers L. Remote infrared
Department of Computer Science; 2007 [updated 2012 Jul 4; cited
signage evaluation for transit stations and intersections. J Rehabil 2012 Jun 30]. Available from: http://gate.ac.uk/sale/tao
Res Dev 1999; 36(4): 341-55.
[26] Finkel JR, Grenager T, Manning C. Incorporating non-local
[6] Roussos G, Kostakos M. RFID in pervasive computing: state-of- information into information extraction systems by gibbs sampling.
the-art and outlook. Pervasive Mob Comput 2009; 5(1): 110-31.
Proceedings of the 43rd Annual Meeting of the Association for
[7] Roussos G. Enabling RFID in retail. Computer 2006; 39(3): 25-30. Computational Linguistics (ACL 2005); 2005 Jun 25-30; Ann


Arbor, MI; Stroudsburg, PA, USA: Association for Computational [35] Tversky B, Lee PU. How space structures language. In: Freksa C,
Linguistics 2005; pp. 363-70. Habel C, Wender KF, Eds. Spatial cognition: an interdisciplinary
[27] Ratinov L, Roth D. Design challenges and misconceptsc in named approach to representation and processing of spatial knowledge.
entity recognition. Proceedings of the 13th Conference on Heidelberg, and Berlin: Springer 1998; pp. 157-75.
Computational Natural Language Learning; 2009 Jun 4-5; Boulder, [36] Allen G. Principles and practices for communicating route
CO; Stroudsburg, PA, USA: Association for Computational knowledge. Appl Cogn Psychol 2000; 14(4): 333-59.
Linguistics 2009; pp. 147-55. [37] Kuipers B. The spatial semantic hierarchy. Artif Intell 2000; 119(1-
[28] Li Y, Bontcheva K, Cunningham H. SVM-based learning system 2): 191-233.
for information extraction. In: Winkler J, Lawrence N, Niranjan M, [38] Bobrow DG, Winograd T. An overview of KRL, a knowledge
Eds. Deterministic and Statistical Methods in Machine Learning. representation. Cogn Sci 1977; 1(1): 3-46.
Heidelberg, Berlin: Springer 2002; pp. 319-39. [39] Brachman RJ, Schmolze JG. An overview of the KL-ONE
[29] Appelt DE, Onyshkevych B. The Common Pattern Specification knowledge representation system. Cogn Sci 1985; 9(2): 171-216.
Language. In: Perose D, Alan M, Eds. Proceedings of a workshop [40] Russell S, Norvig P. artificial intelligence: a modern approach.
on held at Baltimore, Maryland (TIPSTER ’98); 1998 Oct 13-15; Saddle River, NJ, USA: Prentice-Hall 1995.
Baltimore, MD; Stroudsburg, PA, USA: Association for [41] Cowie J, Lehnert W. Information Extraction. Commun ACM 1996;
Computational Linguistics 1998; pp. 23-30. 39(1): 80-91.
[30] Gaunet F. Verbal Guidance Rules for a Localized Wayfinding Aid [42] de Jong G. Frump… Frump… Frump… ACM SIGART Bull 1997;
Intended for Blind Pedestrians in Urban Areas. Univers Access Inf (61): 54-5.
Soc 2006; 4(4): 338-53. [43] Sorokin A, Forsyth, D. Utility data annotation with amazon
[31] Tinti C, Adenzato M, Tamieto M, Cornoldi C. Visual experience is mechanical turk. In: Computer vision and pattern recognition
not necessary for efficient survey spatial cognition: evidence from workshops, 2008. CVPRW '08. IEEE computer society conference
blindness. Q J Exp Psychol (Hove) 2006; 59(7): 1306-28. on computer vision and pattern recognition; 2008 Jun 23-28;
[32] Franklin N. Language as a means of constructing and conveying Anchorage, AL: IEEE 2008; pp. 1-8.
cognitive maps. In: Portugali J, Ed. The construction of cognitive [44] Hsueh P, Melville P, Sindhwani V. Data quality from
maps. Dordrecht, Netherlands: Kluwer Academic Publishers 1996; crowdsourcing: a study of annotation selection criteria. In:
pp. 275-95. Proceedings of the NAACL HLT 2009 Workshop on Active
[33] Couclelis H. Verbal directions for wayfinding: space, cognitive, Learning for Natural Language Processing; 2009 Jun 5; Boulder,
and Language. In: Portugali J, Ed. The construction of cognitive CO. Stroudsburg, PA: Association for Computational Linguistics
maps. Dordrecht, Netherlands: Kluwer Academic Publishers 1996; 2009; pp. 27-35.
pp. 133-53. [45] Grishman R, Sundheim B. Message understanding conference - 6:
[34] Denis M. The Description of Routes: A cognitive approach to the A Brief History. COLING '96: Proceedings of the 16th conference
production of spatial discourse. Curr Psychol Cogn 1997; 16(4): on Computational linguistics - Volume 1; 1996 Aug 5-9;
409-58. Copenhagen, Denmark; Stroudsburg, PA: Association for
Computational Linguistics 1996; pp. 466-71.

Received: April 25, 2012 Revised: June 5, 2012 Accepted: June 26, 2012

© Kulyukin and Nicholson; Licensee Bentham Open.
This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License (http: //creativecommons.org/licenses/by-
nc/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.

Toward Blind Travel Support through Verbal Route Directions: A Path Inference Algorithm for Inferring New Route Descriptions from Existing Route Descriptions

More Related Content

Similar to Toward Blind Travel Support through Verbal Route Directions: A Path Inference Algorithm for Inferring New Route Descriptions from Existing Route Descriptions (20)

More from Vladimir Kulyukin (20)

Recently uploaded (20)

Toward Blind Travel Support through Verbal Route Directions: A Path Inference Algorithm for Inferring New Route Descriptions from Existing Route Descriptions