Why Search is the Problem
Download as PPT, PDF3 likes554 views
The document discusses issues with the current state of search and APIs. It argues that search has become too user-centric, requiring human disambiguation of results. APIs are also primarily designed for user consumption, not machine-to-machine communication. This leads to problems with multiple representations of the same real-world entities across different data sources. The document advocates for more structured data and connections between data to allow for straight-through processing between machines without human intervention.
![“ You are permitted to use the Services only for the purpose of incorporating and displaying results […] as part of a Search Product deployed on Your Web site” Yahoo! BOSS Services Terms of Use http://www.flickr.com/photos/auntiep/17135231/](https://image.slidesharecdn.com/bellsemtech2011-110606232617-phpapp01/85/Why-Search-is-the-Problem-8-320.jpg)
![You must not use or display the Content without a corresponding Google map You are not permitted to use or provide any part of the Service or Content […] in an API that you offer to others http://www.flickr.com/photos/auntiep/17135231/ Google Maps/Places Terms of Use](https://image.slidesharecdn.com/bellsemtech2011-110606232617-phpapp01/85/Why-Search-is-the-Problem-11-320.jpg)
![http://continuations.com/post/4365211963/the-web-stp-challenge-making-apis-useful We need more STP [Straight Through Processing] for the web so that we have fewer stove pipe services and can move to a seamless web instead. The obstacle is no longer a lack of APIs […] the problem is a lack of data mapping/unification services. - Albert Wenger http://twitter.com/#!/cdixon/status/49906284492881920](https://image.slidesharecdn.com/bellsemtech2011-110606232617-phpapp01/85/Why-Search-is-the-Problem-13-320.jpg)
![enable publishers to give us hints about what things they are describing on their sites… markup [will] amplify the value [webmasters ]receive in return improve how their sites appear in major search engines… powering richer search results and new kinds of applications. improve the search experience… alignment between search and our Web of Objects program](https://image.slidesharecdn.com/bellsemtech2011-110606232617-phpapp01/85/Why-Search-is-the-Problem-24-320.jpg)
Why Search is the Problem
- 1. Search is the Problem Tyler Bell @twbell Hobbling a data-centric Internet
- 2. Too User Centric Inserting people between data and its interpretation APIs designed, and enforced, for user consumption only Any Ten Answers Idea of correspondance remains unfamiliar Creating explicit relationships on-the-fly remains rare Tags Over Semantics Tokens are easy, but designed for the search use case Changing the state of our knowledge requires change in tags Where things are going Changing role of APIs: Structured data easier to access via page (FB, Best Buy, etc.) Decline of entity search ‘gatekeeper APIs’ Increase in machine to machine services APIs designed, and enforced, for user consumption only Hidden Slide Overview
- 3. The Problem: Search is now the norm: human disambiguation is demanded
- 7. The Artefact: APIs are designed for user consumption only
- 8. “ You are permitted to use the Services only for the purpose of incorporating and displaying results […] as part of a Search Product deployed on Your Web site” Yahoo! BOSS Services Terms of Use http://www.flickr.com/photos/auntiep/17135231/
- 10. Employs probability matrices to disambiguate placename intent Contains cross-references to entities in other namespaces Designed for toponym and business name disambiguation ‘ Venue harmonization’ with select data partners ID cross-reference (one-way street only: ID tweets)
- 11. You must not use or display the Content without a corresponding Google map You are not permitted to use or provide any part of the Service or Content […] in an API that you offer to others http://www.flickr.com/photos/auntiep/17135231/ Google Maps/Places Terms of Use
- 13. http://continuations.com/post/4365211963/the-web-stp-challenge-making-apis-useful We need more STP [Straight Through Processing] for the web so that we have fewer stove pipe services and can move to a seamless web instead. The obstacle is no longer a lack of APIs […] the problem is a lack of data mapping/unification services. - Albert Wenger http://twitter.com/#!/cdixon/status/49906284492881920
- 14. The Problem: Multiple electronic representations of one conceptual entity
- 16. Too much of this… Not enough of this 21c7c504-537a-47d6-a4cf-e260ccb6620d
- 17. “ With a single click you can recommend that raincoat, news article or favorite sci-fi movie to friends, contacts and the rest of the world”
- 19. “ With a single click you can recommend that Webpage to friends, contacts and the rest of the world”
- 20. http://tripletalk.wordpress.com/2011/01/25/rdfa-deployment-across-the-web/ Peter Mika, Jan 2011
- 21. Webpage URLs are Entity URIs Identifiers for people, places, things http://developers.facebook.com/docs/opengraph/
- 24. enable publishers to give us hints about what things they are describing on their sites… markup [will] amplify the value [webmasters ]receive in return improve how their sites appear in major search engines… powering richer search results and new kinds of applications. improve the search experience… alignment between search and our Web of Objects program
- 26. 17.5m entities pointing to over… 1.5b references found across… 4.7m domains US Local Dataset
- 27. Datawire http://www.flickr.com/photos/tigerplish/250836258/ TL;DR: Search: human disambiguation is expected Few inputs leads to ‘pull’, not ‘push’ Plurality of content is a real bugger Structured content will do more than improve the look of search results Increased recognition of machine-to-machine APIs The socially networked world demands understanding across caissons The Good News:
Editor's Notes
- #5: A single, simple input, augmented over the last decade with geographic, behavioral, and demographic attributes. We are too mired in the explicit query: we now have context, sensors. More push than pull.
- #6: All search results require human disambiguation – this is the problem
- #7: Lucene and Solr have given us excellent tools to improving how we access data. This is good, but creating genuine data-centric intelligence demands we remove the human from the equasion.
- #9: Yahoo BOSS TOS conceived of around the idea of search-and-display to human eyes. http://info.yahoo.com/legal/us/yahoo/search/bosstos/bosstos-2317.html
- #10: Every one of these Local APIs relies on a generic search function and returns multiple results. Placecast and Geoplanet are the two exceptions that attempt to disambiguate. Others provide corss-references to other namespaces.
- #12: Google Maps API also implies entirely visual ( i.e. human-readable) engagement. Restrictions on data stored suggest machine consumption is implicitly proscribed. https://code.google.com/apis/console
- #13: Zach Klein occasionally tweets commentary on the tech sector, all of it spot-on. http://twitter.com/#!/zachklein/status/55986694289227776
- #16: ‘ The Counter’ restaurant in San Jose, CA, represented electronically in 8 different places
- #18: Social engagement is with the web page, not the underlying entity http://googleblog.blogspot.com/2011/06/1-button-for-websites-recommend-content.html
- #19: http://googleblog.blogspot.com/2011/06/1-button-for-websites-recommend-content.html
- #20: http://googleblog.blogspot.com/2011/06/1-button-for-websites-recommend-content.html
- #21: “ This chart shows the deployment of certain microformats and RDFa markup on the Web, as percentage of all web pages, based on an analysis of 12 billion web pages indexed by Yahoo! Search” http://tripletalk.wordpress.com/2011/01/25/rdfa-deployment-across-the-web/
- #22: The Semantic Web community appears to hold a harmful obsession with formats and protocols. Almost any URL can serve as a URI, and be dereferenced to its attributes which may be structured in hcard, og, microdata, etc.
- #23: Yelp pages are marked up in both og and hcard. All good, and equally consumable.
- #25: The PR on schema.org focuses on Search, but each of these orgs is certainly working on an entity-mapping product, of which only Yahoo makes reference. http://www.bing.com/community/site_blogs/b/search/archive/2011/06/02/bing-google-and-yahoo-unite-to-build-the-web-of-objects.aspx http://googleblog.blogspot.com/2011/06/introducing-schemaorg-search-engines.html http://www.ysearchblog.com/2011/06/02/introducing-schema-org-a-collaboration-on-structured-data/
- #28: Wires can connect
- #29: Fin.