Solr 101

Editor's Notes

• #2: \n
• #3: Intros c= me & solr\n
• #4: Who – swiss, sébastien muller, ex solr newbie, 1 yr w/ Solr almost daily, several projects\nWhat – work for findwise Findwizard, information access consultant….. Enterprise search!\nWhere – Oslo for a year\nWhy – Oslo Solr Meetup community semi regular meetings at a pub in oslo\n\nINTRODUCTORY TALK\n
• #5: Internally and/or externally, both for finding information or finding who has the information…. Ecommerce fail w/out search, cant find what you want to buy = no sale\nA lot of which is unstructured\nAccording to research performed by google approx. 85% of organisations can barely access less than 50% of the data they produce\n
• #6: \n
• #7: General description – the “sales pitch”\n\nWeb based app – runs in a servlet container and is deployed as a java war, works with all major application servers such as Tomcat or Jetty\n
• #8: No point in building when there’s open source and licensed options readily available\nOpen source = free but might end up spending a lot to get it to do what you want – no vendor lock in, complete customisability\nLicensed = expensive and likely to spend a lot still\n
• #9: Although it being open source allows for a very low barrier for adoption there is no Service Level Agreement with an open source community\nOpen Source community based project likely to yield better long term QA testing, more personally invested in the quality of the project, but life > *\n\nIssue tracking is public\nAccess to source code – but the documentation isn’t necessarily comprehensive… google :D\n
• #10: Inverted index – like a book = list of keywords paired with location -> makes for v. fast queries rather than searching through documents for specific terms\n\nDocument = collection of fields with optional boosting values book…. Page… database entry etcrepresented by a single result or hit\n\nSingle/multi valued\n\nStored = original pre-analysis value stored and returnable by queries, necessary for some features, increases index size store and retrieve, unless indexed too\n\nIndexed = searchable and facetable, unless stored will not be returned in a search\n\nTokenization = breaking up text sequences, filtering and trasnforming to generate “terms” that are tied to a specific field\n\nRestricts the search space by creating subset of indexed documents against which queries can be made\n\nContributes to relevance calculations in query time, can be customised\n
• #11: \n
• #12: Schema = define data and field types\n\nSolrconfig = search components, replication, request handlers etc\n\nDEMO schema/solrconfig in notepad++ and DEMO POSTing from netbeans/browser\n
• #13: Sort on relevance score, value of fields….\n\nIf there is a tie docs are sorted by date added (indexed time)\n\nMore shiney examples to follow\n
• #14: Sort on relevance score, value of fields….\n\nIf there is a tie docs are sorted by date added (indexed time)\n\nMore shiney examples to follow\n
• #15: Sort on relevance score, value of fields….\n\nIf there is a tie docs are sorted by date added (indexed time)\n\nMore shiney examples to follow\n
• #16: Single solr instance with separate configurations (schema and config) and indexes while maintaining the convinience of unified administration\n\nEasy to add new cores or even replace cores with each other\n\nSTATUS, create, swap, unload, alias rename\n\nAtomically swaps the names used to access two existing cores. This can be useful for replacing a "live" core with an "ondeck" core, and keeping the old "live" core running in case you decide to roll-back.\n
• #17: Basic – good starting point, fine for a small index with few updates and low query load as all updates will slow down querying\n\nMaster/slave – indexing on one, querying on the other, all replications slow down indexing, can modify replication interval – improves query speed/qps\n\n\n1:N – more qps, no more query speed necessarily -> both require index to remain on 1 machine\n
• #18: Updates go to one of N machines – unique key field must be unique across all shards – couple of features aren’t supported eg. More like this and joins\n\nMakes it easier to rebalance index operations across more servers\n\nshards=solr1:8983/solr,solr2:8983/solr&indent=true&q=ipod+solr -> shards parameter syntax -> can be added to a requestHandler specifically for shards\n
• #19: Norwegian Bank and Norwegian based e-commerce group of sites\n
• #20: One bank portal of about 1.5k docs, c. 50% were duplicates\n\nGroup publications made globally available via CMS but individual banks are under no obligation to publish articles and there’s no indication as to whether they had or not\n
• #21: Norwegian Bank and Norwegian based e-commerce group of sites\n
• #22: One bank portal of about 1.5k docs, c. 50% were duplicates\n\nGroup publications made globally available via CMS but individual banks are under no obligation to publish articles and there’s no indication as to whether they had or not\n
• #23: Semantic FAIL\n
• #24: \n
• #25: \n
• #26: One bank portal of about 1.5k docs, c. 50% were duplicates\n\nGroup publications made globally available via CMS but individual banks are under no obligation to publish articles and there’s no indication as to whether they had or not\n
• #27: \n