20100622 e z_find_slides_gig_v2.1

eZ Find 2.2 Customization & Advanced development Gilles Guirand – Technical Director at Kaliop eZ Conference 2010 June 24th Platinum Partner

eZ Find 2.2 Customization & Advanced development eZ Conference 2010 June 24th Summary : Chapter 1 : Introduction about eZ Find Chapter 2 : Speed-up eZ Find development tasks Chapter 3 : Fields & Datatypes in Solr and eZ Find Chapter 4 : Indexing additional fields in Solr Chapter 5 : Enhance eZ Find using the Solr syntax Chapter 6 : Conclusion

eZ Find 2.2 Customization & Advanced development Chapter 1 : Introduction about eZ Find How eZ Find work eZ Publish HTTP REST Add content Update content Delete content Move content …. Querying, faceting, … Templating

eZ Find 2.2 Customization & Advanced development Chapter 1 : Introduction about eZ Find How eZ Find pushes content from eZ Publish to Solr CRON eZContentOperationCollection PHP Class registerSearchObject DelayedIndexing ON DelayedIndexing OFF eZ Setting eZ Setting insertion of the objectID in the ' ezpending_actions ' table for later indexing Indexcontent.php eZSearch PHP Class addObject($object, $needcommit) eZSolr PHP Class addObject($contentObject, $commit = true) addDocs( … ) -> cURL -> eZSolrBase PHP Class SearchEngine=ezsolr eZ Setting (plug-in) php extension/ezfind/bin/php/updatesearchindexsolr.php --siteaccess=monsiteaccess

eZ Find 2.2 Customization & Advanced development eZ Conference 2010 June 24th Summary : Chapter 1 : Introduction about eZ Find Chapter 2 : Speed-up eZ Find development tasks Chapter 3 : Fileds & Datatypes in Solr and eZ Find Chapter 4 : Indexing additional fields in Solr Chapter 5 : Enhance eZ Find using the Solr syntax Chapter 6 : Conclusion

eZ Find 2.2 Customization & Advanced development Chapter 2 : Speed-up eZ Find development tasks Re-index some content after a modification Re-indexing all content to merely test one single, minor modification's impact on one's application can quickly become a drawn-out process. A few, hidden, life-saver arguments exist in the concerned script /bin/php/updatesearchindexsolr.php , allowing for pointing to: a root node an offset a limit It is mandatory to use the 3 parameters simultaneously : phpextension/ezfind/bin/php/updatesearchindexsolr.php --siteaccess=mysiteaccess --topNodeID=2546 --offset=0 --limit=10

eZ Find 2.2 Customization & Advanced development Chapter 2 : Speed-up eZ Find development tasks Check what Solr actually indexed http://localhost:8983/solr/admin/ Fieldname Value

eZ Find 2.2 Customization & Advanced development Chapter 2 : Speed-up eZ Find development tasks Debugging directly in Solr Copy MESSAGE Active console opened INFO: [] webapp=/solr path=/select params={ ... MESSAGE ... } status=400 QTime=5 Past MESSAGE : http://localhost:8983/solr/select/? MESSAGE The obtained result is the exact output sent by Solr to eZ Find before transformation and display of the results. Using this trick is pretty useful when debugging, by, for example, directly manipulating the messages to retrieve the expected result.

eZ Find 2.2 Customization & Advanced development Chapter 3 : Fields & Datatypes in Solr and eZ Find The Solr-side naming of fields attr_[ contentattributename ]_[ contentattributetype ] Note the absence of the content class identifier , opening for nice perspectives like filtering on several content classes having identical names Attribute Name : Title | Attribute datatype : ezstring | Class Name : Article Solr Field 1 : attr_ title _ s Solr Field 2 : attr_ title _ t DatatypeMap[ezstring]= text DatatypeMapSort[ezstring]= string DatatypeMapFacet[] DatatypeMapFilter[] ezfind.ini Indexing process eZ Publish

eZ Find 2.2 Customization & Advanced development Chapter 3 : Fields & Datatypes in Solr and eZ Find The Solr-side naming of fields Metadata fields mapping Fetching process ‘ article /title:test’ Filtering by : ‘ article /title’ Sorting by : eZ find syntax Solr syntax meta_contentclass_id:12 AND attr_title_t:test attr_title_s meta_[metadataname]_[metadatatype] meta_class_identifier_si meta_path_string_s meta_owner_id_si … eZ Publish

eZ Find 2.2 Customization & Advanced development Chapter 3 : Fields & Datatypes in Solr and eZ Find Sub-attributes names (not used by default) Sub-attributes usage ? subattr_[ contentattributename ]-[ contentsubattributename ]_[contentsubattributetype] Example : subattr_relatedimage-alttext_s Index your images matadata fields Natively, the subattribute concept is not or little used, because the standard state and features of eZ Publish does not require it massively. It however is here as an opening for advanced usages, and it is, for instance, a great tool to extend eZ Find and index additional fields . Example : subattr_relatedobject-title_s Index all attributes of an object’s related objects

eZ Find 2.2 Customization & Advanced development Chapter 3 : Fields & Datatypes in Solr and eZ Find Object relation datatype mapping http://projects.ez.no/ezfsolrdocumentfieldobjectrelation Check my contribution : ezfsolrdocumentfieldobjectrelation , indexing all attributes of an object's related objects, storing them as subattributes. This then opens for applying all sorts of operations to there subattributes (search, filtering, facetting), using the 'myclass/myattribute/mysubattribute' syntax.

eZ Find 2.2 Customization & Advanced development Chapter 3 : Fields & Datatypes in Solr and eZ Find Field type management on the Solr side Open and read the file : /ezfind/java/solr/conf/schema.xml. This configuration file contains the hard-coded definition for a certain amount of fields (metadata fields for instance), but also defines the so-called dynamic fields : Solr relies on several configuration files, one of them is used to tell him that : - '_s' at the end of a field name means string , - '_t' for text , etc.

eZ Find 2.2 Customization & Advanced development Chapter 3 : Fields & Datatypes in Solr and eZ Find Field type management on the Solr side This files can also be used to define more complex behaviours for given eZ Publish datatypes, like the keywords datatype ( ezkeyword ). Two different field types definitions can be found 'keyword' for case-sensitive cases 'lckeyword' for lower-case cases This example, keywords fields management, teaches a lot about Solr configuration. One can note the way Solr filters are called, how coma-based word separations are handled ( PatternTokenizerFactory ), case-sensitivity management ( LowerCaseFilterFactory ), duplicate removal ( RemoveDuplicatesTokenFilterFactory ), etc.

eZ Find 2.2 Customization & Advanced development Chapter 4 : Indexing additional fields in Solr Presenting the case : Develop a year and year & month filter with eZ Find Usually in this case, a template operator is developed which builds the appropriate SQL queries. This can quickly become complicated, and often has sharp limitations ( eZArchive ) eZ Archive classical SQL limitation : Only the 'publication_date' parameter is taken into account, and no room is left for using a content-class-specific date attribute Hide empty month (March : 0 items) Items count

eZ Find 2.2 Customization & Advanced development Chapter 4 : Indexing additional fields in Solr Indexing Year and Year/Month values in Solr Create your own PHP class to manage the ‘ ezdate ’ datatype [SolrFieldMapSettings] CustomMap [ezdate]= ezfSolrDocumentFieldDate eZ Find settings ( ezfind.ini , to be overridden in the ezfind.ini.append.php file of your extension ) allow for delegating the indexing process of an eZ Publish datatype to a given PHP class /extension/myextension/classes / ezfsolrdocumentfielddate.php extends ezfSolrDocumentFieldBase

eZ Find 2.2 Customization & Advanced development Chapter 4 : Indexing additional fields in Solr Indexing Year and Year/Month values in Solr Role of the getFieldName() method This method is invoked the attributes names (within eZ Find) to Solr field names. For instance, when building a facet using the following syntax : ' mycontentclass/mydateattribute ', this method should return ' attr_mydateattribute_dt '. Important : to make sure the written code is generic enough, and avoid hard-coding the Solr field names, we will use the handy generateSubattributeFieldName and generateAttributeFieldName methods.

eZ Find 2.2 Customization & Advanced development Chapter 4 : Indexing additional fields in Solr Role of the getData() method This method is invoked to extract data from eZ Publish, and prepare it prior to indexing in Solr . This method is the place to add additional fields like ' year ' et ' yearmonth '. ' mycontentclass/mydateattribute/year ', translated in Solr under : ' subattr_date-year_dt ‘ ' mycontentclass/mydateattribute/yearmonth ', translated in Solr under : ' subattr_date-yearmonth_dt ' Indexing Year and Year/Month values in Solr

eZ Find 2.2 Customization & Advanced development Chapter 4 : Indexing additional fields in Solr Indexing Year and Year/Month values in Solr Building the facet navigation through a template billet/date/yearmonth : subattr_date-yearmonth_dt billet/date/year : subattr_date-year_dt Notre : The ' sort ', ' alpha ' statement does not actually specify an alphabetical sort. It rather helps specifying that no ' count ' sort should occur (number of items matching a given facet). In this case, Solr automatically uses an ' increasing ' sort, based on its index and the datatype of the concerned field (this explains the usage of the reverse operator to get an 'increasing' list). You could manage year / month faceting using eZ Find / Solr in another way. This use case help to learn how manipulate subattributes and additional fields.

eZ Find 2.2 Customization & Advanced development Chapter 4 : Indexing additional fields in Solr Other real life exemple Filtering events using a date range, (multi dates event… each month event) « Event » content class : Dates (ezmatrix ? Custom datatype ) Titre (textline) Intro (XML) … PHP Class attr_date_1 attr_date_2 attr_date_3 … Front_end filtering http:// www.tourismebretagne.com

eZ Find 2.2 Customization & Advanced development Chapter 4 : Indexing additional fields in Solr Other real life exemple Faceting on object(s) relation subattribute « Tourism product » content class : Main location (object relation) Titre (textline) Intro (XML) … PHP Class attr_mainlocation-title_s Front_end filtering http:// www.tourismebretagne.com « location » content class : Title (string) http://projects.ez.no/ezfsolrdocumentfieldobjectrelation

eZ Find 2.2 Customization & Advanced development Chapter 5 : Enhance eZ Find using the Solr syntax Is it possible to mix eZ Find and Solr syntax ? YES, it could make my day (and save my project) YES, but don’t think about it : BAD PRACTICE. This potentially endangering the lower layer’s evolutivity (Solr), and your own project evolutivity Solr field name

eZ Find 2.2 Customization & Advanced development Chapter 5 : Enhance eZ Find using the Solr syntax Resolve a commun issue : Sort an attribute present in several content classes « Article » content class : My custom Date (date) Titre (textline) Intro (XML) … « Post » content class : My custom Date (date) Titre (textline) Intro (XML) … Fetching and mixing « article » and « news » content result : 2010/06/04 – My Article 1 2010/06/02 – My Post 1 2010/05/22 – My Article 2 2010/05/15 – My Post 2 Good luck without using eZ Find : custom Fetch, amazing SQL… My Custom Date Desc

eZ Find 2.2 Customization & Advanced development Chapter 5 : Enhance eZ Find using the Solr syntax Resolve a commun issue : Sort an attribute present in several content classes The solution, using eZ Find : Remember ! The fortunate absence of the content class identifier in the field name means we can leverage this homonymy as we wish, through searches, filters or sorts depending « eZ Publish » Fetch : Only 1 content class « eZ Find » Fetch : several content class, using the field homonymy Solr field name

eZ Find 2.2 Customization & Advanced development Chapter 5 : Enhance eZ Find using the Solr syntax Complex search filters Remember ! Solr is a Lucene service : http://lucene.apache.org/java/2_9_1/queryparsersyntax.html ‘ filter ’, array(‘ attr_title_s :[ A TO G ] AND ezf_df_text : google ~0.7’) Only returns results of which the ' title ' starts by A,B,C,D, E or F (G excluded), and the content of which approximately contains the ' google ' expression (means it may also contain : Google, iGoogle, etc.). Note : the 'ezf_df_text' field is built dynamically, by copying the content of all of the document's 'string' , 'text' ou 'keyword' fields.See the schema.xml file, and the definition of these “ copyField ” fields for more details. ‘ filter ’, array(‘NOT ( attr_title_t : ez+find ) OR attr_intro_t : ez+find ) )’) Only returns results which contain the ‘ ez find ’ or ‘ eZ Find ’ expression in the ‘ title ’ or ‘ Intro ’ attributes. Note the usage of the ‘ text ’ (_t) of the ‘title’ attribute, bringing case-insensitivity , unlike the ‘ string ’ type

eZ Find 2.2 Customization & Advanced development Chapter 6 : Conclusion eZ Find : first step to the next generation CMS ? DONE Advanced content Indexing (indexing, querying, faceting, browsing) Dynamic content storage COULD BE BETTER ( no SQL ) FileSystem FrameWork API COULD BE BETTER Custom API / Zeta Hooks / workflow / Restfull API Read the Paul B. roadmap

Follow me (and find eZ Find tutorials) : eZ Conference 2010 June 24th FR : http:// www.gandbox.fr EN : http:// share.ez.no

20100622 e z_find_slides_gig_v2.1

More Related Content

What's hot (20)

Similar to 20100622 e z_find_slides_gig_v2.1 (20)

Recently uploaded (20)

20100622 e z_find_slides_gig_v2.1