2011_IGARSS_WV2_final.ppt
- 1. Land cover identification for finding hazelnut fields using WV2 imagery Kadim Ta şdemir 1 and Selcuk Reis 2 1 Monitoring Agricultural Resources Unit Institute for Environment and Sustainability European Commission Joint Research Centre 2 Aksaray University, Turkey
- 2. Outline Introduction S ignificance of the problem Detection of orchards Hazelnut study area Methodology T extural (Gabor) features and spectral values Self Organizing Maps (SOMs) and learning vector quantization (LVQ) Experimental results Study area Performance measures (accuracy, kappa, etc.) Additional value of WV2 imagery Conclusions and broader applications
- 3. Significance of the problem Land cover identification from remote sensing images has been essential for agricultural management and monitoring . An economically very essential agricultural practice is permanent crops . Among permanent crops, nuts (hazelnuts, almonds, walnuts, pistachios, locust beans) are recognized for their role in rural development (their orchards are often grown in lands where cultivation is difficult) and environmental respect (the adopted practices for nuts cultivation enable to efficiently fight against erosion). An accurate land cover identification delineating fields of permanent crops (hazelnuts in this study) will aid in better agricultural management.
- 4. Detection of orchards Interactive detection of individual orchards by analysis of panchromatic or multispectral bands OliCount, Peedell et al. ESRI 1999 NUTGIS report, JRC, 2006 Daliakopoulos et al. PE&RS 2009 Textural features calculated from panchromatic band to delineate orchards, vineyards, and forest, thanks to their (ideally) discriminative spatial patterns Wavelet features, Ranchin et al. PE&RS 2001 Autocorrelation of neighbor pixels, Warner and Steinmaus, PE&RS 2005 Variogram analysis, Trias-Sanz, IEEE TGRS 2006 Structural features, Yalniz and Aksoy, Pattern Recognition, 2010 Combination of spectral and textural features Aksoy et al., IEEE TGRS 2010 Combination of spectral, temporal and (Gabor) textural features (Quickbird) Reis and Ta ş demir, ISPRS Photo. 2011
- 5. Hazelnut study area The major hazelnut producer in the world (about 75%) : Turkey along the Black Sea coast (where relief is rather strong) hazel orchards are often small with a high planting density, and various plantation patterns natural vegetation in the area can be spatially discontinuous. A photo from the study area A color composite image of hazelnut fields
- 6. Methodology 1. Feature selection Textural features for spatial properties, obtained from panchromatic imagery Spectral values Merged features obtained by combining normalized spectral values and Gabor features 2. Classification (using SOM and LVQ) Determine land cover types 5 classes: hazelnut, other woodlands, agriculture, soil, urban areas C1: Classify textural features good performance for woody vegetation (hazelnut and woodlands) C2: Classify merged features good overall classification performance. 3. Decision rule Modify the classification result of C2, by updating hazelnut assignment according to C1 Spectral values Textural features Classifier 2 Classifier 1 Multispectral imagery Panchromatic imagery Decision rule Classification result
- 7. Feature selection Textural features: multi-scale multi-oriented Gabor features to represent distinctive spatial properties (regular plantation of orchards) 4 scales, 6 orientation: select the highest response for each scale to obtain orientation independent features, resulting in 4 features Spectral values: to represent spectral reflectance at various bands An RGB color composite of 3 scales of Gabor features Woody vegetation (nuts, forests, woodlands) have high responses for small scales (greenish, brownish regions); Urban areas have high responses for all/large scales (white, bluish regions) whereas Smooth areas (soil, agriculture) have low responses (dark regions)
- 8. Classification method*: SOMs A common approach for land cover identification is the use of the self-organizing map (SOM) SOMs are shown useful especially for remote sensing applications with complex cluster structures (Villmann et al. 2003, Merenyi et al 2005, Ta ş demir and Merenyi 2009) Learning vector quantization (LVQ): supervised vector quantization based on SOM, which can be successfully used with few labeled samples *: For this study, the aim is to analyze the performance of WorldView2 imagery, therefore selection of the classifier is not of primary importance. Instead of SOM and LVQ usage, other common approaches such as Maximum Likelihood Classifier, Radial Basis Functions and Support Vector Machines may also be preferred.
- 9. Self-organizing maps A topology preserving 2-d spatial mapping of acoustic frequencies on the auditory cortex A Self-Organizing Map (SOM) an unsupervised artificial neural network learning paradigm intends to mimic this data processing (Kohonen, 1982).
- 10. Self-organizing maps A 2-d SOM rigid lattice Data manifold d -dimensional prototypes Voronoi polyhedra of some neighbor prototypes 1. Competition i = arg min j ||x - w j ||, j=1, … , N 2. Cooperation The neighborhood of i is activated wrt a neighborhood function h j,i(x) (t). 3. Synaptic adaptation w j (t+1) = w j (t)+ ε 0 (t) h j,i(x) (t) (x - w j (t))
- 11. Self-organizing maps A 2-d SOM rigid lattice Data manifold d -dimensional prototypes adaptive vector quantizer prototypes ordered by similarities on a 2-d rigid grid nonlinear topology preserving mapping Voronoi polyhedra of some neighbor prototypes
- 12. Learning vector quantization LVQ is a supervised learning, which slightly modifies the SOM units when a misclassification occurs: Each SOM unit is labeled by maximum vote of the labels of training samples mapped to it. Labeled training samples v are iteratively mapped to the labeled SOM, and the closest SOM unit wi and the second closest SOM unit wj to a randomly selected training sample v are adapted, using the equation below, if wi and v are in different classes whereas wj and v are in the same class.
- 13. Outline Introduction S ignificance of the problem Detection of orchards Hazelnut study area Methodology T extural (Gabor) features and spectral values Self Organizing Maps and learning vector quantization Experimental results Study area Performance measures (accuracy, kappa, etc.) Additional value of WV2 imagery Conclusions and broader applications
- 14. Study area WorldView 2 imagery Panchromatic and 8 multispectral bands 2920*4775 pixels ~3.5 km2
- 15. Study area An example representation of the study area using natural color composite For agricultural management and monitoring required in this study, 5 classes (hazelnut, woodland, agriculture, soil and urban regions) are selected.
- 16. Classification maps – 8 band Classification map produced when only 8 multispectral bands were used: PA_hazelnut= 85.4%; UA_hazelnut =68.6%; F_b=0.76; OA = 76.4%; Kappa = 0.64 PA: producer accuracy, UA: user accuracy, F_b: geometric mean of PA and UA of hazelnut OA:overall accuracy
- 17. Classification maps – 4 band Classification map produced when only 4 (R,G,B, NIR1) multispectral bands were used: PA_hazelnut= 87.8%; UA_hazelnut =70.2%; F_b=0.78; OA = 79.5%; Kappa = 0.69 PA: producer accuracy, UA: user accuracy, F_b: geometric mean of PA and UA of hazelnut OA:overall accuracy
- 18. Classification maps - Gabor Classification map produced when Gabor features (calculated from pan) were used: PA_hazelnut= 95.0%; UA_hazelnut =80.7%; F_b=0.87; OA = 77.9%; Kappa = 0.66 PA: producer accuracy, UA: user accuracy, F_b: geometric mean of PA and UA of hazelnut OA:overall accuracy
- 19. Classification maps – 8bands&Gabor Classification map produced when 8 multispectral bands and Gabor features were used: PA_hazelnut= 93.6%; UA_hazelnut =74.7%; F_b=0.83; OA = 83.2%; Kappa = 0.74 PA: producer accuracy, UA: user accuracy, F_b: geometric mean of PA and UA of hazelnut OA:overall accuracy
- 20. Classification maps – Merged Classification map produced when a decision rule was added to 8 bands and Gabor features PA_hazelnut= 90.6%; UA_hazelnut =85.1%; F_b=0.88; OA = 87.8%; Kappa = 0.81 PA: producer accuracy, UA: user accuracy, F_b: geometric mean of PA and UA of hazelnut OA:overall accuracy
- 21. Performance results of WV2 imagery WorldView 2 imagery Measure Class MS 8band MS 4 band Gabor MS+Gabor Merged PA woodland 64,68 75,51 79,82 71,73 85,8 nut 85,41 87,77 95 93,64 90,6 urban 93,24 93,19 39,46 82,64 82,64 agri 80,26 45,99 50,54 84,37 84,37 soil 75,81 84,76 46,64 93,07 93,07 UA woodland 84,05 91,05 81,34 89,56 87,97 nut 68,62 70,19 80,68 74,66 85,09 urban 77,55 84,28 88,83 93,09 93,09 agri 77,76 64,26 54,27 92,35 92,35 soil 92,75 94,1 60 92,62 92,62 Overall accuracy 76,38 79,47 77,94 83,24 87,8 F_b 0,76 0,78 0,87 0,83 0,88 Kappa 0,64 0,69 0,66 0,74 0,81 Gabor features has good PA and UA values for woodlands and hazelnut, in expense of very poor results for other classes Combined usage of 8 bands and Gabor features increases performance significantly A decision rule in addition to combined usage of 8 bands and Gabor features produces the best classification performance Confusion matrix for Merged Total Woodland Hazelnut Urban Agri. Soil PA 90072 77280 12591 127 70 4 85,80 82824 6572 75035 0 1217 0 90,60 14488 1290 0 11973 0 1225 82,64 18442 2317 561 1 15560 3 84,37 16611 388 0 761 2 15460 93,07 UA 87,97 85,09 93,09 92,35 92,62 87,80 4 bands produces better classification than 8 bands (for this study) in expense of poor performance for agriculture class (confusion with dense hazelnut fields) Low performance measures
- 22. Additional value of 4 new bands The use of 8-bands produced higher performance measures for the proposed method When merged features considered only, the use of 8-bands: higher F_b on average; has a lower performance wrt Kappa and overall accuracy has better discrimination for 3 classes Main confusion is between woodland and hazelnut
- 23. Comparison to QB performance The performance improvement achieved by using WV2 imagery, mainly due to the increased spatial resolution (0.5m versus 0.6m of QB imagery), significant to achieve an accurate land cover identification for finding hazelnut fields Imagery --> WorldView 2 imagery Quickbird imagery Measure Class MS 8band MS 4 band Gabor MS 4 band Gabor PA woodland 64,68 75,51 79,82 65,66 74,96 nut 85,41 87,77 95 81,37 92,16 urban 93,24 93,19 39,46 85,96 32,98 agri 80,26 45,99 50,54 37,65 11,85 soil 75,81 84,76 46,64 16,86 17,55 UA woodland 84,05 91,05 81,34 76,06 75,52 nut 68,62 70,19 80,68 63,8 71,3 urban 77,55 84,28 88,83 88,8 80,06 agri 77,76 64,26 54,27 35,55 28,37 soil 92,75 94,1 60 55,28 25,8 Overall accuracy 76,38 79,47 77,94 66,72 69,19 F_b (hazelnut) 0,76 0,78 0,87 0,72 0,8 Kappa 0,64 0,69 0,66 0,49 0,51 A better performance by using spectral bands (either 8-band or 4-band) of WV2 imagery A better performance by using Gabor features extracted from WV2 panchromatic imagery
- 24. Conclusions A classification system which accurately finds hazelnut fields together with lands in good agricultural condition. If the aim of the study is solely to determine hazelnut fields, the use of textural features can achieve a similar detection accuracy. However, including multispectral bands improves the overall accuracy and F_b values significantly. The use of 8-band produces a better performance than the use of 4-band, but the improvement is not of great significance and a cost/benefit analysis should be assessed based on the application requirements. The resulting precise delineation of nut fields aids their management and control support schemes regulated by the CAP of the EU.
- 25. Conclusions The proposed system can be applied to find other permanent crops, since they often have a regular plantation pattern that can be represented by textural features. The proposed system may also be used to determine lands in good agricultural condition, especially for Bulgaria and Romania, where it is necessary to annually assess agricultural land eligible for the CAP subsidies Moreover, it may be used find eligible/ineligible features within the agricultural lands, which need to be extracted according to the CAP regulations.
- 26. Thank you for your attention!
Editor's Notes
- #5: An approach to determine permanent crops is interactive detection of individual orchards by analysis of panchromatic or multispectral bands using semi-automatic schemes with user-tuned parameters [1], [2], [3]. These schemes are successful to some extent but a considerable user interaction is necessary and, in case of detecting nut orchards, determination of the covered land is important for subsidies rather than extraction of individual trees. Some studies focus on textural features calculated from panchromatic band to delineate orchards, vineyards, and forest, thanks to their (ideally) discriminative spatial patterns [4], [5]. June 14, 2010 DRAFT DETECTION OF ORCHARDS BY SELF ORGANIZING MAPS, SUBMITTED TO IEEE TGRS, 14/06/2010 3 Ranchin et al. [4] uses multi-level wavelet-based textures for capturing different scales of spatial patterns for vineyard detection. Warner and Steinmaus [5] review textural features for discriminating orchards, vineyards, and other fields; and use an autocorrelation of neighbor pixels along four different directions (vertical, horizontal and diagonals), to produce a high accuracy. Trias-Sanz [6] extends the use of autocorrelation in [5] by using a variogram analysis for multispectral bands to differentiate different classes using orientations and periodicities. This approach produces a better accuracy than in [5] and estimates periodicity and orientation of regular patterns. Another method, which estimates orientations and scales of regular patterns, uses multi-scale isotropic filters and projection profiles to calculate a regularity index to accurately locate structural textures [7]. They produce accurate estimations for orientations and scales, when regular pattern exists. However, when different orchard types or forest with regular plantation exist in the scene, even though regularity features may provide additional information, spectral values are necessary to detect the orchard type. A new study briefly uses spectral information by merging a spectral feature (normalized difference vegetation index) with texture features (multiscale Gabor and granulometry) through a rule-based classifier for a successful mapping of linear woody vegetations [8]. Apart from permanent crops, a common approach for land cover identification using remote sensing images is the use of pixel based methods based on machine learning or statistical approaches. The self-organizing map (SOM), a learning paradigm that produces an adaptive vector quantization and topology preserving mapping [9], and its variants have been shown useful especially for remote sensing applications with complex cluster structures [10], [11], [12], [13]. However in clustering or classification of orchards, the aim is to find the land cover rather than focusing individual pixel memberships that can be sensitive to various factors such as shadows, relief and spectral mixing between classes. In addition, discriminative spatial pattern of orchards cannot be represented by pixel-based approach. A recent study, Image Features Map (IFM) [14], combines the advantageous properties of the SOM by spatial information, through considering spectral values within a neighborhood, without any computation of textural features. IFM is shown to work well for unsupervised clustering of rural regions with few types of homogeneous land covers.