An innovative ensemble approach to image categorization was proposed that uses a hybrid of k-nearest neighbors (kNN) and support vector machines (SVM). The approach finds the k closest neighbors to a query sample and trains a local SVM on those neighbors. This preserves distance relationships while avoiding some issues with high variance that kNN faces. Empirical tests on challenging image datasets showed the hybrid method outperformed leading learning-based parametric image classification approaches.