Effective partitioning of the feature space for high classification accuracy with due attention to rare class members is often a difficult task. In this paper, the border vector detection and adaptation (BVDA) algorithm is proposed for this purpose. The BVDA consists of two parts. In the first part of the algorithm, some specially selected training samples are assigned as initial reference vectors called border vectors. In the second part of the algorithm, the border vectors are adapted by moving them toward the decision boundaries. At the end of the adaptation process, the border vectors are finalized. The method next uses the minimum distance to border vector rule for classification. In supervised learning, the training process should be unbiased to reach more accurate results in testing. In the BVDA, decision region borders are related to the initialization of the border vectors and the input ordering of the training samples. Consensus strategy can be applied with cross validation to reduce these dependencies. The performance of the BVDA and consensual BVDA were studied in comparison to other classification algorithms including neural network with backpropagation learning, support vector machines, and some statistical classification techniques.