Gastroscopy is important tool for the clinical examination of gastric diseases, and the abnormality detection on the gastroscopic images will help physicians to diagnose. An improved patches assembled by local weights is presented in this paper. First, a series of classifiers on image patches with different sizes have been analyzed to find the suitable size. The boosted stumps are employed as the image patch classifiers. At last, considering the relationship between the neighboring image patches, the patches assemble method based on the local information is applied to enhance the coherence of patches. The experiment results show that the assemble method the local weights get the true positive rate (TP) at 75.7%, the true negative rate (TN) at 86.4% and the mean error at 16.7%. Comparing with the other assemble methods like mean filter, local weights has better performance.