The analysis of nucleus shape is useful for the pathological diagnosis or prognosis. The shape of the nucleus in the digital pathological images from different sources may vary greatly. Although, Convolutional Neural Network (CNN) has proven its success in automatic nucleus segmentation, segmentation performance may be reduced due to varies in nucleus size. In this paper, we proposed a CNN based size adaptive nucleus segmentation method. This method adapts the CNN model to image clusters with similar estimated nucleus size to increase its nucleus segmentation performance. Furthermore our method can automatically segment nuclei and the only parameter that needs to be set is the number of clusters. We compared this method to several existing methods on two datasets from the nucleus segmentation challenge. The proposed method achieved satisfactory results with mean Intersection-over-Union (IoU) 0.718 and 0.798 and mean F1-score 0.780 and 0.864. In addition, compared with the method without size adaptation, the proposed method improves the segmentation performance and is easy to implement.