Human detection has been widely used in many applications. In the meantime, it is still a difficult problem with many open questions due to challenges caused by various factors such as clothing, posture and etc. By investigating several benchmark methods and frameworks in the literature, this paper proposes a novel method which successfully implements the Real AdaBoost training procedure on multi-scale images. Various object features are exposed on multiple levels. To further boost the overall performance, a fusion scheme is established using scores obtained at various levels which integrates decision results with different scales to make the final decision. Unlike other score-based fusion methods, this paper re-formulates the fusion process through a supervised learning. Therefore, our fusion approach can better distinguish subtle difference between human objects and non-human objects. Furthermore, in our approach, we are able to use simpler weak features for boosting and hence alleviate the training complexity existed in most of AdaBoost training approaches. Encouraging results are obtained on a well recognized benchmark database.