On the use of construct reliability in MIS research: A meta-analysis

This paper reports results of a meta-analysis of construct reliability measures reported in MIS research. Based on 418 measures from 63 articles published in four major journals, it is observed that although scales developed more recently are no more reliable than those developed in the 1980s, the reliability of most scales reported in the literature is above generally accepted levels. However, for scales used in basic research, more than 40% of them had a reliability lower than the minimal acceptable level, at least 0.80. Scales using interviews for data collection; with more items, obtained from previous studies; and generated from both literature review and interview, were found to have higher reliability. Other research design characteristics, including sample size, type of subjects, scale type, scale format and number of scale points, were found to be insignificant in affecting reliability. Implications of the findings are discussed and guidelines for researchers on the design of their research methodologies with respect to having scales with the level of reliability suitable for the types of research at hand are provided.