Mitigating threats to validity in empirical software engineering: A traceability case study

Nasser Mustafa; Yvan Labiche; Dave Towey

doi:10.1109/COMPSAC.2019.10227

Mitigating threats to validity in empirical software engineering: A traceability case study

Nasser Mustafa, Yvan Labiche, Dave Towey

School of Computer Science

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

5 Citations (Scopus)

Abstract

The issue of validity threats in empirical software engineering research is important. However, some authors overlook this, focusing on validating their work through application of fundamental testing techniques, instead. However, testing is different to empirical validation, with the latter being more concerned about how experimental conclusions are justified. An important factor that can render an experimental conclusion incorrect is researcher’s bias, which can be especially relevant when setting the experimental parameters. Therefore, consideration of validity threats is essential to enable confidence in research results and assure the research quality. This paper provides a practical approach for mitigating threats to validity in empirical software engineering using a sequence of software activities. The paper is based on a real-world traceability case study for illustration purposes.

Original language	English
Title of host publication	Proceedings - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019
Editors	Vladimir Getov, Jean-Luc Gaudiot, Nariyoshi Yamai, Stelvio Cimato, Morris Chang, Yuuichi Teranishi, Ji-Jiang Yang, Hong Va Leong, Hossian Shahriar, Michiharu Takemoto, Dave Towey, Hiroki Takakura, Atilla Elci, Susumu Takeuchi, Satish Puri
Publisher	IEEE Computer Society
Pages	324-329
Number of pages	6
ISBN (Electronic)	9781728126074
DOIs	https://doi.org/10.1109/COMPSAC.2019.10227
Publication status	Published - Jul 2019
Event	43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019 - Milwaukee, United States Duration: 15 Jul 2019 → 19 Jul 2019

Publication series

Name	Proceedings - International Computer Software and Applications Conference
Volume	2
ISSN (Print)	0730-3157

Conference

Conference	43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019
Country/Territory	United States
City	Milwaukee
Period	15/07/19 → 19/07/19

Keywords

Empirical software engineering
Traceability
Validity threats

ASJC Scopus subject areas

Software
Computer Science Applications

Access to Document

10.1109/COMPSAC.2019.10227

Cite this

Mustafa, N., Labiche, Y., & Towey, D. (2019). Mitigating threats to validity in empirical software engineering: A traceability case study. In V. Getov, J.-L. Gaudiot, N. Yamai, S. Cimato, M. Chang, Y. Teranishi, J.-J. Yang, H. V. Leong, H. Shahriar, M. Takemoto, D. Towey, H. Takakura, A. Elci, S. Takeuchi, & S. Puri (Eds.), Proceedings - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019 (pp. 324-329). Article 8754344 (Proceedings - International Computer Software and Applications Conference; Vol. 2). IEEE Computer Society. https://doi.org/10.1109/COMPSAC.2019.10227

Mustafa, Nasser ; Labiche, Yvan ; Towey, Dave. / Mitigating threats to validity in empirical software engineering : A traceability case study. Proceedings - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019. editor / Vladimir Getov ; Jean-Luc Gaudiot ; Nariyoshi Yamai ; Stelvio Cimato ; Morris Chang ; Yuuichi Teranishi ; Ji-Jiang Yang ; Hong Va Leong ; Hossian Shahriar ; Michiharu Takemoto ; Dave Towey ; Hiroki Takakura ; Atilla Elci ; Susumu Takeuchi ; Satish Puri. IEEE Computer Society, 2019. pp. 324-329 (Proceedings - International Computer Software and Applications Conference).

@inproceedings{4cdff69f7a5d4a78998c206aaf5d6db7,

title = "Mitigating threats to validity in empirical software engineering: A traceability case study",

abstract = "The issue of validity threats in empirical software engineering research is important. However, some authors overlook this, focusing on validating their work through application of fundamental testing techniques, instead. However, testing is different to empirical validation, with the latter being more concerned about how experimental conclusions are justified. An important factor that can render an experimental conclusion incorrect is researcher{\textquoteright}s bias, which can be especially relevant when setting the experimental parameters. Therefore, consideration of validity threats is essential to enable confidence in research results and assure the research quality. This paper provides a practical approach for mitigating threats to validity in empirical software engineering using a sequence of software activities. The paper is based on a real-world traceability case study for illustration purposes.",

keywords = "Empirical software engineering, Traceability, Validity threats",

author = "Nasser Mustafa and Yvan Labiche and Dave Towey",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE; 43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019 ; Conference date: 15-07-2019 Through 19-07-2019",

year = "2019",

month = jul,

doi = "10.1109/COMPSAC.2019.10227",

language = "English",

series = "Proceedings - International Computer Software and Applications Conference",

publisher = "IEEE Computer Society",

pages = "324--329",

editor = "Vladimir Getov and Jean-Luc Gaudiot and Nariyoshi Yamai and Stelvio Cimato and Morris Chang and Yuuichi Teranishi and Ji-Jiang Yang and Leong, {Hong Va} and Hossian Shahriar and Michiharu Takemoto and Dave Towey and Hiroki Takakura and Atilla Elci and Susumu Takeuchi and Satish Puri",

booktitle = "Proceedings - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019",

address = "United States",

}

Mustafa, N, Labiche, Y & Towey, D 2019, Mitigating threats to validity in empirical software engineering: A traceability case study. in V Getov, J-L Gaudiot, N Yamai, S Cimato, M Chang, Y Teranishi, J-J Yang, HV Leong, H Shahriar, M Takemoto, D Towey, H Takakura, A Elci, S Takeuchi & S Puri (eds), Proceedings - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019., 8754344, Proceedings - International Computer Software and Applications Conference, vol. 2, IEEE Computer Society, pp. 324-329, 43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019, Milwaukee, United States, 15/07/19. https://doi.org/10.1109/COMPSAC.2019.10227

Mitigating threats to validity in empirical software engineering: A traceability case study. / Mustafa, Nasser; Labiche, Yvan; Towey, Dave.
Proceedings - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019. ed. / Vladimir Getov; Jean-Luc Gaudiot; Nariyoshi Yamai; Stelvio Cimato; Morris Chang; Yuuichi Teranishi; Ji-Jiang Yang; Hong Va Leong; Hossian Shahriar; Michiharu Takemoto; Dave Towey; Hiroki Takakura; Atilla Elci; Susumu Takeuchi; Satish Puri. IEEE Computer Society, 2019. p. 324-329 8754344 (Proceedings - International Computer Software and Applications Conference; Vol. 2).

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Mitigating threats to validity in empirical software engineering

T2 - 43rd IEEE Annual Computer Software and Applications Conference, COMPSAC 2019

AU - Mustafa, Nasser

AU - Labiche, Yvan

AU - Towey, Dave

PY - 2019/7

Y1 - 2019/7

N2 - The issue of validity threats in empirical software engineering research is important. However, some authors overlook this, focusing on validating their work through application of fundamental testing techniques, instead. However, testing is different to empirical validation, with the latter being more concerned about how experimental conclusions are justified. An important factor that can render an experimental conclusion incorrect is researcher’s bias, which can be especially relevant when setting the experimental parameters. Therefore, consideration of validity threats is essential to enable confidence in research results and assure the research quality. This paper provides a practical approach for mitigating threats to validity in empirical software engineering using a sequence of software activities. The paper is based on a real-world traceability case study for illustration purposes.

AB - The issue of validity threats in empirical software engineering research is important. However, some authors overlook this, focusing on validating their work through application of fundamental testing techniques, instead. However, testing is different to empirical validation, with the latter being more concerned about how experimental conclusions are justified. An important factor that can render an experimental conclusion incorrect is researcher’s bias, which can be especially relevant when setting the experimental parameters. Therefore, consideration of validity threats is essential to enable confidence in research results and assure the research quality. This paper provides a practical approach for mitigating threats to validity in empirical software engineering using a sequence of software activities. The paper is based on a real-world traceability case study for illustration purposes.

KW - Empirical software engineering

KW - Traceability

KW - Validity threats

UR - http://www.scopus.com/inward/record.url?scp=85072662358&partnerID=8YFLogxK

U2 - 10.1109/COMPSAC.2019.10227

DO - 10.1109/COMPSAC.2019.10227

M3 - Conference contribution

AN - SCOPUS:85072662358

T3 - Proceedings - International Computer Software and Applications Conference

SP - 324

EP - 329

BT - Proceedings - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019

A2 - Getov, Vladimir

A2 - Gaudiot, Jean-Luc

A2 - Yamai, Nariyoshi

A2 - Cimato, Stelvio

A2 - Chang, Morris

A2 - Teranishi, Yuuichi

A2 - Yang, Ji-Jiang

A2 - Leong, Hong Va

A2 - Shahriar, Hossian

A2 - Takemoto, Michiharu

A2 - Towey, Dave

A2 - Takakura, Hiroki

A2 - Elci, Atilla

A2 - Takeuchi, Susumu

A2 - Puri, Satish

PB - IEEE Computer Society

Y2 - 15 July 2019 through 19 July 2019

ER -

Mustafa N, Labiche Y, Towey D. Mitigating threats to validity in empirical software engineering: A traceability case study. In Getov V, Gaudiot JL, Yamai N, Cimato S, Chang M, Teranishi Y, Yang JJ, Leong HV, Shahriar H, Takemoto M, Towey D, Takakura H, Elci A, Takeuchi S, Puri S, editors, Proceedings - 2019 IEEE 43rd Annual Computer Software and Applications Conference, COMPSAC 2019. IEEE Computer Society. 2019. p. 324-329. 8754344. (Proceedings - International Computer Software and Applications Conference). doi: 10.1109/COMPSAC.2019.10227

Mitigating threats to validity in empirical software engineering: A traceability case study

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this