Study and scale quality in second-language survey research: the case of anxiety and willingness to communicate

Sudina, Ekaterina (2022) Study and scale quality in second-language survey research: the case of anxiety and willingness to communicate. Doctoral thesis, Northern Arizona University.

Text Sudina_2022_study_scale_quality_second-language_survey_research_case_a.pdf - Published Version Restricted to Repository staff only Download (3MB) | Request a copy

Abstract

Apart from providing a bird’s eye view of a given domain of inquiry (Ioannidis et al., 2015), systematic reviews such as methodological syntheses offer a magnifying glass on prevalent existing problems in primary research and furnish empirically grounded solutions to rectify these issues (Page et al., 2021). As survey research in second language acquisition grows in popularity, especially in the substantive domain of individual differences, the adherence to best practices associated with study and questionnaire design alongside reporting practices related to transparency is critical for a better understanding of factors that influence second language (L2) development. To ensure that a self-report scale designed to measure a latent construct such as anxiety and willingness to communicate (WTC) indeed targets the construct of interest and does it consistently and accurately, authors of primary research should demonstrate that their instrument possesses acceptable psychometric properties (i.e., evidence of validity and reliability) and is suitable to use with their chosen population of L2 learners or teachers. However, previous studies have raised concerns about the psychometric properties of questionnaires in L2 research (Al-Hoorie & Vitta, 2019; Gu, 2016). Building on works such as Plonsky (2013, 2014) and Sudina (2021), this methodological synthesis focuses on the state of study and scale quality in L2 anxiety and WTC research by examining key methodological issues of quantitative survey research and offering empirically grounded suggestions for future studies. The scope of the study is confined to a total of 232 peer-reviewed articles that used 385 L2 anxiety and WTC scales and were published in 22 leading applied linguistics journals over a 20-year period. The search channels for locating and retrieving eligible studies included journal websites, the Second-language Research Corpus (Plonsky, n.d.), and two databases: Linguistics and Language Behavior Abstracts (LLBA) and PsycINFO. Features targeting sample, survey, and scale characteristics and reporting practices at both the study and scale levels have been examined in depth as well as compared over time. Particularly problematic is the lack of evidence for scale content and construct validity—including testing for measurement invariance and referring to previous validation studies for existing scales—as well as reporting rates of survey responses and missing data. The results and implications of this study contribute to concurrent attempts at methodological reform in applied linguistics and aim to promote ethically appropriate research practices.

Item Type:	Thesis (Doctoral)
Publisher’s Statement:	© Copyright is held by the author. Digital access to this material is made possible by the Cline Library, Northern Arizona University. Further transmission, reproduction or presentation of protected items is prohibited except with permission of the author.
Keywords:	Second language acquisition; Communication; Reliability; Methodology; Best practices;
Subjects:	P Language and Literature > P Philology. Linguistics
NAU Depositing Author Academic Status:	Student
Department/Unit:	Graduate College > Theses and Dissertations College of Arts and Letters > English
Date Deposited:	01 Jun 2023 16:30
Last Modified:	01 Jun 2023 16:30
URI:	https://openknowledge.nau.edu/id/eprint/5935

Actions (login required)

IR Staff Record View

Downloads