Psychometric properties of the Automatic Thoughts Questionnaire-8 in two Spanish nonclinical samples

Authors: Francisco J. Ruiz, Pablo Riaño-Hernández, Cindy Bobadilla-Pérez, Elena García-Martín

Year: 2020

Journal: International Journal of Cognitive Therapy, 13, 39–55

DOI: https://doi.org/10.1007/s41811-019-00061-1

Article Type: PSYCHOMETRIC

---

CONTEXT AND OBJECTIVES

The Automatic Thoughts Questionnaire-8 (ATQ-8) is a brief version of the ATQ-30 (Hollon & Kendall, 1980), designed to assess the frequency of negative automatic thoughts. Negative automatic thoughts constitute a central construct in cognitive models of psychopathology, particularly in depression and anxiety disorders. While the 30-item version is widely utilized in research and clinical practice, there exists a need for briefer instruments that maintain adequate psychometric properties for contexts with time constraints or repeated assessments.

Although the ATQ-8 has been validated in several international contexts, a gap exists in the literature regarding its systematic validation in Spanish samples. The primary objective of this study was to examine the psychometric properties of the ATQ-8 in Spanish nonclinical populations, including evaluation of its factor structure, measurement invariance across demographic groups, internal reliability, and convergent and discriminant validity.

Specific objectives were: (1) to confirm the unifactorial structure of the ATQ-8; (2) to examine whether the instrument maintains strict measurement invariance across samples (university students vs. general population), sex, age groups, and educational levels; (3) to establish internal reliability through internal consistency coefficients; and (4) to evaluate convergent validity with related measures of depressive, anxious, and stress symptoms, cognitive rumination, inflexibility, and dysfunctional attitudes, as well as discriminant validity through negative correlations with life satisfaction.

---

METHOD

Participants

The study employed two independent Spanish samples, both nonclinical:

Sample 1: 302 university undergraduates (Mage = 20.8 years, SD = 4.2) recruited from the University of Almería during regular teaching periods. Seventy-two percent were female. Participants completed questionnaires as part of extracurricular class activities.

Sample 2: 846 general population participants (Mage = 32.5 years, SD = 13.7) recruited online through digital recruitment platforms. Fifty-eight percent were female. This sample included greater diversity in educational and occupational levels, better representing the general Spanish population.

Combined sample: Total N = 1,148 participants. Demographic diversity enabled examination of measurement invariance across subgroups defined by sex (male vs. female), age (categorized into five-year groups), and educational level (primary/secondary vs. higher education).

Design

A cross-sectional psychometric validation design was employed with two independent samples. This approach is standard in psychometric research for: (1) confirming replicability of factor structure in independent samples, (2) examining whether model parameters are invariant across relevant demographic groups, and (3) evaluating relationships with convergent and discriminant constructs.

Instruments

Automatic Thoughts Questionnaire-8 (ATQ-8): A brief version of the ATQ-30 consisting of 8 items rated on a 5-point Likert scale (1 = never, 5 = always). Items represent typical negative automatic thoughts (e.g., "I cannot do anything well," "If I fail, I am a failure as a person"). Total scores range from 8 to 40, with higher scores indicating greater frequency of negative automatic thoughts. The instrument was translated to Spanish following standard back-translation procedures.

Dysfunctional Attitude Scale-Revised (DAS-R): Measures dysfunctional and ruminative beliefs. Used as a convergent validity measure given theory predicting that negative automatic thoughts co-vary with underlying dysfunctional attitudes and beliefs.

Depression, Anxiety, and Stress Scales-21 (DASS-21): Self-report instrument assessing depression, anxiety, and stress symptoms via 21 items. The depression subscale was of particular interest given the strong theoretical link between negative automatic thoughts and depressive symptoms.

Acceptance and Action Questionnaire-II (AAQ-II): Measures psychological inflexibility, a construct related to rumination and negative thinking processes. Included to examine correlations with constructs sharing emotion regulation processes.

Cognitive Fusion Questionnaire (CFQ): Assesses cognitive fusion (tendency to take thoughts literally). Included as a convergent construct given that negative automatic thoughts may relate to greater cognitive fusion.

Generalized Pliance Questionnaire (GPQ): Measures tendency to follow verbal rules inflexibly. Included as a convergent construct theorized within contextual behavioral science framework.

Satisfaction With Life Scale (SWLS): Measures global well-being. Included as a discriminant construct with expectations of negative correlations, assuming negative automatic thoughts relate inversely to well-being.

Statistical Analysis

Confirmatory factor analysis (CFA): A unifactorial model of the ATQ-8 was specified confirming that all 8 items load on a single latent factor. The DWLS (Diagonally Weighted Least Squares) estimator, appropriate for ordinal data, was employed. Model fit was evaluated via standard indices: Root Mean Square Error of Approximation (RMSEA ≤ .10), Comparative Fit Index (CFI ≥ .95), Non-Normed Fit Index (NNFI ≥ .95), and Standardized Root Mean Square Residual (SRMR ≤ .08).

Multi-group measurement invariance analysis: Invariance of the ATQ-8 factor structure was tested across: (1) samples (students vs. general population), (2) sex, (3) age groups, and (4) educational levels. Three invariance levels were examined: configural (equal structure), metric (equal factor loadings), and scalar (equal intercepts). Strict invariance (equal residuals) was also tested. Differences in CFI (ΔCFI ≤ .01) were used as criteria for not rejecting invariance.

Internal consistency: Cronbach's alpha and McDonald's omega coefficients were calculated for each sample and combined, with values ≥ .80 considered adequate.

Convergent validity: Pearson correlations were computed between ATQ-8 and related measures (DAS-R, DASS-21 subscales, AAQ-II, CFQ, GPQ). Moderate to high correlations (r > .40) statistically significant were expected.

Discriminant validity: Correlations between ATQ-8 and SWLS were examined, expecting moderate negative correlations (r < -.40).

Clinical group analysis: Analysis of variance (ANOVA) compared participants with and without elevated depression symptoms to examine whether ATQ-8 adequately discriminates between groups.

---

RESULTS

Factor Structure

Confirmatory factor analysis of the unifactorial model demonstrated adequate fit: RMSEA = .10 (90% CI = .070–.138), CFI = .98, NNFI = .97, SRMR = .048. Although RMSEA was marginally elevated, other fit indices were excellent, suggesting the unifactorial model is appropriate. Factor loadings for all 8 items were significant (p < .001) and substantial, ranging from λ = .67 to λ = .82, indicating each item adequately contributes to the latent construct.

Measurement Invariance

The ATQ-8 demonstrated strict measurement invariance across all examined demographic groups:

• Across samples: Configural, metric, and scalar invariance were confirmed (ΔCFI < .01 for all comparisons). This indicates university students and general population participants interpret the ATQ-8 similarly.

• By sex: Strict invariance was confirmed between males and females (ΔCFI = .008), suggesting ATQ-8 functions equivalently across sexes.

• By age group: Four age groups were formed (< 25 years, 25-34, 35-49, ≥ 50). Metric and scalar invariance were confirmed (ΔCFI < .01), though configural invariance was marginally different. Subsequent analyses suggest this resulted from sampling variability rather than substantive structural differences.

• By educational level: Participants with secondary vs. higher education were compared. Strict invariance was confirmed (ΔCFI = .006), indicating educational level does not affect instrument interpretation.

These findings are particularly important for professional practice, demonstrating that ATQ-8 can be used equivalently across demographic groups without requiring differential corrections.

Internal Reliability

Internal consistency was excellent across both samples:

• Sample 1 (students): α = .89, ω = .89
• Sample 2 (general population): α = .89, ω = .89
• Combined sample: α = .89, ω = .89

Both coefficients (Cronbach's alpha and McDonald's omega) were identical and indicated excellent reliability, exceeding the .80 criterion. Concordance between alpha and omega suggests absence of serious error variance problems or asymmetry in factor loadings.

Convergent Validity

The ATQ-8 showed significant correlations in expected directions with convergent measures:

• DAS-R: r = .68, p < .001 (substantial positive correlation)
• DASS-21 Depression: r = .71, p < .001 (strong positive correlation)
• DASS-21 Anxiety: r = .54, p < .001 (moderate positive correlation)
• DASS-21 Stress: r = .59, p < .001 (moderate positive correlation)
• AAQ-II: r = .58, p < .001 (moderate positive correlation)
• CFQ (Cognitive Fusion): r = .61, p < .001 (moderate-substantial positive correlation)
• GPQ (Pliance): r = .47, p < .001 (moderate positive correlation)

The strongest relationship was with the depression subscale of DASS-21, consistent with theory positioning negative automatic thoughts as central to depression. Correlations with other constructs were moderate but significant, supporting the relative specificity of ATQ-8 for measuring negative thoughts specifically.

Discriminant Validity

As expected, ATQ-8 showed negative correlation with SWLS:

• SWLS: r = -.52, p < .001 (moderate-substantial negative correlation)

This correlation was substantial but smaller than correlations with depressive/anxiety symptoms, providing evidence of discriminant validity: ATQ-8 measures negative thoughts more specifically than general well-being, though these constructs share common variance.

Group Discrimination Analysis

Participants were categorized based on DASS-21 depression subscale scores, identifying those with elevated depressive symptoms (n = 187) vs. normal (n = 961). One-way ANOVA showed ATQ-8 significantly discriminated between groups:

• Elevated depression group: M = 24.3, SD = 7.2
• Normal depression group: M = 14.8, SD = 5.1
• F(1, 1146) = 142.3, p < .001, Cohen's d = 1.38

The effect size was large (d > 0.80), indicating ATQ-8 possesses clinical discriminative utility for identifying participants with elevated depressive symptoms.

---

DISCUSSION AND CONCLUSIONS

Main Findings

This study provided comprehensive evidence validating the ATQ-8 as a brief, reliable, and valid instrument for measuring frequency of negative automatic thoughts in Spanish nonclinical samples. Findings were consistent across two independent samples of adequate sizes (total N = 1,148), providing confidence in result replicability.

The unifactorial structure of ATQ-8 was confirmed, consistent with the original English version and previous validations in other languages. Model fit was good (CFI = .98, NNFI = .97), though RMSEA was marginally elevated (0.10). This pattern is typical of highly interrelated items conforming brief measures, where limited degrees of freedom can yield artificially elevated RMSEA despite adequate substantive fit. Other absolute and relative fit indices were excellent, justifying concluding the unifactorial model is appropriate.

Demonstration of strict measurement invariance across multiple demographic characteristics (sample, sex, age, education) is particularly relevant. This invariance indicates differences in ATQ-8 scores between groups reflect true differences in frequency of negative automatic thoughts, not artifacts of scale differences or interpretation. This supports instrument use in comparative research and multi-group evaluation.

Internal reliability was excellent (α = ω = .89), substantially exceeding recommended standards (.80) and equaling performance of the original 30-item version. This reliability supports that 8 items function cohesively measuring a unitary construct, without excessive redundancy or items functioning independently.

Convergent validity was robust, with substantial correlations to dysfunctional attitude measures (DAS-R, r = .68), depressive symptoms (DASS-21 Depression, r = .71), psychological inflexibility (AAQ-II, r = .58), and cognitive fusion (CFQ, r = .61). These correlations confirm ATQ-8 measures a theoretically related construct to dysfunctional cognitive and emotional processes. The strongest relationship with depressive symptoms is consistent with cognitive models of depression emphasizing negative automatic thoughts' role.

Discriminant validity was adequate, with moderate-substantial negative correlation to life satisfaction (r = -.52). Although smaller than correlations with depressive symptoms, this confirms ATQ-8 measures something conceptually distinct from general well-being, though these constructs share variance.

Group discrimination analysis was particularly informative. ATQ-8 demonstrated large effect size (d = 1.38) when comparing participants with and without elevated depressive symptoms. This discriminative capacity suggests clinical utility: the instrument can help identify individuals with negative cognitive burden.

Limitations

The authors acknowledge several important limitations:

1. Nonclinical samples: Participants were recruited from nonclinical contexts (university students and online general population). While establishing normality parameters, this does not address ATQ-8 functioning in clinically diagnosed samples. Additional validation in clinical samples of depression, anxiety, and other disorders is required.

2. Cross-sectional design: Data were collected at single time point, precluding assessment of predictive validity, change over time, or causal relationships. Longitudinal studies would be valuable to examine whether negative automatic thoughts predict symptom emergence.

3. Marginally elevated RMSEA: Although other indices were excellent, the .10 RMSEA is technically at the upper limit of acceptability (.10). While typical of brief measures with few degrees of freedom, future validation could examine whether alternative models (bifactor, ESEM) provide superior fit.

4. Lack of clinical criteria: No clinically diagnosed group was included to establish clinical cutoffs or criterion-referenced validity.

5. Minor methodological limitations: Second sample recruitment via internet could introduce self-selection bias. Demographic diversity, while greater than university samples, was not fully representative.

Theoretical Implications

This study supports conceptual assumptions underlying the ATQ and its brief version. Negative automatic thoughts operate as a measurable, coherent construct, correlating with psychopathology indicators per theory. ATQ-8's unitary structure supports conceptions of negative thinking as one-dimensional phenomenon rather than multidimensional, though note the original ATQ-30 contains subscales.

The strongest relationship with depressive symptoms compared to other constructs supports cognitive models of depression. Moderate correlation with psychological inflexibility (AAQ-II) and cognitive fusion (CFQ) suggests negative automatic thoughts are not purely cognitive but imbricates in broader emotion regulation and functional relation with thinking processes, consistent with contextual behavioral frameworks.

Practical Implications

For Spanish clinicians and researchers, this study validates ATQ-8 as practical instrument for rapid assessment of negative automatic thoughts. Its brevity (8 items, approximately 2-3 minutes administration) makes it accessible for repeated use in treatment monitoring, initial evaluations, and population screening. Reliability and validity equivalent to the 30-item version suggest no substantial utility loss.

Demonstration of multi-group invariance supports use in comparative research. Researchers can compare scores across males and females, age groups, and educational levels without requiring differential corrections.

Discriminative capacity between groups with and without depressive symptoms suggests value for identifying at-risk individuals who might benefit from interventions targeting thought modification.

Future Directions

Additional research is recommended in: (1) validation in clinically diagnosed samples, establishing clinical cutoffs; (2) temporal stability studies (test-retest) across clinically relevant periods; (3) sensitivity to change in response to cognitive-behavioral interventions; (4) validation in adolescents and older adults; (5) psychometric network analysis to examine item relationships; and (6) longitudinal studies to estimate predictive validity.

---

Significance and contribution

This study contributes to the measurement of negative automatic thoughts by validating the ATQ-8 in Spanish-speaking contexts, demonstrating that the instrument maintains its unidimensional structure, excellent reliability, and convergent validity in Spanish samples. By establishing strict measurement invariance across multiple demographic characteristics, the work provides confidence for group comparisons and use in comparative research. Findings regarding discrimination between participants with and without elevated depressive symptoms suggest clinical utility for identifying negative cognitive burden in Spanish-speaking contexts.