Psychometric properties of the Reward Probability Index in a Colombian sample

Full reference: Reyes-Buitrago, P. L., Bianchi, J. M., Suárez-Falcón, J. C., & Ruiz, F. J. (2023). Psychometric properties of the Reward Probability Index in a Colombian sample. Revista Latinoamericana de Psicología, 55, 1-9. https://doi.org/10.14349/rlp.2023.v55.1

Study type: Scale validation and adaptation / Factor structure study

Background and objectives

Response-Contingent Positive Reinforcement (RCPR) represents a critical factor in psychological well-being and mental health. Decreased RCPR is associated with depressive symptoms, apathy, and passivity—core features of clinical depression from behavioral perspectives. Previous research has documented the empirical relationship between low RCPR and emotional distress, which grounds evidence-based interventions such as behavioral activation (BA).

The Reward Probability Index (RPI) was developed by Carvalho et al. (2011) to assess the ability to experience environmental rewards. Although previous studies documented the psychometric properties of the RPI in English- and French-speaking populations, no rigorous empirical evidence existed for its properties in Spanish-speaking contexts. A Spanish version was translated by Collado et al. (2014) for a preliminary efficacy study of BA in U.S. Latino populations, but this translation had not undergone formal psychometric analysis.

The objective of the present study was to systematically analyze the psychometric properties, factor structure, and measurement invariance of the RPI in a large sample of Colombian adults. This research represents the first formal analysis of the Spanish version of the instrument, enabling its reliable use in research and clinical practice in Spanish-speaking countries.

Method

Participants

The sample consisted of 1129 adult participants (M = 27.56 years, SD = 8.92; 74.2% women, 25.3% men, 0.44% did not report gender) from 28 Colombian departments distributed across the five regions of the country (Bogotá, Atlantic, Central, Eastern, and Pacific). Participants were voluntarily recruited through snowball sampling via social networks and the Internet and provided informed consent. Twenty-six participants who did not meet age criteria and 6 who did not complete the survey were excluded. Exclusion criteria included presence of psychosis or current diagnosis of severe psychiatric or neurological disorder; no participants reported these conditions. Participants received no financial compensation.

Demographically, participants were primarily employed (39.4%), single (72.9%), with university education (48.9%), at a medium socioeconomic level (42.9%), and from the Bogotá region (46.5%). Thirty-four participants (3%) reported a history of hospitalization for mental health reasons.

Instrument(s) under study

Reward Probability Index (RPI). The RPI is a 20-item self-report measure designed to approximate RCPR by evaluating the magnitude of environmental rewards. The instrument assesses four dimensions of RCPR across two factors: Reward Likelihood (11 items) and Environmental Suppressors (9 items). Participants responded on a 4-point Likert scale (1 = strongly disagree; 4 = strongly agree) regarding the past few months, with higher scores indicating greater reward levels. The original instrument (Carvalho et al., 2011) demonstrated strong internal consistency (α = .88) and good two-week test-retest reliability (r = .69).

For this study, the Spanish version of the RPI was adapted. In Phase II, a pilot test was conducted with 60 undergraduates to detect clarity problems in items, with appropriate adjustments made based on observations.

Other outcome measures

Environmental Reward Observation Scale (EROS). A 10-item instrument that assesses perception of environmental reward on a 4-point Likert scale (range: 10-40). It detects the magnitude of reinforcing events, availability of environmental reinforcement, and the individual's ability to elicit such reinforcement. The Colombian validation (Valderrama et al., 2016) reported appropriate internal consistency (α = .87) and a unidimensional structure explaining 46.82% of variance.

Behavioral Activation For Depression Scale (BADS). A 25-item instrument on a 7-point Likert scale that assesses behaviors responsible for change according to the BA model. Comprises four dimensions: Activation, Avoidance/Rumination, Work/School Difficulties, and Social Impairment. Scoring range: 0-150. Higher scores indicate greater activation. The Colombian validation (Leal et al., 2016) showed appropriate internal consistency (α = .92).

Zung's Self-rating Depression Scale (Zung-SDS). A 20-item instrument on a 4-point Likert scale (range: 20-80). Scores ≥40 indicate probable clinical depression. The Colombian validation (Campo-Arias et al., 2005) reported adequate internal consistency (α = .85), sensitivity of 88.6%, and specificity of 74.8%.

Data analysis

Data were examined prior to statistical analyses with no missing values found. The sample was randomly divided into two subsamples (n₁ = 563; n₂ = 566) using SPSS 19.

For the first subsample (EFA): Exploratory factor analysis was conducted using Factor 10.5, with polychoric correlation technique, robust DWLS estimation method with direct oblimin rotation. Optimal implementation of parallel analysis with minimum rank factor analysis and Hull method using robust CFI and RMSEA were applied to determine the number of factors to retain. Unidimensional Congruence (UniCo), Explained Common Variance (ECV), and mean of item residual absolute loadings (MIREAL) indexes were computed to assess essential unidimensionality (values >0.95 and >0.85 in UniCo and ECV respectively suggest unidimensionality; for MIREAL, <0.30 suggests unidimensionality). Corrected item-total correlations were obtained to analyze item discrimination.

For the second subsample (CFA): Robust DWLS method with polychoric correlations was adopted using LISREL 8.71 to conduct confirmatory factor analysis of single-factor and two-factor models. Satorra-Bentler chi-square test and goodness-of-fit indexes were computed: RMSEA, CFI, NNFI, SRMR, ECVI, PNFI. Criteria: RMSEA <0.05 (excellent fit), <0.08 (good fit); CFI/NNFI >0.95 (excellent fit), >0.90 (good fit); SRMR <0.05 (good fit), <0.08 (reasonable fit).

Reliability and validity: Composite Reliability (CR) coefficient was calculated for construct reliability (>0.70 indicates high reliability). Convergent validity criteria: statistically significant factor loadings (>0.40, ideally >0.70); CR >0.70; AVE ≥0.50. For discriminant validity: inter-construct correlations <0.80; √AVE of each factor > inter-construct correlations.

Measurement invariance: Multi-group confirmatory factor analysis on total sample (N=1129) comparing nested models (baseline, metric, scalar, strict). Criteria: ΔRMSEA <0.01; ΔCFI, ΔNNFI ≥-0.01.

Internal consistency: Cronbach's alpha and McDonald's omega were computed using MBESS package in R with 95% percentile bootstrap confidence intervals. Pearson correlations between RPI and other scales interpreted according to Cohen (1988) guidelines: r 0.10-0.30 (small), 0.30-0.50 (medium), >0.50 (large); >0.70 (very strong, necessary for convergent validity).

Results

Psychometric quality of items

All 20 RPI items demonstrated good discrimination in the first subsample (n=563), with corrected item-total correlations ranging from .32 (Item 19) to .64 (Item 11). Internal consistency was excellent: Cronbach's α = .88 (95% CI [.87, .89]); McDonald's omega = .88 (95% CI [.87, .89]) for the total sample. Both dimensions also showed good internal consistency: Reward Probability: α = .85 (95% CI [.84, .87]), ω = .86 (95% CI [.84, .87]); Environmental Suppressors: α = .83 (95% CI [.82, .85]), ω = .83 (95% CI [.82, .85]).

Validity evidence based on internal structure

Dimensionality. Exploratory factor analysis confirmed that two factors should be extracted, explaining 51% of total variance. Parallel analysis suggested two factors; the Hull method applied to robust CFI and RMSEA also recommended retaining two common factors to obtain excellent fit (CFI = .970; RMSEA = .067). Factor loadings were adequate: Reward Probability from .479 (Item 1) to .864 (Item 15); Environmental Suppressors from .336 (Item 3) to .887 (Item 16). Unidimensionality indexes (UniCo = .925, ECV = .765, MIREAL = .314) suggest that data should not be treated as essentially unidimensional.

Confirmatory factor analysis. In the second subsample (n=566), the single-factor model showed poor fit (χ²/df = 9.724, p < .001; RMSEA = .124, CFI = .885, NNFI = .871; SRMR = .111). In contrast, the two-factor model produced clearly superior results (χ²/df = 3.440, p < .001; RMSEA = .066, CFI = .968, NNFI = .964; SRMR = .067). ECVI and PNFI values also supported the superiority of the two-factor model.

Construct reliability and convergent and discriminant validity. CR values were greater than .70 for both factors (Reward Probability = .89; Environmental Suppressors = .87), indicating high construct reliability. Regarding convergent validity, all factor loadings were statistically significant, with six standardized loadings >0.70 and 13 >0.40. Only Item 19 showed a loading slightly below the cutoff (.39). AVE values were below the .50 cutoff (Reward Probability = .44; Environmental Suppressors = .43); however, convergent validity is considered acceptable when CR >0.70 and AVE >0.40, conditions which were met. For discriminant validity, the inter-construct correlation was <0.80 (r = .55 between Reward Probability and Environmental Suppressors), and the square roots of AVE (√AVE Reward Probability = .66; √AVE Environmental Suppressors = .66) exceeded the inter-construct correlation.

Measurement invariance

Measurement invariance analysis (N=1129) supported metric, scalar, and strict invariance across gender, as changes in RMSEA, CFI, and NNFI were below .01.

Validity evidence based on relationships with other variables

Correlations of the RPI with other relevant constructs were theoretically coherent and statistically significant (p < .001). The RPI showed strong correlations with measures of activation and avoidance (BADS Activation = .57; BADS Avoidance/Rumination = -.60), environmental reward (EROS = .81), and depression (Zung-SDS = -.75). These correlations support the convergent construct validity of the RPI, in both total score and in both factors separately.

Discussion and conclusions

The findings of this study support the adequacy of the two-factor RPI model in the Colombian population. The two-factor structure (Reward Probability and Environmental Suppressors) demonstrated excellent fit to the data, replicating the structure reported by Carvalho et al. (2011) in U.S. general population and Wagener & Blairy (2015) in the French version. Unidimensionality indexes rejected a single-factor model, validating the importance of maintaining both factors.

This is the first study providing empirical evidence of RPI measurement invariance across gender, extending previous findings by Voss et al. (2021) who documented factorial equivalence across ethnic groups and education levels. Measurement invariance is relevant because it allows valid comparison of RPI scores across different groups of participants. Additionally, construct reliability was high (CR > .87) and internal consistency excellent (α = ω = .88), similar to previous studies.

A significant strength was convergent construct validity with high-magnitude correlations (>0.70) with conceptually related constructs: perception of environmental reinforcement (EROS, r = .81), activation (BADS, r = .57), and depression (Zung-SDS, r = -.75). These relationships coherent with theory support the use of the RPI as a clinical indicator, especially in intervention of depression and other disorders, as well as in research seeking to recognize availability of reinforcement and exposure to aversive events.

Important limitations include: (1) psychometric properties were analyzed in a nonclinical sample; future studies should include clinical samples with higher prevalence of low RCPR; (2) the 3:1 ratio of women to men allowed better generalization to female population, although this was mitigated by demonstration of measurement invariance; (3) convergent validity evidence was obtained only with depression-related measures; exploring relationships with constructs such as experiential avoidance and behavioral inhibition is recommended; (4) analysis was conducted in a single Spanish-speaking country; studies in other Spanish-speaking nations are needed; (5) data on RPI sensitivity to psychological interventions such as BA were not obtained; future studies should evaluate this property.

Significance and contribution

This study represents the first rigorous psychometric evaluation of the Spanish version of the RPI, establishing its validity and reliability for use in Spanish-speaking contexts. The demonstration of stable two-factor structure, high reliability, and significant convergent validity supports its use in research and clinical assessment in Spanish-speaking countries, particularly in behavioral activation interventions for depression. By validating a measurement tool of response-contingent positive reinforcement with solid psychometric properties, the study enriches the arsenal available to researchers and clinicians in Latin American populations, enabling more precise assessments of a construct central to behavioral theory of depression.