Assessing religion and spirituality in a cross-cultural sample: development of religion and spirituality items for the Global Flourishing Study

ABSTRACT

The Global Flourishing Survey (GFS) was initiated to provide an open-access, longitudinal study of health and well-being using a participant panel from 22 countries around the world. At the core of the GFS are questions on religion and spirituality—notoriously difficult to assess in a cross-cultural context. Additionally, the longitudinal aspect will allow for tests of within-person change over time. In developing the religion and spirituality items, we received suggestions and feedback from over 130 scholars, and the items underwent several rounds of peer-review by experts in the field. The preliminary survey items were also made publicly available to gather more feedback. Experts at Gallup then translated candidate items to ensure consistency across languages/cultures. Here, we present the results of cognitive interviews of 230 participants in 22 religiously diverse countries regarding the efficiency, efficacy, and difficulty of our candidate items. In the spirit of open science, we wish to share our findings from the interviews and provide recommendations regarding Likert scale usage, item specificity, assessment of God representations, and inclusivity when assessing religion and spirituality across cultures. In this, we aim to assist other researchers and support confidence in the reliability and validity of GFS data when it becomes publicly available.

KEYWORDS:

• Global Flourishing Survey
• cross-cultural
• measures
• religion
• religious experience

The current study presents the results of pre-pilot cognitive interviews collected for the Global Flourishing Study (GFS), a multi-wave, longitudinal, international survey intended to investigate changes in religion and spirituality and how these changes are related to health, character development, and well-being from a global perspective (Crabtree et al., 2021). Ultimately, the data collected by the GFS will be made open-access for the benefit of all researchers to address these and numerous other questions.

Background

In recent years, it has become apparent that the participants in most psychological research are “WEIRD”: Western, Educated, Industrialized, Rich, and Democratic (Henrich et al., 2010). Many findings that were previously thought to represent human universals have turned out to be specific to WEIRD cultures, such as notions of fairness, attributions of others’ behavior (e.g., the Fundamental Attribution Error), and behavior in economic games (Muthukrishna et al., 2020). Additionally, much of the psychology of religion has relied heavily on Christian samples, and many influential findings in these samples do not seem to apply to other religious cultures. For example, it has been proposed that analytic thinking and interest in science are associated with religious unbelief. Recent cross-cultural studies have shown that this association is non-significant in other cultures (Gervais et al., 2018; McPhetres et al., 2021).

Understanding how religious traditions (and what aspects of these traditions) positively predict human flourishing is critical for both basic and applied science. Social support and engagement in religious communities can improve physical and mental health, provide meaning in life, and be a source for meaningful close relationships (VanderWeele, 2017). However, while religious service attendance seems to be a dominant predictor of flourishing in Western cultures (Balboni et al., 2022; VanderWeele, 2017), it is not clear the extent to which this is true in other parts of the world, or which aspects of religion promote flourishing across religious traditions or national cultures, or whether stable across time. To answer these questions, cross-cultural (i.e., both WEIRD and non-WEIRD) and religiously-diverse research is critical (Rad et al., 2018).

Cross-cultural and multi-wave data sets afford tremendous opportunities to investigate the associations between religion, spirituality, and well-being. However, in addition to very limited measures of religiosity (e.g., specific beliefs, values, practices), there is often inconsistent administration of these items across time periods or cultures, and international studies very rarely, if ever, survey the same individuals across time (Scott et al., 2019). Thus, researchers are typically at a loss to infer causality or to rigorously investigate change (and the causes of change) over extended time periods.

Assessing religiosity and spirituality across cultures can be difficult for many reasons including translation difficulties, variability among field workers in different locations (e.g., motivation, compensation, training), national laws and political systems, and respondents’ familiarity with the process of survey participation (Smith, 2017). Thus, it is critical that the survey be overseen by both global and local experts. Moreover, indicators of religiosity developed in Western or Christian cultures may not reflect the same degree of or reasons for religiosity in other cultures. As one example, researchers might expect that people who pray more frequently tend to be more religious. However, a Muslim and a Christian who each pray twice per day do not necessarily share the same level of devotion because Muslims are obligated to pray five times each day. Even commonly used measures of religious orientation are not invariant across cultures (Cohen et al., 2017).

The Global Flourishing Study

The Global Flourishing Study (GFS) was initiated to address the lack of international, longitudinal data regarding religion and spirituality, and great care was taken in developing the religion and spirituality questions. As a starting point, we used the Brief Multi-Dimensional Measure of Religion/Spirituality (BMMRS; Fetzer Institute, 1999; Idler et al., 2003) in generating an initial set of items. The BMMRS had been crafted by a panel of social and biomedical science experts on the study of religion. For the present survey, a Religion/Spirituality advisory group was formed to further select and refine the items for the GFS. In panel discussions, alternative items were proposed or adapted when those in the BMMRS were either not sufficiently general to be applicable for use among diverse global religions, or when further research in the field had suggested other wording was preferable, or for domains of religion not explicitly addressed by the BMMRS.

The second phase of selecting religious and spirituality items (and broader survey design) involved soliciting recommendations for additional items from sociologists, political scientists, psychologists, and economists around the world. Then, in phase three, the GFS directors sent out a preliminary version of the complete survey and received comments and criticisms—again, from scholars around the world. The fourth phase of survey development took place at an in-person meeting consisting of a small group of senior members of Gallup who were experts on translation and cross-cultural issues, with expertise in well-being assessment. Next, all items proposed for the survey were posted on the Human Flourishing Program website at Harvard University and on a blog posting on Psychology Today, as well as notifications to numerous research list-serves to allow further opportunities for open feedback from anyone who would desire to comment. These comments and concerns were addressed in a final version of the survey (e.g., questions that referred to God were revised to apply to all faiths; revisions were made to better assess the beliefs and practices of diverse religious traditions including those who identify as spiritual but not religious). The final draft of the survey also included other related measures used for “cognitive interviews” administered in the early months of 2021.

Cross-cultural cognitive interviews

As researchers consider the influence of religion and spirituality across cultures, cognitive interviewing can provide a valuable tool for inquiry at the initial stage. Cognitive interviewing is a qualitative approach that uses both psychological and measurement theory to analyze how participants interpret and respond to self-report items (Drennan, 2003). In a cognitive interview, trained interviewers ask probing and clarifying questions in a semi-structured format (Willis, 2015). For example, after a participant answers a structured survey item, the interviewer might ask them to think about how difficult it was to answer that item or to consider how well the item captured the intended construct. These qualitative responses are used to enhance or modify items for future use. Cognitive interviews are commonly used in preparing health research (Drennan, 2003), but they have also been used in other fields (Willis, 2015).

The cognitive interview data in the present study were gathered in collaboration with Gallup, a global analytics and advice corporation with extensive experience with cross-cultural data collection. The results presented here pertain only to the religion and spirituality items and represent the first round of several planned data collection time points. The interviews were meant to provide preliminary information before launching the survey worldwide (see Lomas, 2021, for more details regarding the development of the GFS. A complete list of the cognitive interview items may be obtained from the first author.

Method

Participants

Participants were 230 adults from 22 countries selected with goals of cultural and religious diversity: Argentina, Brazil, China, Egypt, Germany, India, Indonesia, Israel, Japan, Kenya, Mexico, Nigeria, Philippines, Poland, Russia, South Africa, Spain, Tanzania, Turkey, Ukraine, the United Kingdom, and the United States. Table 1 provides a list of majority religions by country. Ten participants were selected from each country through Gallup survey panels to approximate a broad sample of the population, except for 20 participants from India to sample the greater cultural diversity within that country.

Table 1. Major world religions by country.

Country	Majority religion	Country	Majority religion
Argentina	Christian	Nigeria	Christian and Muslim
Brazil	Christian	Philippines	Christian
China	Unaffiliated & Folk Religion	Poland	Christian
Egypt	Muslim	Russia	Christian
Germany	Christian and Unaffiliated	South Africa	Christian
India	Hindu	Spain	Christian
Indonesia	Muslim	Tanzania	Christian and Muslim
Israel	Jewish	Turkey	Muslim
Japan	Unaffiliated and Buddhist	Ukraine	Christian
Kenya	Christian	UK	Christian and Unaffiliated
Mexico	Christian	US	Christian

Note: Source (Pew Research Center, 2015).

Males and females were evenly represented (51.3% female), and ages ranged from 18 to 74 (M_age = 42.63, SD = 14.45). Participants reported belonging to diverse religious traditions, including Catholic (24.8%), non-Catholic Christian (22.8%), Islam (18.1%), Hinduism (5.2%), Judaism (5.7%), Buddhism (3.3%), no religious affiliation (11.9%), and other religions (8.1%). Participants in China and Egypt were not asked to provide religious affiliation. Participants were from rural areas (29.3%), small towns (21.8%), large cities (36.7%), and suburban areas (12.2%). All participants provided informed consent before proceeding with the survey. Gallup’s IRB approved data collection following full board review (2/19/21).

Materials and procedure

Data were gathered via telephone using cognitive interviewing procedures, which included structured items related to religion and semi-structured questions about item difficulty. The surveys were presented in participants’ native languages and included questions about health, well-being, and virtues described in detail in other publications (Lomas, 2021). The survey began with questions about well-being, followed by community-related questions, demographics, political questions, personality questions, and concluded with the religion/spirituality questions.

There were two survey forms, designated Form A with eight religion items (Table 2) and Form B with 15 religion items (Table 3). Half of the participants in each country in the global sample answered Form A (N = 116), and the other half answered Form B (N = 114). Differences in the two forms can be seen in Tables 2 and 3. For example, most Form A items were answered using a 5-point Likert scale, whereas most Form B items were answered using a 10-point Likert scale. Form B contained additional God representation measures.

Table 2. Questions and percent difficulty^a for religion items on Form A.

#	Item	Interview Scale	Difficulty^a
A1	What was your religion when you were 12 years old?	[List of religions]	4%
A2	How often did your mother attend religious services when you were around 12 years old?	1= More than once a week 2 = Once a week 3 = 1–3 times a month 4 = A few times a year 5 = Never	18%
A40	[COUNTRY'S LEADING RELIGION] is the most common religion in your country. To what extent do you agree or disagree with the following statement: The teachings of [MAJOR RELIGIOUS FIGURE] are very important in my life.	0 = strongly disagree …  10 = strongly agree	12%
	Please tell me the extent to which you agree or disagree with the following statements:
A41	You feel loved or cared for by God, the main god you worship, or the force that guides your life.	1 = strongly agree to 5 = strongly disagree	10%
A42	You find strength or comfort in your religion or spirituality.	1 = strongly agree to 5 = strongly disagree	11%
A43	Your religious beliefs and practices are what really lie behind your whole approach to life.	1 = strongly agree to 5 = strongly disagree	13%
A44	You feel God, a god, or a spiritual force is punishing you.	1 = strongly agree to 5 = strongly disagree	17%
A45	People in my religious community are critical of me or my lifestyle.	1 = strongly agree to 5 = strongly disagree	12%

^a Approximate percentage of participants expressing either some or substantial difficulty answering the item, including persistent problems with scale interpretation.

Table 3. Questions and percent difficulty^a for religion items on Form B.

#	Item	Scale	Difficulty^a
B19	To what extent do you consider yourself a religious or spiritual person?	0–10; 10 = very religious	8%
B20	How often do you attend religious services?	1= More than once/week 2 = Once a week 3 = 1–3 times/month 4 = A few times a year 5 = Never	7%
B21	How often do you pray or meditate?	1 = More than once/day 2 = About once a day 3 = Sometimes 4 = Never	6%
B22	Do you believe in one god, more than one god, an impersonal spiritual force, or none of the above?	1 = One god 2 = More than one god 3 = Impersonal spiritual force 4 = none of the above	11%
B23	Do you believe in life after death, or not?	Yes = 1; No = 2	7%
B24	Have you had a profound religious or spiritual awakening or experience that changed the direction of your life, or not?	Yes = 1; No = 2	17%
B25	How often do you read or listen to sacred texts or other religious literature?	1 = More than once/day 2 = About once a day 3 = Sometimes 4 = Never	3%
	How well do you feel that each of the following words describes your view of God, the main god you worship, or the force that guides your life?
B26a	Helping	1 = A great deal to 4 = Not at all	26%
B26b	Forgiving	1 = A great deal to 4 = Not at all	18%
B26c	Commanding	1 = A great deal to 4 = Not at all	17%
B26d	Judging	1 = A great deal to 4 = Not at all	22%
	Please tell me the extent to which you agree or disagree with the following statements:
B27	You feel God, a god, or a spiritual force is punishing you.	0 = strongly disagree to 10 = strongly agree	18%
B28	People in my religious community are critical of me or my lifestyle.	0 = strongly disagree to 10 = strongly agree	20%
B29	Your religious beliefs and practices are what really lie behind your whole approach to life.	0 = strongly disagree to 10 = strongly agree	21%
B30	You tell other people about your religion or spirituality even when they have different beliefs.	0 = strongly disagree to 10 = strongly agree	14%

^a Approximate percentage of participants expressing either some or substantial difficulty answering the item, including persistent problems with scale interpretation.

The religion and spirituality items of the GFS were intended to assess: (1) religious influences in childhood (Form A only), (2) religious practices (Form B only), (3) belief in an afterlife (Form B only), (4) religious or spiritual identity (Form B only), (5) religious commitment (Forms A and B), (6) beliefs about the nature and attributes of God (Forms A and B), and (7) religious reputation concerns (Forms A and B). The three items presented on both Form A and B were: “religious beliefs and practices are what really lie behind [my] whole approach to life,” “[I] feel God, a god, or a spiritual force is punishing [me],” and “People in my religious community are critical of me or my lifestyle.” Participants were also able to respond “I don’t know” or refuse to answer any question.

Assessments of item difficulty

After obtaining an answer to a question, interviewers also rated the extent to which participants had difficulty answering the item, reporting 1 = no difficulty, 2 = some difficulty, or 3 = a lot of difficulty. Interviewers were instructed that no difficulty indicated the respondent answered quickly without needing anything repeated; some difficulty indicated the respondent asked to have the question repeated or thought for more than ten seconds before answering; and a lot of difficulty indicated the respondent asked for the question to be repeated and struggled to provide an answer or the participant thought for more than 15 s before answering. The percentage of participants expressing some or substantial difficulty are shown in Tables 2 and 3. In addition to their own impressions of participants’ difficulties, interviewers asked participants, “Was it easy or difficult to answer this question?” If participants found the question difficult, interviewers were instructed to ask, “Why was it difficult?” or “I noticed that you hesitated. Can you tell me what you were thinking? Tell me more.”

Interviewers asked two additional probing questions of participants completing Form B. First, following the item, “Have you had a profound religious or spiritual awakening or experience that changed the direction of your life, or not?” participants were asked, “In your own words, what is this question asking?” Second, following the block of questions about the attributes of God (i.e., “Helping,” “Forgiving,” “Commanding,” and “Judging”), participants were asked, “Were these questions easy or difficult to answer? If difficult, what made them difficult?” Here, interviewers were instructed to record participants’ responses word-for-word. Probing questions like these are typical in cognitive interviewing methods (Willis, 2015) and provide qualitative data regarding participants’ thought processes during the interview. Interviewers were also asked to record any other relevant notes during the interview process.

Results

The results presented here are meant to provide preliminary and useful information regarding the efficiency, efficacy, and difficulty of the survey items. Statistical power is limited in this pre-pilot data because the sample size for each country is small (n = 10 or 20) and, therefore, inferential statistics are not provided. However, the percentage of responses for each item are provided in Figure 1.

Figure 1. Participant responses to cognitive interview items.

Difficulties by country and religious group

Participants had little or no difficulty with the content of most of the items. Specifically, an average of 86.47% of participants had no difficulty, 10.73% had some difficulty, and only 2.80% had a lot of difficulty. The item with the least difficulty was “religion as a child,” with 96.49% of participants having no difficulty responding to the question.

Chi square tests of independence revealed that the degree of difficulty for each item did not differ significantly across countries or religious groups with several exceptions. First, there were significant differences in difficulty answering B29 regarding intrinsic religiosity (χ² [42] = 64, p = .015), with 100% of the Russian having difficulty, sometimes stating they preferred to say their “worldview” or “spiritual” beliefs (rather than religious beliefs) were behind their whole approach to life. There were also significant differences across religious groups in answering A43 regarding intrinsic religiosity, (χ² [14] = 26, p = .024), with 33% of the Jewish participants having great difficulty. Third, although nearly all participants found B20 regarding religious attendance easy to answer, there were significant differences across countries (χ² [42] = 64, p = .015) with 40% of the Russians having great difficulty. For instance, some questioned whether private practices constitute religious service attendance. Fourth, there were significant differences in difficulty answering B30 assessing participant evangelism, (χ² [14] = 29, p = .010) with 33% of Buddhists having great difficulty. Finally, a one-way ANOVA revealed there were significant differences in difficulty answering the four B26 items regarding God representations, F (21) = 3.29, p < .001, with Ukrainians significantly more likely to have more difficulty than any others except Brazilians.

One recurring difficulty in this cross-cultural study was confusion with Likert scale ratings. There were 66 reported instances of confusion about the scale (e.g., forgetting the scale range, asking for repeating the scale, providing verbal answers incongruent with the numeric rating). As our first recommendation, we suggest that, in populations that are not familiar with Likert scale ratings, researchers may want to consider using either “yes” “no” responses or short Likert scales without reverse coded items.

Difficulties by item

Considering all participants, the most difficulty (>20%) occurred with four items, all on Form B: B29 (intrinsic religiosity; 21.05% had some or a lot of difficulty), B28 (religious criticism; 20.19% had some or a lot of difficulty), B26a (God is helping; 26.32% had some or a lot of difficulty), and B26d (God is judging; 21.93% had some or a lot of difficulty). We discuss each of these items below.

Items B29/A43; intrinsic religiosity

Seventeen participants struggled to answer the extent to which “Your religious beliefs and practices are what really lie behind your whole approach to life,” a single item recommended to assess intrinsic religiosity (Gorsuch & McPherson, 1989). Some expressed difficulty differentiating the importance of beliefs and practices. One Russian respondent commented, “It’s about the ten commandments … it’s very difficult to comply with religious laws; religious beliefs are not very clear to us; better about morality, conscience, etc.” Another commented, “Worldview, yes—but religious? Religion is something from the outside.” Similar comments were made in other countries. This is consistent with research and theory showing that the centrality of belief versus practice varies between religious groups (Cohen et al., 2003). Nevertheless, this item was retained as a measure of religiosity in the final version of the GFS.

Items B28/A45; religious reputation

The religious reputation item stated, “People in my religious community are critical of me or my lifestyle.” We found that 18 participants were confused about the boundaries of “religious community” (e.g., should the community be thought of as one’s congregation, people living nearby who are religious, religious vs. non-religious family members). Three participants questioned why a religious community would be critical of one’s lifestyle. Nine participants simply said they did not understand the question at all. Chi-Square tests of independence by country and by religious affiliation revealed no significant differences in difficulty for this item, all p’s > .414. The question was retained in the final version of the survey but the interview responses highlight the need to consider individual differences as to what constitutes the “religious community.”

Items B26a–d; God representations

There were three main concerns regarding the God representation items: (1) uncertainty about God’s attributes (e.g., Helping), (2) reluctance to describe God as Judging, and (3) how God is referred to in the question stem.

God as helping

The cognitive interview data indicated that participants had difficulty answering whether God was “helping.” A possible explanation is that it is sometimes difficult—even for theists—to describe God at all. The difficulty of responding is likely even greater from those in religious traditions that are not principally theistic (e.g., Buddhism). Uncertainty can occur in cases where a believer has not thought much about the nature and attributes of God. Although we had not assessed uncertainty about God’s attributes in our cognitive interviews, one recommendation for future studies would be to add items assessing belief in an “unknown” or “unimaginable” God (Johnson et al., 2019).

Beliefs about God’s engagement in human affairs are another critical dimension of God representations (Froese & Bader, 2010). For instance, an individual might believe that God exists but that God is uninterested in an individual’s concerns and, therefore, not helping. To better assess beliefs about the nature and attributes of God, other researchers may want to include an additional item assessing beliefs about the extent to which God is “involved in human affairs.”

God as judging

There was some confusion or reluctance to describe God as “judging.” For instance, one Indonesian stated, “All is easy to answer, except about judging … because I’ve never felt judged by God.” Another stated, “The first statements were simpler; I already thought about it. But I cannot say about God He who judges.” Another participant said, “It was a bit difficult. Especially the usage of the word panghuhusga [judgment]; I could not connect it to the main point of the question.” Following Johnson et al. (2019), one solution might be to replace the attribute “judging” with another descriptor such as “strict,” “stern,” or “wrathful.”

We note, however, that the questions about God as loving or punishing seemed to be well-understood by participants. The correlation between God as loving and God as punishing in the present research was r (112) = .30, p < .001. Notably, the data for both measures was skewed (−1.0 for loving and 1.0 for punishing) and a cross tabs table showed there were no participants who agreed that God is extremely punishing but not at all loving.

References to God

In addition to variance in beliefs about the attributes of God, people also differ in thinking about the very nature of God. For example, people may think of God as personified versus abstract (Johnson et al., 2023), and believers may be monotheist (e.g., Jews, Christians, Muslims) or polytheist (e.g., folk religions). To be as inclusive as possible when asking about God’s attributes, God’s loving care, and God’s punishment, we had referred to “God, a god, or a spiritual force” on Form B, item B27, and to “God, the main god you worship, or the force that guides your life” on Form A, item A44. The latter wording was also used in asking about God’s attributes on Form B, items B26a–d.

Our intent was for participants to feel free to think about God in whatever way was most familiar to them (i.e., personified, abstract, one God, many gods). However, participants often took additional time to think through these long references to God and what may have been unfamiliar ways of thinking about God. Several participants asked for clarification or suggested wording changes. Some questioned the qualification about the “God you worship.” For example, one participant queried, “The main god you worship? Maybe some do not worship. Maybe for some, it is the god you follow or that you believe in.” Others sometimes objected to the implication that they might think of God in some unfamiliar (e.g., “spiritual force”) or blasphemous way (e.g., “the main god”). In contrast, one participant complained, “These are attempts to humanize God; a higher power does not correspond to this. God is above all these concepts.”

Resolving difficulties

There is no clear and easy solution to alleviate confusion, unfamiliarity, or defensiveness in responding to items about the nature or attributes of the Divine. Moreover, the GFS is designed to assess a wide range of topics in a limited amount of time and a deeper study of beliefs about the nature and attributes of the Divine is beyond the scope of the GFS. We recommend that other researchers continue to refine measures assessing representations of the Divine across cultures. Nevertheless, in the final version of the survey, we included two questions pertaining to the nature and attributes of the Divine: (1) asking whether participants agree or disagree they are “loved or cared for by God, the main god [you] worship, or the spiritual force that guides [your] life;” and (2) asking whether participants agree or disagree they “feel God, a god, or a spiritual force is punishing [you].”

Other qualitative feedback

In addition to the difficulties discussed above, participants also expressed concerns about religion vis-à-vis spirituality, the boundary conditions of “religious services,” and the defining characteristics of “religious experience.”

Religion and spirituality

Our goal in creating survey items was to be as inclusive as possible. In the literature religion or religiousness refers to formal, institutionalized, outward expressions of the sacred, whereas spirituality has been conceptualized by social scientists as the internalized, personal seeking or expression of the Transcendent (Koenig, 2008). Religious and Spiritual-but-not-Religious individuals often have different beliefs and practices (Farias & Lalljee, 2008; Johnson et al., 2018). Therefore, great care was taken to assess beliefs and practices relevant to both religion and spirituality. However, we had not expected participants to sometimes object to including “religion” and “spirituality” in the same question(s). For instance, we asked about the extent to which participants found comfort in their religion or spirituality (A42), the extent to which they considered themselves to be a religious or spiritual person (B19), or whether they shared their religion or spirituality with others (B30).

It may be that religious people did not want to trivialize their religious commitments or belief in one God by agreeing to items about spiritual forces. In contrast, spiritual (but not religious) people sometimes stated they were reluctant to agree with statements that also included religion. Nevertheless, the consensus of the GFS advisory board was that all or nearly all the questions should be inclusive of religion and spirituality. However, the query regarding religiousness and spirituality (B19) was modified in the final version of the survey, “Is religion an important part of your daily life?” allowing for participants to state their religious identity.

Religious services

The term “religious services” (Items A2/B20) also confused participants. People explained that they did not know if we were asking about formal, institutionalized services or whether religious services included visits to temples and shrines, weddings and funerals, or even private gravesite prayers. This confusion highlights, again, the need for researchers to specify boundary conditions. For example, researchers might want to specify “how often do you attend religious services in a church, synagogue, shrine, or mosque” or “how often do you attend religious services led by a religious leader (e.g., priest, rabbi, monk, imam)” or “how often do you attend religious services other than weddings and funerals”—where the specific wording should be designed in accord with the particular research hypothesis.

Religious experience

Perhaps the most interesting finding in the cognitive interview data was that it is unclear whether laypeople and scholars have the same notions about what constitutes a religious experience. We had asked our participants (B24), “Have you had a profound religious or spiritual awakening or experience that changed the direction of your life or not?” A probe followed: “In your own words, what is this question asking?” Regardless of whether they had such an experience, about 24% of the respondents gave what we would ultimately code as incorrect, incoherent, or unexpected explanations of what constitutes “religious experience” (see examples in Table 4). For instance, participants replied that religious experience was “religious education,” “trust in your religion,” “coping with disease,” or “what will happen after death.”

Table 4. Selected participant-supplied conceptualizations of “religious experience.”

If you were in a serious accident, or if you suffered a serious disease that kept you bedridden for a long time

If there is something that I saw in my Christianity walk that maybe damaged me in my religious way or made me backslide and think about changing my Christianity

Not yet. I lost a friend, and I thought, “If I go to church, will it change something for me?” So, I was able to get over it without going to church. Maybe by believing very strongly, I will be able to change my life.

“[It is] about solicitation of religion”

Asking about religion and guidance

Asking if I possess a power that I use for my religion

Atonement for sins or something like that

If I became religious or left the religion

On one occasion, I doubted God, did not want to know anything about him; I felt rage because my son was killed, but I made it through, and we continue with faith

It was disappointment in the institution of the church

Relevance in life—what is it that you really want, want to do, or what God wants

This question asks whether I get God’s guidance, doesn’t it? This guidance means to get deeper understanding about religion, doesn’t it? For example, reading more religious books, having a discussion with a religious person, listening to a religious sermon. By participating in those activities, we usually get God’s guidance.

At face value, the wording of our question seemed straightforward. However, “religious experience” can be idiosyncratic and wide-ranging (Taves, 2020); and the participants in our study did not uniformly view religious experiences as emotionally charged, life-changing, or ineffable. We concluded that, depending upon the specific research hypotheses, researchers may want to specify whether they are inquiring about positive or negative religious experiences or, for instance, whether “religious experience” ought to be construed as non-ordinary/mystical.

Discussion

The Global Flourishing Survey (GFS) was initiated to provide an open-access, international, longitudinal study of health, religion, virtues, and well-being. Some of the primary purposes of the GFS are to facilitate social scientists’ understanding of (1) how and why religion and spirituality change within-person across the lifespan, (2) how and why well-being changes within-person across the lifespan, (3) what relationships exist between religion/spirituality and well-being, and (4) how cultural and contextual factors might influence (1), (2), or (3).

Taking into account previous research and the advice of scholars around the world, we compiled a set of religion and spirituality survey items. However, as previously discussed, great care must be taken to ensure (as far as possible) that participants around the globe clearly understand the questions being asked and that the survey items are understood in the same way across religious traditions and national cultures. Therefore, as one step in the development process, we conducted cognitive interviews of ten (or twenty) participants in each of 22 countries to assess the efficiency, efficacy, and difficulty of the religion and spirituality items.

We identified several difficulties or concerns, from the participants’ point of view, regarding the proposed religion and spirituality survey items. We have addressed these concerns while still considering the recommendations of our panel of experts, and the final, slightly modified items are provided in Appendix A. There is of course tension between asking questions that are applicable across the world’s religion and also in more secular settings versus questions that are perhaps tailored and relevant to specific religious traditions that would not be applicable everywhere. We believe that the results of our cognitive interviews highlight essential considerations for cross-cultural research. Such interviewing is helpful not only in item refinement and modification, but perhaps even more importantly, in understanding the limits of the questions being posed. In the spirit of open science, we provide the following recommendations which we believe may assist others in their own research:

• Because answering religion and spirituality questions can be stressful in some cultures, we recommend using shorter scales (i.e., five-item or binary “yes/no”) with consistent lengths across survey questions to reduce cognitive load.

• Be specific in defining terms in accord with the study hypotheses. For example:

  • Religious community may be transnational, localized, congregational, or familial

  • Religious service may be conceptualized as a wedding, funeral, temple visit, or even personal prayers or meditation

  • Religious experience may be characterized as positive or negative, emotionally charged or neutral, and may or may not be “metaphysical”

• When assessing God representations:

  • Reassure participants that their responses are important even if they do not believe in God so they might be less defensive in responding to certain items. Consider providing an option to respond, “not relevant”

  • If survey length allows, consider assessing uncertainty about God by adding an item such as “unknown” or “unimaginable” (see Johnson et al., 2019) to the list of attributes

  • Consider assessing beliefs about God’s engagement in human affairs (Froese & Bader, 2010; Jang et al., 2018)

  • Be aware that some theists may view God as a Higher Power, Ultimate Reality, or other abstract representation rather than as a personal being (Johnson et al., 2023)

• Be aware that questions simultaneously addressing “religion and spirituality” can be perceived as offensive in cultures where individualistic spirituality is not normalized

  • Consider including at least one item that assesses spirituality apart from religiousness (e.g., “I am a spiritual person” or “My spirituality is very important to me”)

Conclusion

Religion and spirituality are important to people around the world and have been associated with reduced behavioral risks, expanded social support, enhanced coping skills, and a cushion against life stressors via physiological mechanisms (Idler et al., 2003). Moreover, a plethora of research indicates that religion and spirituality significantly influence interpersonal relationships, political persuasions, and social attitudes.

To date, measures of religiosity have been sparse in cross-cultural datasets and there is often inconsistent administration of these items across time and between countries. Moreover, international studies rarely, if ever, survey the same individuals across time (Scott et al., 2019). As a remedy, the longitudinal data gathered around the world from the Global Flourishing Study will provide an open-access resource allowing researchers to investigate within-person changes in religion and spirituality and how these changes are related to health, character development, and well-being from a global perspective. More information regarding accessing the Global Flourishing Study data can be found at https://hfh.fas.harvard.edu/global-flourishing-study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

We thank the John Templeton Foundation (Grant 61665), Templeton World Charity Foundation (#2021-20605), Templeton Religion Trust (#2021-10318), Fetzer Institute (#4351), Baylor University, Harvard University, Paul Foster Foundation, David & Carol Myers Foundation, and Well Being Trust for funding this research.

Appendix A

Final religion and spirituality items on the Global Flourishing Survey

Table

Item #	Intake & Year 1 items	Scale
[A1]	What was your religion when you were 12 years old?	[List of religions]
N1	How often did YOU attend religious services or worship at a temple, mosque, shrine, church, or other religious building when YOU were around 12 years old?	1= At least once a week 2 = 1–3 times a month 3 = Less than once a month 4 = Never
99 [A2]	How often did YOUR MOTHER attend religious services or worship at a temple, mosque, shrine, church, or other religious building when YOU were around 12 years old?	1= At least once a week 2 = 1–3 times a month 3 = Less than once a month 4 = Never
N2	How often did YOUR FATHER attend religious services or worship at a temple, mosque, shrine, church, or other religious building when YOU were around 12 years old?	1= At least once a week 2 = 1–3 times a month 3 = Less than once a month 4 = Never
REL1	Could you tell me what your religion currently is?	[List of religions]
REL2F	If no religion, Which of the following best describes you?	1 = atheist 2 = agnostic 3 = neither
REL3F [B19]	If no religion, Would you say you are spiritual, religious, both, or neither?	1 = spiritual 2 = religious 3 = both 4 = neither
REL2G	Is religion an important part of your daily life?	1 = Yes 2 = No
N3/GFX [A40]	To what extent do you agree or disagree with the following statement: The teachings of [INSERT MAJOR RELIGIOUS FIGURE, ACCORDING TO RESPONSE IN REL1] are very important in my life.	0 = strongly disagree …  10 = strongly agree
GF49D [B26d]	[MAJOR RELIGION OF THE COUNTRY] is the most common religion in this country. To what extent do you agree or disagree with the following statement: The teachings of [MAJOR RELIGION] are very important in my life.	0 = strongly disagree … 10 = strongly agree

Table

Item #	Annual Survey Items	Scale
	In general, how often do you feel connected to a religion or a form of spirituality?	1 = Always 2 = Often 3 = Rarely 4 = Never
GF40 [B20]	How often do you attend religious services?	1= More than once a week 2 = Once a week 3 = One to three times a month 4 = A few times a year 5 = Never
GF41	How often do you participate in groups that are not religious, such as book clubs, sports, or political organizations?	1= More than once a week 2 = Once a week 3 = One to three times a month 4 = A few times a year 5 = Never
GF42 [B23]	Do you believe in life after death, or not?	1 = Yes 2 = No 8 = Unsure
GF43 [B24]	Have you had a profound religious or spiritual awakening or experience that changed the direction of your life, or not?	1 = Yes 2 = No
GF44 [B25]	How often do you read or listen to sacred texts or other religious literature?	1 = More than once a day 2 = About once a day 3 = Sometimes 4 = Never
GF45 B21	How often do you pray or meditate?	1 = More than once a day 2 = About once a day 3 = Sometimes 4 = Never
GF46 [B22]	Do you believe in one God, more than one god, an impersonal spiritual force, or none of the above?	1 = I believe in one God 2 = I believe in more than one god 3 = I believe in an impersonal spiritual force 4 = none of the above 8 = Unsure
GF47 [A43/B29]	My religious beliefs and practices are what really lie behind my whole approach to life.	1 = Agree 2 = Disagree 7 = Not relevant 8 = Unsure
GF48 [A42]	I find strength or comfort in my religion or spirituality.	1 = Agree 2 = Disagree 7 = Not relevant 8 = Unsure
GF49 [A41/B27]	I feel loved or cared for by God, the main god I worship, or the spiritual force that guides my life.	1 = Agree 2 = Disagree 7 = Not relevant 8 = Unsure
GF50 [A44/B28]	I feel God, a god, or a spiritual force is punishing me.	1 = Agree 2 = Disagree 7 = Not relevant 8 = Unsure
GF51 [A45]	People in my religious community are critical of me or my lifestyle.	1 = Agree 2 = Disagree 7 = Not relevant 8 = Unsure
GF52 [B30]	I tell other people about my religion or spirituality even when they have different beliefs.	1 = Agree 2 = Disagree 7 = Not relevant 8 = Unsure