Metrics
The Happiness Initiative

Survey Methodology ... The Happiness Initiative Survey Methodology for the survey developed by the Personality and Well-Being Laboratory at San Francisco State University.

Burgess COMMENTARY

Peter Burgess

Survey Methodology ... The Happiness Initiative Survey Methodology for the survey developed by the Personality and Well-Being Laboratory at San Francisco State University.

There is a growing interest in developing a more accurate measurement of societal progress and well-being, because the typical measure that was used until now, GDP, cannot distinguish when economic activities have a good or bad impact on well-being. That is, an overreliance on GDP for measuring societal progress is limited because such an approach does not take into account the hidden costs of economic development and devalues importance of factors of well-being (such as natural capital, knowledge, health, and social capital). When measuring well-being we should not only look at observable factors (e.g., health, socioeconomic status, material wealth, etc.) or objective well-being but we should also consider psychological factors (e.g., happiness, domain satisfaction, quality of life, etc.) or subjective well-being. This research extends the work of Easterlin (1974, 1995, 2001) and other scholars who argue that monetary gains have relatively small effects on quality of life. Thus, it is useful to measure subjective well-being independently of objective well-being.

Because people are considered to be the best judges of their own happiness (Frey & Luechinger, 2007), subjective well-being is typically measured by asking a representative sample of individuals to provide a judgment of their global happiness by answering a single question (General Social Survey, World Values Survey, etc.) or a multiple-item survey (Satisfaction with Life Scale, Diener & Pavot, 1993; etc.). However, a more adequate and holistic way to measure progress is using a non-monetary multidimensional approach that measures satisfaction and advancements across various life domains (e.g., community, time balance, culture, governance, social, economic, and environmental). For these reasons, the Gross National Happiness survey developed by the Happiness Initiative methodologically parallels the non-monetary approach to the measurement of progress employed by the Happy Planet Index (HP; Marks et al, 2006) and Gross National Happiness (GNH; Centre for Bhutan Studies; Ura & Galay, 2004) where progress is measured by an increases in Net Positive Development while avoiding Negative Transitory Cycles.

To accomplish the goal, the Personality and Well-being lab at San Francisco State University (see more about the work at this lab here: https://sites.google.com/site/howellhappinsslab/home) developed and validated the current Gross National Happiness survey used by the Happiness Initiative in five phases. In Phase I and II we developed an hour long survey (440 total items) in two steps. First, for Phase I, we used the valid data from the original opt-in Seattle survey to select items which had highest corrected-item correlations with the constructs they were intending to measure. This resulted in the selection of about half of the original items. Second, in Phase II, we searched for published surveys that measured the same constructs and selected items which (based on face validity) appeared to measure the domains of subjective well-being that have the greatest impact on quality of life (i.e., psychological well-being, physical health, time balance, community vitality, social connectedness, education, cultural vitality, environmental quality/access to nature, democratic governance, material well-being, and work experience). To better assess each of these domains approximately 250 items were selected and adapted from the following validated community and large-scale national quality-of-life surveys: The Gross National Happiness Abridged Survey, Detroit Area Survey 2001, General Social Survey 2002, The World Health Organization Quality of Life Survey, Centre for Economic Performance Recommendations for Measuring Subjective Well-Being, The European Social Survey, and University of Michigan and ABC News/Money Magazine Consumer Confidence Survey. We also asked participants to complete Diener’s Flourishing scale, Kasser’s Time / Material balance scale, Dolan, Layard, and Metcalfe’ s domain satisfaction suggestion, and numerous material well-being scales.

The hour long online survey was emailed the Sustainable Seattle and Take Back Your Time email lists and was received by more than 10,000 people in total; a total of 515 individuals volunteered their time to complete the survey. At the end of the survey, we asked participants to provide any comments on how to improve the survey. The goal of Phase II was to use this data in order to select the best items per domain. The criteria that was those items which: (1) had the strongest loadings within each of the 10 factor structures (i.e., we conducted a factor analysis with each of the items proposed to measure each well-being construct based on the item generation phase), (2) had the highest corrected-item total correlation, (3) most strongly correlated with the original items from the Seattle opt-in well-being constructs. We also strongly consider the comments provided by the volunteers as we selected items based on the readability of each item. Thus, in Phase II we reduced the number of items per domain to 15 by using factor analysis, corrected item-total correlations, reliability analyses, convergent correlations, and participant feedback.

Next, for Phases III (n = 404), we posted the modified survey (the best 150 items from Phase II) on Amazon.com’s Mechanical Turk (MTurk) web-site – MTurk participants are paid a nominal fee for their participation. Also, to better test for convergent validity the survey asked each participant to “please indicate [their] level of satisfaction with…” (1 = extremely dissatisfied; 9 = extremely satisfied) the operational definitions of the 10 GNH domains as defined by the Happiness Institute. For example, the first rating asked participants to rate their level of satisfaction with “Your mental well-being (e.g., your life satisfaction and sense of optimism, self-esteem, and competence);” the second rating asked participants to rate their level of satisfaction with “Your physical health (e.g., consider your exercise, sleep, nutrition).” The goal of Phase III was to reduce the number of items per domain to five (ideally) with the criteria being that the items would maximize the internal consistency and predictive validity of the construct while also selecting items that would consider the breadth of coverage (i.e., attempting to not be redundant in content) and were not evaluatively extreme as well as considering the face validity and readability of each item. We began by examining the internal consistency of each of the 10 GNH domains in order to select the fewest items (minimum 5 items) to ensure the reliability of each domains – that is, we attempted to measure the construct in five items; however, for those constructs that were not reliable (i.e., their alpha coefficients was not greater than .70) with five items, we continued to add additional items (based on the corrected item-total correlations) until the construct met the minimum threshold for reliability. To test the robustness of this new short GNH survey we, for Phase IV (n = 133), reposted the survey on Mturk (excluding any of the original survey takers) with one additional modification: we altered all the response scales to all be 5-point Likert scales. We then examined all 10 of the domains to ensure: (1) they formed a single factor (using factor analysis), (2) they were internally consistent (their alpha coefficients was greater than .70), and (3) the sum of the scale significantly (p < .05) correlated with the satisfaction rating at the beginning of the survey it was intended to predict (for example, the domain sum of the community vitality items correlated with the satisfaction with community rating at the beginning of the survey).

Finally, for Phase V, we recruited a nationally representative sample of adults through SurveyMonkey.com. The 578 participants completed the three global ratings of the well-being, the domain satisfaction questions, and the final version of the HI GNHI survey. In addition to determining the average levels of well-being for each of the satisfaction questions and each well-being construct we confirmed that each domain (1) formed a single factor, (2) was internally consistent, and (3) was correlated with the satisfaction rating it was intended to predict. Thus, the final version of the HI GHNI survey demonstrated the expected factor structure, internal consistency, and predictive validity in three separate samples (Phases III, IV, and V).