Construct Validity Findings

GCI CONSTRUCT
VALIDITY FINDINGS

A TECHNICAL REPORT ON THE RESULTS OF
A CONSTRUCT VALIDATION STUDY ON THE
GIFTSCOMPASS INSTRUMENT

Dr. Hannah-Hanh D. Nguyen
Industrial-Organizational Psychologist
California State University, Long Beach

Submitted to

Gifts Compass Inc.

August 2007

PURPOSE OF THIS REPORT

This technical report presents the key data analytic findings that may evidence the construct validity of the GiftsCompass Instrument (revised version; GCI-R). Specifically, this report presents the psychometric properties of the GCI-R (item and scale reliability), the factor structure of the measure, as well as the initial evidence for the GCI-R convergent, divergent and discriminant validity.

This technical report consists of two sections: (a) the main text describing and summarizing the data analytic findings, and (b) the appendices presenting the detailed results. An e-file of the recoded data set (run on SPSS for Windows) is also part of the technical report packet and will be submitted separately.

BACKGROUND

The GiftsCompass Instrument (GCI) was developed by the GiftsCompass Inc. and commercially used for the purposes of adult career counseling and development. The GCI was developed based on the personality theory of Carl Jung (1921/1971), which posits that people exhibit preferences for two different attitudes (extraversion or introversion) and four different functions (sensation, intuition, feeling, or thinking). A combination of each attitude with each function results in an eight-dimensional personality typology: ES, EN, EF, ET / IS, IN, IF, IT.

Dr. Hannah-Hanh Nguyen, an industrial-organizational psychologist, has been commissioned as a methodology consultant by the company to design a multi-phase construct validation study for the GCI since 2005. Dr. Nguyen worked closely with the company researchers as well as with three other independent subject matter experts to revise the GCI and conduct a content validation study. The result was a revised 40-item Likert-type measurement instrument (GCI-R, 2006), the content validity of which has been established. Furthermore, each item in the GCI-R is accompanied with a clarifying example the content of which was also validated by subject matter experts.

As a following step of the construct validation study, the company administered the GCI-R to 250 young adults and adults via the World Wide Web between fall 2006 and spring 2007. Voluntary participants were also asked to respond to a non-proprietary “Big Five” personality measure (the International Personality Item Pool; IPIP), the original GCI, and a demographic questionnaire pertaining to their education and work history. Each respondent received his or her complimentary personality profile as the compensation for their participation in the study.

After the company collected and forwarded the data set to her, the consultant proceeded to examine the psychometric properties of the GCI-R, its factor structure, as well as other evidence of construct validity of the measure. In the following sections, the consultant described, summarized and interpreted the data analytic results. The consultant would also make recommendations in terms of the follow-up steps that the company might want to carry out in the future.

Due to the high number of possible analyses that could be run with the data set, the consultant has selected several essential analytic procedures that would answer the client's key questions regarding the reliability and validity of the GCI-R and reported the results in this technical report. Supplemental analyses may be also conducted per client's request.

EXECUTIVE SUMMARY

To establish the reliability and construct validity of the measure, the consultant used the data collected from a development sample of up to 250 respondents to conduct several analyses that would answer the key questions about the GCI-R reliability and construct validity.

To quantitatively answer the question whether the GCI-R subscales have internal consistency, the consultant examined the item and scale reliability of individual subscales (i.e., a combination between each of two Extraversion and Introversion attitudes and each of four psychological functions as theorized by Jung). The consultant also examined the reliability of an encompassing scale of either Extraversion items or Introversion items regardless of original functions.

The consultant found that the reliability results for the GCI were generally desirable. Specifically, five subscales have acceptable or satisfactory levels of internal consistency; the consistency of three subscales are lower but not poor and can be improved by fine-tuning the content of some scale items. At the high-order level (either Extraversion or Introversion), the internal consistency is satisfactory, showing that each set of items/attributes do assess either attitude.

The consultant then conducted the inter-scale relation analyses. The results showed that the four Extraversion subscales indeed measured a similar construct and yet each subscale might capture additional, unique information associated with any of the four functions (sensing, intuition, feeling and thinking). So did the four Introversion subscales.

The GCI-R subscales were correlated with the International Personality Item Pool. With one exception, the GCI-R appears to have both convergent and divergent validity when both GCI-R Extraversion and Introversion subscales were correlated with the Extraversion subscale of the IPIP. Furthermore, most of the GCI-R subscales did not have significant relationships with indicators of respondents' academic achievement and/or cognitive ability. Any significant relationships were found to be in the negative direction and/or having only moderate or weak magnitudes. Altogether, the data supported the convergent and divergent validity of the GCI-R—another piece of construct validity evidence.

To answer the question whether the data supported an 8-factor typology as conceptualized by Jung, the consultant conducted a series of exploratory factor analyses. The results were mixed: the structure of subscales held up relatively well at either the full-measure level or at the Extraversion/Introversion level (ET, EF, EN, IT, IF) pending some revision; However, the other three factors were not very clear cut in terms of scale structure and scale meaning (e.g., items from different subscales ending up in a factor). The factor analytic findings might be because of the relatively small sample size (i.e., fewer than 10 respondents for each item or 400 respondents) though.

The discriminant validity of the GCI-R was explored. The results showed tentative support for discriminant validity: certain personality subscales as measured with the GCI-R might have a relationship with employment status.