SUMMARY of the CONTENT VALIDITY STUDY RESULTS
of the GIFTSCOMPASS™ INVENTORY
April, 2005 – July, 2006
Prepared by: Margie Spino, M.A.
Research Director, GiftsCompass, Inc.
Research Consultant: Hannah-Hanh Nguyen, Ph.D.
WHAT IS MEASUREMENT VALIDITY?
The purpose of a validity study is to determine if an instrument is measuring what it was developed to measure. For example, does
a ruler accurately measure (or represent) length? If it does, then it is considered to be a valid tool. What if your high score on
a math achievement test really only reflects your knowledge of English? Then it would not be a valid tool to measure
The first step in establishing measurement validity for an instrument is usually conducting a content validity study based on the
judgments of Subject Matter Experts (SMEs). A SME is someone who is considered to have extensive knowledge in a particular content
HOW WAS the CONTENT VALIDITY of the GIFTSCOMPASS™ INVENTORY EVALUATED?
A content validity study was conducted in order to determine if the items in the GiftsCompass™ Inventory (GCI) represent
Carl Jung's theory of psychological types. To answer this question, the GCI developers commissioned a research consultant from
Michigan State University who developed a multi-step validation strategy. Following the consultant's recommendations, the GCI
developers engaged in the following research activities:
- Reviewed Jungian theory, existing personality measures, and psychometric principles in order to edit and refine 40
GCI attribute statements and create 80 example statements for these attributes.
- Identified three Subject Matter Experts (SMEs) to evaluate the attribute and example statements. Each SME was invited
because of his knowledge of Jung's type theory and/or psychological measurement.
- Conducted an initial content validity study in which the SMEs evaluated the degree to which the 40 attribute statements
and 80 examples were relevant to and representative of Jung's constructs of psychological types.
Results of initial content validity study: Using a two-third rule (i.e., endorsed by at least two
out of three SMEs), 15 attribute and example pairs were found to be relevant and representative of Jung's constructs.
- Further reviewed Jung's theory and revised 25 attribute and example pairs based on the SMEs' feedback and the consultant's
- Conducted a follow-up study in which the SMEs evaluated the revised 25 attribute and example pairs.
Results of follow-up study: 22 attribute-example pairs were found to be relevant and representative of
Jung's constructs by at least two out of three SMEs.
- Revised again the three cases of disagreement (i.e., remaining attribute-example pairs that did not pass the two-third rule)
based on the results of the follow-up study.
- Compiled 40 pairs of attribute and example statements that had been judged to be relevant and representative of Jung's
constructs. These statements would be used in the next phase of validity research (i.e., construct validation).
WHAT can be CONCLUDED?
"The content validation phase has been successfully conducted for the GiftsCompass™ Inventory.
Overall, both initial and follow-up stages of the content validation study of the GiftsCompass™ Inventory (GCI) were
conducted as systematically and as rigorously as possible. Data-driven revisions of attributes and examples were made, and
the majority of the GCI items subsequently received SMEs' approval as representing Jungian typology. Specifically, in the
follow-up study, most of the revised item-example pairs were satisfactorily classified into their GCI intended dimensions by
the same judges who had not classified them as such before. This fact shows a marked improvement of the quality and validity
of the GCI item content. The measurement content was also rated by the SMEs as clear and simple enough to be understood by
educated individuals, as an SME commented in the follow-up study, ‘I did feel more overall comfort with these items
-- Hannah-Hanh Nguyen, Ph.D., Research Consultant