April, 2005 – July, 2006

Prepared by: Margie Spino, M.A.
Research Director, GiftsCompass, Inc.

Research Consultant: Hannah-Hanh Nguyen, Ph.D.
Industrial-Organizational Psychology

The purpose of a validity study is to determine if an instrument is measuring what it was developed to measure. For example, does a ruler accurately measure (or represent) length? If it does, then it is considered to be a valid tool. What if your high score on a math achievement test really only reflects your knowledge of English? Then it would not be a valid tool to measure mathematical knowledge.

The first step in establishing measurement validity for an instrument is usually conducting a content validity study based on the judgments of Subject Matter Experts (SMEs). A SME is someone who is considered to have extensive knowledge in a particular content area.

A content validity study was conducted in order to determine if the items in the GiftsCompass™ Inventory (GCI) represent Carl Jung's theory of psychological types. To answer this question, the GCI developers commissioned a research consultant from Michigan State University who developed a multi-step validation strategy. Following the consultant's recommendations, the GCI developers engaged in the following research activities:

  1. Reviewed Jungian theory, existing personality measures, and psychometric principles in order to edit and refine 40 GCI attribute statements and create 80 example statements for these attributes.
  2. Identified three Subject Matter Experts (SMEs) to evaluate the attribute and example statements. Each SME was invited because of his knowledge of Jung's type theory and/or psychological measurement.
  3. Conducted an initial content validity study in which the SMEs evaluated the degree to which the 40 attribute statements and 80 examples were relevant to and representative of Jung's constructs of psychological types.

    Results of initial content validity study: Using a two-third rule (i.e., endorsed by at least two out of three SMEs), 15 attribute and example pairs were found to be relevant and representative of Jung's constructs.

  4. Further reviewed Jung's theory and revised 25 attribute and example pairs based on the SMEs' feedback and the consultant's recommendations.
  5. Conducted a follow-up study in which the SMEs evaluated the revised 25 attribute and example pairs.

    Results of follow-up study: 22 attribute-example pairs were found to be relevant and representative of Jung's constructs by at least two out of three SMEs.

  6. Revised again the three cases of disagreement (i.e., remaining attribute-example pairs that did not pass the two-third rule) based on the results of the follow-up study.
  7. Compiled 40 pairs of attribute and example statements that had been judged to be relevant and representative of Jung's constructs. These statements would be used in the next phase of validity research (i.e., construct validation).

"The content validation phase has been successfully conducted for the GiftsCompass™ Inventory.

Overall, both initial and follow-up stages of the content validation study of the GiftsCompass™ Inventory (GCI) were conducted as systematically and as rigorously as possible. Data-driven revisions of attributes and examples were made, and the majority of the GCI items subsequently received SMEs' approval as representing Jungian typology. Specifically, in the follow-up study, most of the revised item-example pairs were satisfactorily classified into their GCI intended dimensions by the same judges who had not classified them as such before. This fact shows a marked improvement of the quality and validity of the GCI item content. The measurement content was also rated by the SMEs as clear and simple enough to be understood by educated individuals, as an SME commented in the follow-up study, ‘I did feel more overall comfort with these items and examples’."

-- Hannah-Hanh Nguyen, Ph.D., Research Consultant