In the representation step, the sorting and rating data were entered into the computer, the MDS and cluster analysis were conducted, and materials were produced for the interpretation step.

The concept mapping analysis begins with construction from the sort information of an NxN binary, symmetric matrix of similarities, Xij. For any two items i and j, a 1 was placed in Xij if the two items were placed in the same pile by the participant, otherwise a 0 was entered (Weller and Romney, 1988, p. 22). The total NxN similarity matrix, Tij was obtained by summing across the individual Xij matrices. Thus, any cell in this matrix could take integer values between 0 and 11 (i.e., the 11 people who sorted the statements); the value indicates the number of people who placed the i,j pair in the same pile.

The total similarity matrix Tij was analyzed using nonmetric multidimensional scaling (MDS) analysis with a two-dimensional solution. The solution was limited to two dimensions because, as Kruskal and Wish (1978) point out:

Since it is generally easier to work with two-dimensional configurations than with those involving more dimensions, ease of use considerations are also important for decisions about dimensionality. For example, when an MDS configuration is desired primarily as the foundation on which to display clustering results, then a two-dimensional configuration is far more useful than one involving three or more dimensions (p. 58).

The analysis yielded a two-dimensional (x,y) configuration of the set of statements based on the criterion that statements piled together most often are located more proximately in two-dimensional space while those piled together less frequently are further apart.

This configuration was the input for the hierarchical cluster analysis utilizing Ward's algorithm (Everitt, 1980) as the basis for defining a cluster. Using the MDS configuration as input to the cluster analysis in effect forces the cluster analysis to partition the MDS configuration into non-overlapping clusters in two-dimensional space. There is no simple mathematical criterion by which a final number of clusters can be selected. The procedure followed here was to examine an initial cluster solution that on average placed five statements in each cluster. Then, successively lower and higher cluster solutions were examined, with a judgment made at each level about whether the merger/split seemed substantively reasonable. The pattern of judgments of the suitability of different cluster solutions was examined and resulted in acceptance of the fifteen cluster solution as the one that preserved the most detail and yielded substantively interpretable clusters of statements.

The MDS configuration of the ninety-six points was graphed in two dimensions and is shown in Figure 1. This "point map" displayed the location of all the brainstormed statements with statements closer to each other generally expected to be more similar in meaning. A "cluster map" was also generated and is shown in Figure 2. It displayed the original ninety-six points enclosed by boundaries for the eighteen clusters.

The 1-to-5 rating data was averaged across persons for each item and each cluster. This rating information was depicted graphically in a "point rating map" (Figure 3) showing the original point map with average rating per item displayed as vertical columns in the third dimension, and in a "cluster rating map" which showed the cluster average rating using the third dimension. The following materials were prepared for use in the second session:

(1) the list of the brainstormed statements grouped by cluster

(2) the point map showing the MDS placement of the brainstormed statements and their identifying numbers (Figure 1)

(3) the cluster map showing the eighteen cluster solution (Figure 2)

(4) the point rating map showing the MDS placement of the brainstormed statements and their identifying numbers, with average statement ratings overlaid (Figure 3)

(5) the cluster rating map showing the eighteen cluster solution, with average cluster ratings overlaid

Representation Results

The final stress value for the multidimensional scaling analysis was .2980101.

Methods for estimating the reliability of concept maps are described in detail in Trochim (1993). Here, six reliability coefficients were estimated. The first is analogous to an average item-to-item reliability. The second and third are analogous to the average item-to-total reliability (correlation between each participant's sort and the total matrix and map distances respectively). The fourth and fifth are analogous to the traditional split-half reliability. The sixth is the only reliability that examines the ratings, and is analogous to an inter-rater reliability. All average correlations were corrected using the Spearman-Brown Prophesy Formula (Weller and Romney, 1988) to yield final reliability estimates. The results are given in Table 2.

Back to Contents
Copyright 1996, William M.K. Trochim