We turn now to the question of what makes a visually inspectable representation picture-like, map-like, diagram-like, chart-like or text-like.
Our example is more or less taken from Example 5 in Barwise and Etchemendy's ``Visual Information and Valid Reasoning'' (baretch1).
Here are two representations of the same situation, one a diagram, the other a textual description.
Figure 1: A diagram of five chairs and their occupants
Figure 2: A textual description of five chairs and their occupants
Figure 1 diagrams and Figure 2 describes a situation in which five chairs are arranged in a row and a person A occupies the leftmost chair, no one is allowed to occupy the second from the left, and C, D and E occupy the next three. The people are indicated by large letters in both figures; the chairs are indicated by lines in the first and small letters in the second.
What makes the first figure more picture-like, the second more text-like? We will begin our investigation by examining a list of criteria offered by Barwise and Etchemendy.
In (baretch1), Barwise and Etchemendy list six ways in which diagrammatic representation differs from linguistic representation: the former exhibit closure under constraints, conjunctive rather than disjunctive information, and homomorphic representation. They support symmetry arguments and perceptual inference.
The point with respect to symmetry arguments is that such arguments are often involved in reasoning with diagrams (for example the reasoning problem connected with Example 5). This point about reasoning with diagrams is not presented as either a necessary or a sufficient condition for something being a diagrammatic representation, so we are going to set it aside. The point about perceptual inference we will defer until later.
(We should emphasize that we are not indulging in the old-fashioned philosophical exercise of searching for the essence of pictures or diagrams or RGM. We are engaged in the new-fangled cognitive science exercise of looking for contributing factors to differences that we intuitively feel and exploit, that will lead to better and more useful classifications of the phenomena in which those differences are found, and may support increasingly detailed empirical and mathematical studies of the phenomena.)
So, as we were saying, what is the essence of pictures?
Homomorphism is at best a necessary condition. If we consider Figure 1 and Figure 2 we have homomorphism in both cases. We will make the point by showing the correspondence between the representing facts and the represented facts. We will call the lines in Figure 1, ``1'',''2'',''3'',''4'' and ``5''; we will designate the people and chairs with large and small letters, respectively.
For the pictorial representation:
Figure 3: Homomorphism from diagram to chair situation
For the linguistic representation:
Figure 4: Homomorphism from description to chair situation
Now Wittgenstein, noticing something like the sort of homomorphism we just presented, advanced the idea that sentences were pictures (Wittgenstein; see also [Etchemendy 1976]). He might be right at a suitably deep level, but at the level at which we are operating, we draw the conclusion that a homomorphism between the representing and the represented is not enough to make the representation diagram-like.
In the case of real pictures, it is not so clear that there is a perfect homomorphism. In a picture that uses perspective, one element being above another can signify that one thing is behind another or that one thing is above another. This may simply show that we have not chosen the representing relations carefully enough to find the homomorphism. We will assume that homomorphism is a good approximation of a necessary condition for being picture-like or diagram-like.
Now let's look at ``closure under constraints''. As Barwise and Etchemendy note, diagrams are physical situations and so they obey their own set of constraints. They say,
By using a representational scheme appropriately, so that the constraints on the diagrams have a good match with the constraints on the described situation, the diagram can generate a lot of information that the user never need infer. Rather, the user can simply read off facts from the diagram as needed. This situation is in stark contrast to sentential inference, where even the most trivial consequence needs to be inferred explicitly (baretch1).
As we understand it, the property in question is more fully describable as,
Constraints on the facts in a representation R that represent facts about a relation Q are such that IF Q-factsare explicitly represented in R, and
, THEN R will explicitly represent
Here is an example. Let these three blocks be our situation:
Figure 5: The three-blocks situation
Let our picture-like representation be based on the idea that the representation will be a row of letters on a line from left to right, so that a letter being to the left of another represents the fact that the represented blocks are in the left of relation. We'll use ``B'' to name the block, ``D'' to name the diamond, and ``T'' to name the triangle.
Figure 6: Diagram-like representation of the three-blocks situation
Let our language-like representation be a sequence of sentences of the form ``X is to the left of Y''. If a sequence of letters X,Y flanks the ``is to the left of'' predicate, that represents that the block X stands for is to the left of the block Y stands for.
Figure 7: Linguistic representation of three-blocks situation
Now if we put in our diagram-like representation a representation to the effect that the box is to the left of the diamond, and one to the effect that the diamond is to the left of the triangle, we will have eo ipso put one in to the effect that the box is to the left of the triangle.
But, if we write the sentence ``B is the left of D'' and the sentence ``D is to the left of T'' we will not have thereby written the sentence ``B is to the left of T''.
So our diagram-like representation is closed under constraints, and our language-like one is not.
Why is this so? In Figure 6 the transitivity of the ``is to the left of in a row'' relation among tokens of letters matches the transitivity of being to the left of in a sequence of blocks. But as Figure 7 shows, the relation of having letters that flank the words ``is to the left of'' is not transitive. The relation holds between ``B'' and ``D'' and between ``D'' and ``T'' but not between ``B'' and ``T''.
Closure under constraints is a real difference between a diagram and a typical representation that is not diagram-like. But it is not a logically sufficient condition for being diagram-like. One can imagine a magic slate that always automatically produced the closing representation--i.e., would just write ``A is to the left of C'' when someone had written on it, ``A is to the left of B'' and ``B is to the left of C''. That would not be a diagram-like representation.
(Approximate) homomorphism and closure under constraints arise when (but perhaps not only when) we have systematic, constrained and localized representation. This requires that three conditions are met. First, a whole system of relations is systematically interpreted as representing another system of relations, rather than the interpretations being assigned piecemeal. Second, the representing relationships obey the constraints that correspond to those obeyed by the represented relationships. Third, there is only one token for each individual object.
Consider a diagram one might draw to show someone how to lay out a croquet court. A great many croquet courts of different sizes and oriented in different directions might satisfy the diagram. It is the relative distances and relative directions that count. For each court that satisfies the diagram there will be a homomorphism between distances between wicket symbols on the map and distances on the court, and between orientation on the diagram and directions. The homomorphism is not fixed piecemeal; once it is fixed that one distance on the diagram represents a certain distance in the world, all the other interpretations are fixed, and similarly with directions.
One could have systematic interpretation in a text-like representation; the distance relations might all by expressed by inter-related linguistic formulae, such as ``being n meters'', ``being n+1 meters'' etc. But such a linguistic relation would not be closed under constraints.
Note that the oddity or unnaturalness of our text-like homomorphisms
comes, at least in part, from the fact that we allow more that one
token for a given object in a given representation.
Our
diagram of a croquet court, however, meets what we call ``the unique
token requirement''. There is one and only one representation for
each wicket. All the representing facts about that wicket--the facts
that represent its distance from other wickets, its direction, and any
other facts about it that are represented--will involve that one
representation.
Multiple-token representation is ubiquitous in language, of course. It has the effect of destroying the constraints that guarantee closure. Returning to the example involving Figure 5, if we had allowed ourselves to use two tokens of ``D'' in our representation of the row of shapes in Figure 6, then we could have had a representation that explicitly represented B being to the left of D, and D being to the left of T, without having an explicit representation of B being to the left of T.
Finally, Barwise and Etchemendy say that diagrammatic representations are conjunctive rather than disjunctive. This should not be taken to mean that a particular representing fact cannot represent a range of alternatives. There are many actual croquet courts, facing different directions and with different distances between the wickets, that satisfy the diagram we are imagining. The point is rather that the effect of adding a new representing fact to a picture or diagram-like representation is to conjoin a fact to what is already represented, not provide an alternative. This is a consequence of systematic, constrained and localized representation. One creates new representations by placing new representations for individual objects onto the diagram. The new representation cannot represent an alternative for one of the wickets already represented, by the unique token requirement.
This property of unique token representation is related to the use of locations for grouping information, that Larkin and Simon emphasize (larksim). They provide three reasons why diagrams can be superior to verbal descriptions for solving problems.
The first two reasons emphasize the way diagrams use location to group information about a single object. This is lost when one uses the system of types and token. Many different tokens of the same type designating the same individual object may be scattered around a document, so that the information the document contains about that individual is not localized. It is a feature of perception that the perceptually accessible information about an individual is centered on that part of the perception that we think of as being of the individual. Monadic or intrinsic facts about the individual will be picked up by inspection of the individual, and relational facts will involve of part of the scene that involves the individual. This sort of localization makes looking at a diagram or picture like looking at the things themselves, and permits the inferential abilities of the perceptual system to be exercised on the diagram or picture.
Stenning and Oberlander find the difference between graphical representations and textual representations in a property they call specificity. We suggest that there are several aspects to specificity that are worth distinguishing. The major division is between determinateness and regimentation. Determinateness we further divide into two kinds, issue determinateness and Berkeley determinateness.
The basic idea of issue determinateness is that if a representation raises an issue, it settles it. Let our representation be the following two sentences:
The representational resources of this representation allow us to raise two further issues: Is David charming? Does Madeline work at SRI? But the representation does not settle them.
We will say that an issue, in the situation-theoretical sense of a relation and a suitable sequence of arguments, is available from a representation if the representation contains items that stand for the relation and each of the arguments. Issue determinateness means that all available issues are settled by the representation--that is, that it explicitly represents that the answer for the issue is yes or no (polarity 1 or 0).
This property requires more of a picture or diagram than the property of closure under constraints that Barwise and Etchemendy mentioned. Suppose we have a representation of the fact that A is larger than B and a representation of the fact that C is below D. The closure condition does not require that we have representations that tell us whether or not A is below B, or C is larger than D, but this property does. However, it does seem that systematic, constrained and localized systems of representation meet this condition. In such a system, an element represents things about the object it designates in virtue of having various properties and standing in various relations. Each of the other elements will have properties of the same kind and stand in relationships of the same kind. So issues that are settled for one object, will be settled for all. For example, when one puts a dot representing Omaha on a map, making issues about Omaha available in our sense, that dot will be a certain distance from all the other dots. Putting the dot on the map, which makes the issues available, also settles them.
The second notion of determinateness is suggested by Stenning and Oberlander's citation of Berkeley, so we call it ``Berkeley determinateness''. What impressed Berkeley was the fact that you couldn't draw a picture of a triangle or have a mental image of a triangle that wasn't a picture or mental image of some definite type of triangle, scalene, isosceles, right angle, etc.
To state what Berkeley determinateness entails, we need the determinable-determinate distinction. This is exemplified by color and red, or height and 5'3'', or shape and 3/4/5 right triangle or weight and 180 pounds. Any object that has a determinable property (shape, color, size) has some determinate value of it. But it is not the case in general that a representation that represents an object as having a determinable property represents the object as having some determinate value of it. If we say, ``David has an interesting shape,'' I imply that he has a shape, but I don't say exactly what his shape is.
It is a property a representational system might have, relative to some category of properties, that if it represents an object as having a determinable property then it represents that object as having some determinate value of it. This is what we call Berkeley determinateness with respect to that category of properties.
However, it is not generally the case with pictures that they are Berkeley determinate with respect to the visually detectable properties they depict. An artist need not decide whether she is painting a picture of tall people standing in front of a tall tree or short people standing in front of a short tree. She represents the people and the tree as having height (and arguably, weight) but not specific heights and weights. But if she included a scale, and thus provided a systematically homomorphic representation, the picture would have this property.
The fact is that there are some correspondences that are so natural, that it is difficult to imagine alternatives. If you are going to use closed figures to represent shapes, what shape should a triangle represent? It seems that the natural answer is, triangles. What should an isosceles triangle represent? An isosceles triangle. The representation will have a determinate value of each of its determinable properties, and so if it represents exactly the same properties it has, the representation will be Berkeley determinate.
But this doesn't generalize. If we are going to use objects with size to represent sizes, what size should a one inch figure represent? The answer is not so obvious. We are familiar with representing one size with another, and one distance with another, but not with representing one color with another, or one shape with another.
Given a systematic, constrained and localized system of representation, we need to fix rather than merely constrain the homomorphisms between representing and represented relations to get Berkeley specificity. With our croquet court diagram we could do this by adding a scale and a north arrow.
Our conclusion then is that systems that are systematic, constrained and localized need not be Berkeley determinate. That is an additional property. Some systems may have it because there is a very natural built in ``interpretation-fixer'' that we assume at least as a default: red represents red, yellow represents yellow, etc. Other systems may have it because devices like a scale or a north arrow fix the homomorphism.
The fact that some representing properties such as colors and shapes seem to have a built in interpretation-fixer can create problems for designers of representational systems. Lingraphica is a system designed by Dick Steele for the use of global aphasics, who have lost the ability through brain injury to remember the meanings of words. Steele designed an iconic, MacIntosh-based system for communicating with aphasics. He concentrated for a while on recipes, which he found that his patients could follow unassisted, by figuring out the meaning of the icons.
He had trouble coming up with an icon for ``stir''. The natural way to do it is with a dynamic icon showing a bowl of stuff being stirred by a spoon, say. But how to make this an icon for ``stir'' and not ``stir with a spoon''?
Stenning and Oberlander give this list of representation systems, to indicate some points along a dimension they call ``regimentation'':
...a quantified abstraction; a disorderly text; an orderly text;...an alphabetized table of intercity distances; the same table with cities ordered by longitude in the column labels and latitude in the row labels; and finally a map.
Stenning and Oberlander connect this dimension with the number of ways there are for making a representation true, and this connects regimentation with determinateness. We'll bypass the issues here, and note that at least one aspect of regimentation connects with localization and the unique token requirement.
In an ordinary piece of text, there are no limitations on the number of different tokens that might stand for a given individual object, nor any restriction on where they might occur. Still, a good orderly presentation will exhibit some localization of information. Consider for example the CSLI brochure that is given to prospective Industrial Affiliates. There is a paragraph headed by the name of each researcher, which contains certain vital facts about them. A reader might turn to this page to know who, for example, John Etchemendy is. But tokens of the name ``John Etchemendy'' might also occur elsewhere in the brochure; not all the information about Etchemendy is localized.
Stenning and Oberlander compare such more or less orderly texts with a mileage chart and a map. In a mileage chart, there are two tokens of each name, one labeling a column, one labeling a row. Finally, with a map, we have one token per object represented.
This suggests a dimension defined by the constraints on the number and location of tokens of names, with the unique token constraint at one end and the total lack of constraints found in disorderly text at the other. We will use just a rough and ready classification: no constraints, some constraints, and the unique token requirement.