Last Modified 1-1-2011

IUSB

Software Engineering

User Interface Design

Software Engineering User Interface Design

A system in which the interaction occurs at a level which is understandable to the user will be accepted faster than a system where it is not. A system which is available at irregular intervals or gives incomprehensible error messages, is likely to meet resistance. A 1992 survey found that 48% of the code of applications was devoted to the user interface and about 50% of the development time was devoted to implementing that part of the application. Often the user interface is one of the most critical factors as regards to the success or failure of a computerized system. Yet, most software engineers know fairly little about this aspect of our trade.

Users judge the quality of a software system by the degree in which it helps them to accomplish their tasks and by the sheer joy they have in using it. This judgment is to a large extent determined by the quality of the user interface. Good user interfaces contribute to a system's quality in the following ways:

• Increased efficiency: If the system fits the way its users work and if it has a good ergonomic design, users can perform their tasks efficiently. They do not lose time struggling with the functionality and its appearance on the screen.


• Improved productivity: A good interface does not distract the user, but rather allows him to concentrate on the task to be done.


• Reduced Errors: Many so-called 'human errors' can be attributed to poor user interface quality. Avoiding inconsistencies, ambiguities, and so on, reduces user errors.


• Reduced Training: A poor user interface hampers learning. A well-designed user interface encourages its users to create proper models and reinforces learning, thus reducing training time.


• Improved Acceptance: Users prefer systems whose interface is well-designed. Such systems make information easy to find and provide the information in a form which is easy to use.

In a technical sense, the user interface often comprises one or more layers in the architecture of the system. Two well known architectural styles are Seeheim Model and the Model-View-Controller that highlight the place and role of the user interface in interactive systems. A common denominator of these and other schemes is that they separate the functionality of the system from the interaction with the user. In a similar vein, many software engineering methods also separate the design of the functionality from the design of the user interface. The design of the user interface then reduces to a mere design of the screen layout, menu structure, size and color of buttons, format of help and error messages, etc. User interface design then becomes an activity is only started after the requirements engineering phase has finished. It is often done by software engineers who have little specialized knowledge of user interface design.

Software engineers are inclined to model the user interface after the structure of the implementation mechanism, rather than the structure of the task domain. For instance, a structure based editor may force you to input ↑ 10 2 to obtain 10², simply because the former is easier to recognize. This resembles the interface to early pocket calculators, where the stack mechanism used internally shows itself in the user interface. Similarly, user documentation often follows implementation patterns and error messages are phrased in terms that reflect the implementation rather than the user tasks.

We advocate a rather different approach. This approach may be summarized as 'The user interface is the system'. This broader view of the concept User Interface and the disciplines that are relevant while developing user interfaces are discussed in What is the User Interface. Within the approach discussed, the design of the user interface replaces what we used to call requirements engineering. The approach is inspired by the observation that the usability of a system is not only determined by its perceptual appearance in the form of menus, buttons, etc. The user of an interactive system has to accomplish certain tasks. Within the task domain, for example sending electronic mail or preparing documents, these tasks have a certain structure. The human-computer interaction or HCI, then should have the same structure, as far as this can be accomplished. Discovering an adequate structuring of the task domain is considered part of the user interface design. This discovery process and its translation into user interface representations requires specific expertise, expertise that most software engineers do not possess.

In order to develop a better understanding of what is involved in designing user interfaces, it is necessary to take a closer look at the role of the user in operating a complex device such as a computer. Two types of model bear upon the interplay between a human and computer: the user's mental model and the conceptual model.

Users create a model of the system they use. Based on education, knowledge of the system or application domain, knowledge of other systems, general world knowledge, and so on, the user constructs a model, a knowledge structure, of that system. This is called the mental model. During interaction with the system, this mental model is used to plan actions, and predict and interpret system reactions. The mental model reflects the user's understanding of what the system contains, how it works, and why it works the way it does. The mental model is initially determined through metacommunication, such as training and documentation. It evolves over time as the user acquires a better understanding of the system. The user's mental model need not be, and often is not, accurate in technical terms. It may contain misunderstandings and omissions.

The conceptual model is the technically accurate model of the computer system created by designers and teachers for their purposes. It is a consistent and complete representation of the system as far as user-relevant characteristics are involved. The conceptual model reflects itself in the systems's reaction to user actions.

The central question in human-computer interaction is how to attune the user's mental model and the conceptual model as well as possible. When this is achieved to a higher degree, an interactive system becomes easier to learn and easier to use. Where the models conflict, the user gets confused, makes errors, and gets frustrated.; A good design starts with a conceptual model derived from an analysis of the intended users and their tasks. The conceptual model should result in a system and training materials which are consistent with the conceptual model. This, in turn, should be designed such that it induces adequate mental models in the users.

In The Role of Models In Human-Computer Interaction various models are discussed that play a role in the Human-Computer Interface. As well as the aforementioned mental and conceptual models, attention is given to a model of human information processing. When interacting with a system, be it a car or a library information system, the user processes information. Limitiations and properites of human information processing have their effect on the interaction. Knowledge of how humans process information may help us to develop systems that users can better cope with.

There are many factors that impact human-computer interaction. In this discussion we just scratch the surface. Important topics not being discussed include the socio-economic context of human-computer interaction, input and output media and their ergonomics, and workplace ergonomics.

Where is the User Interface?

A computerized library system includes a component to search the library's database for certain titles. This component includes code to implement it's function as well as code to handle the interaction with the user. In the old days, these pieces of code tended to be entangled, resulting in one large, monolithic piece of software.

In 1983, a workshop on user interface management systems took place at Seeheim in West Germany. At this workshop, a model was proposed which separates the application proper from the user interface. This model has become known as the Seeheim model.

Figure UID-1: Graphical Representation of Seeheim Model

The Seeheim models describes the user interface as the outer layer of the system. This outer layer is an agent responsible for the actual interaction between the user and the application. It, in turn, consists of two layers:

• the presentation The perceptible aspects including screen design and keyboard layout.


• the dialog: The syntax of the interaction including metacommunication (help functions, error messages, and state information.) If the machine is said to apply a model of its human partner in the dialog, by choosing the user's native language for command names, this model is also located in the dialog layer.

This conceptualization of the user interface does not include the application semantics, or functionality. In the Seeheim model, the tasks the user can ask the machine to perform are located in another layer, the application interface. Figure UID-1 shows the separation of concerns into three parts. For efficiency reasons, an extra connection is drawn between the application and the display. In this way, large volumes of output data may skip the dialog layer.

The Seeheim model provides some very relevant advantages. For example, we may provide the same outer layer to different applications. We may apply the same look and feel to a text editor, a spreadsheet, and so on, as in Microsoft products. In this way the user does not have to learn different dialog languages for different applications. Conversely, we may provide a single application to be implemented behind several different outer layers, so as to allow different companies to adopt the same application with their own corporate interface style.

Part-Whole Decomposition

In both these cases, it is assumed that the changes are likely to occur in the interface part of the system, while the application part remains largely unaffected. Alternatively, we may assume that the functionality of the system will change. We then look for an architecture in which parts of the system can be modified independently of each other. A first decomposition of an interactive system along these lines is depicted in the diagram below:

Figure UID-2: A Part-Whole decomposition of interactive systems.

Model View Controller

Each component in this decomposition handles part of the application together with presentation and the dialog. In a next step, we may refine this architecture to depict the input/output devices as well. The result of this modification is shown in the Model-View-Controller diagram below:

Figure UID-3: Model - View - Controller System Perspective

The dialog and application together constitute the main component, while the input is called Controller and the output is called View.

Both the Seeheim and the Model-View-Controller concern the quality arguments that pertain to flexibility. However, with the Seeheim model, the primary concern is with change in the user interface, while in the Model-View-Controller the primary concern is with changes in functionality. This difference is not surprising once you consider the environments where each of these models were developed.

The Seeheim model was developed by a group of computer grpahics design specialists, hence the primary concern with the user interface. The Model View Controller model was designed by a group involved with the exploratory Smalltalk environment, hence the primary concern with functionality.

Both models have their advantages and disadvantages. When considering the model to use as your guide, you must first consider the project at hand and its quality requirements. The design team should then select an appropriate separation of concerns between the functionality requirements and the user communication requirements, and then choose the most appropriate model to meet those requirements.

In many applications, the user interface could actually be running on one machine, the client, while the application section could be running on a different machine, the server. This, of course, is also the case for many web applications where the user interface could be the browser. On one hand, where there is a tendency towards thin clients In thin clients, only the user interface is located at the client side, while all the processing takes place at the server side. In many applications, however, it is preferable that the user may continue to operate to some degree if connection to the internet becomes disrupted, and restore the data once the connection is re-established. This, then, gives rise to fatter clients. Therefore, the separation between user interface and applications, is not always the same as that between browser and server applications.

User Interface Aspects

The concept of a User Interface has several meanings. In one case, it could denote the layout of the screen, such as in a Windows Environment, perhaps the shell, a layer in the architectur of a system, or perhaps even the entire application. Each of these meanings denotes a designer's point of view. Alternatively, the user interface can be designed from the point of view of the intended user of a system. In most cases, users do not make a distinction between layers in the architecture and they may not even have a clear view of the difference between hardware and software. For most users, an information system is simply a tool to perform certain tasks. From their perspective, the User Interface is the system.

In this discussion, we will use the term User Interface to denote all aspects of the information system that are relevant to the user. This includes not only everything that a user can perceive or experience, but also aspects of internal structure and processes as far as the user needs to be aware of them. For example, suppose a car salesman was trying to impress customers, and would rifle off the horse-power of each and every car in the shop. Most customers would not know how to interpret those figures, and many simply wouldn't care. A Rolls Royce dealer would know this. The answer to a question about the vehicle's horsepower would simply be 'Enough!!!'. The same holds for an information systems user and the many aspects of the internal structure of that system. On the hand, a user of a suite of programs including a text editor, a spreadsheet, and a graphics editor, should be aware that the contents of a clipboard is a memory structure that will remain unchanged until overwritten, and how to access the contents of that structure.

The user interface will be defined in the braod sense as the user virtual machine (UVM). The user virtual machine includes both hardware and software. It includes the terminal or work station that the user is in physical contact with, and any internal structures directly involved with information being displayed such as the network and any remote data collections. In this case, the UVM, including application semantics, are the subject of User Interface Design.

In many cases, several groups of users will need to be distinguished with respect to their tasks. For example, consider the case of an ATM. One type of user, the bank clients, place their card into the machine to perform some financial transaction. Other users are specially trained people who maintain proper functioning of the machine, and supply it with boatloads of cash. Their role is ATM maintenance and cash stuffing. Lawyers could constitute a third class of user. They need to argue, for or against, whether a certain transaction was indeed fraudulent. They would need to have access to a log or some type of transaction trace that is maintained by the system. Each of these users, has a different perspective on proper functioning of the ATM. Each of these roles will have a somewhat different user interface. Each role has to be aware of different processes and internal structures. Consequently, each user class will possess a different user interface. and, each user interface will have to be properly designed, with the proper relations established between each user interface design. In Software Engineering jargon: Within the task of the domain of the ATM, we must design a set of related User Virtual Machines, with respect to the methods that are part of each user class.

Not only will several groups of users have different interfaces to the same application, but sometimes a single user will require different interfaces to the same application as well. This would be the case when that same user is accessing a system through several different devices, such as a mobile phone, handheld pda, laptop or desktop computer. Each of these devices has different characteristics, such as the size and resolution of the screen, the number and type of buttons on each display, and perhaps we should even consider the environment where each of these devices might be used. For example, the user may be in a hurry and have gobs of distracting noise while working on the mobile phone, while the desktop may be used while gulping down a couple of margaritas in a much more quiet environment. Conceptually, this is not much different than the multi-role situations described previously. Technically, however, the multi-device user poses a new set of challenges.

We need to look at the user interface from different viewpoints:


• Design Aspect: Design all that is relevant to the user.


• Human Aspect: What does the user need to understand, and what capabilities do they require

In principle these aspects will need to be combined, otherwise the user may not understand the sytem's features. We will consider both aspects.

Human Computer Interaction Considerations

Attention to the user interface is often not considered until later phases of the software life cycle. The proper design approach, however, requires attention to the human user from the very start of the design process. The various design activities need to be carried out in parallel, and determine how each process is affected by other processes. A large design team may need to allocate specialists in the tasks of analyzing, specifying and evaluating user interface aspects. The design of the user interface consists of a complex set of activities, all of which are intended to focus on the human side of the sytem.

The human side cannot be covered by a single discipline or even a single technique. At least 3 different disciplines need to be employed:

1. Humanities.


2. Artistic Design


3. Ergonomics

Humanities

In the humanities view, we pay attention to people based on a psychological approach. In this view we consider how humans perceive, learn, remember, think and feel. We then must consider organization and culture: How will the users work together and how will their work situation affect their needs and expectations? Relevant disciplines include cognitive psychology, anthropology and ethnography. These disciplines provide a theoretical base and the associated techniques for collecting information on people's work environment as well as techniques for assessing newly designed tools and procedures. Designers of the User Virtual Machine will need some insight into the theories and experience with techniques from these disciplines. For example, let us consider the design of a control panel. We may consider that less information and fewer apparent options make it easier for the user to identify the most common used functions of the system (the psychological phenomenon of attention and distraction). On the other hand, if too little relevant information is displayed, the user may have to remember more. And one thing psychology teaches us is that human working memory has a very limited and variable capacity.

Artistic Design

Creative and performing artists have developed a knowledge base on a variety of ways to convey meaning to their public. Graphical artists know how shapes, colors, and spatial arrangements affect the viewer. Consequently, their expertise teaches interface designers how to draw the attention of users to important elements of the interface. For example, colors should be used sparing in order not devalue their possible meaning. Well chosen colors can help signify important relationships between different elements on the screen and can even support the users searching for relevant information. Design companies currently employ graphical artists to participate in the design of representational aspects of the user interface.

Complex systems often need a representation of complex processes, where several flows of activity influence each other. Some examples include a cockpit crew on an intercontinental passenger flight or a team monitoring a complex nuclear power plant operation. In such situations, users need to understand complex relations over time. The representation of the relevant processes and their relations over time is not trivial. Representing past, present and possible future scenarios is only part of the task at hand. Occasionally, these complex processes can become safety critical. This means a human supervisor may need to make the correct decision immediately after an abnormal phenomenon has occurred. He also has to understand the situation and the possible options and outcome scenarios that could occur. Experts in theater direction have knowledge for just this type of situation. This type of interface may be compared with a theater show, where an optimal direction of the action helps to make the audience aware of the complex intentions of the author, cast, and perhaps even save lives. Consequently, theater sciences are another source for designing interfaces to complex processes.

Another type of artistic expertise that turns out to be very relevant for interface design is cinematography. Film design has resulted in systematic knowledge of representation of dynamics and processes over time. For example, there are special mechanisms to represent the suggestion of causality between processes and events. If it is possible to graphically represent the causing process with a directional movement, the resulting event or state could be depicted as a prediction of likely outcomes. For example, in an electronic commerce system, buying an object may be represented by dragging that object to a shopping cart. If the direction of this movement is to the right of the screen, the resulting change in the shopping cart price should also be shown to the right.

In the same way, there are laws for representing continuity over time. In a movie, the representation of a continuing meeting between two partners can best be achieved by ensuring that the camera viewpoints do not cross the line that connects the location points of the two partners. As soon as this line is crossed, the audience will interpret this as a jump across time. This type of expertise helps the design of animated representations of processes and so on.

In general, artists are able to design attractive solutions, to develop a distinctive style for a line of products or for a company, and to relate the design to the professional status of the user. However, there are tradeoffs that must be decided on. For example, artistic design sometimes conflicts with ergonomics. Do you want to look good or feel good? When strolling through a consumer electronics shop you will find artistic variants of mobile phones, vending machines, and home theater systems, where the designer seems to have paid a tribute to artistic shape and color, while making the device less intuitive and less easy to use, from the point of view of fitting the relevant buttons to the size of a professional basketball player's hand size. A similar fate can befall a user of an information system.

Ergonomics

Human-Computer Interaction Models

The concept of a model has a distinguished position in the fields of human-computer interaction and cognitive ergonomics. Models represent relevant characteristics of a part of reality that we need to understand. At the same time models are abstract: they represent only what is needed, thus helping us find our way in complex situations. However, we need to be aware of the different types and inconsistent naming conventions of modeling. We begin by discussing the difference between internal and external models in human-computer interaction.

Internal models are models for 'execution'. Internal models use an agent to make a decision based on the behavior of the model. If the agent is human, the model is termed a mental model in psychology. Humans apply these models whenever they have to interact with a complex system.

If the agent is a machine, the internal model is a program or a knowledge system, known as an Intelligent Agent. For example, A user interface may retain a model of the user. In that case, the user interface retains a user model or model of the user. The model could help the interface react differently to different users, or alternatively, adjust to the current user depending on the machine's understanding of that user's goals or level of understanding. User models of this type may be designed to learn from user behavior and are commonly used in so-called intelligent user interfaces. A third type of model in machines is a model of the task domain, which enables the user and system to collaborate in solving problems about the task. The latter type of model leads to systems that can reason, critique user solutions, provide diagnosis, and suggest user actions.

External models are used for communication, and, hence, are first of all represented in some type of formalism, like a graphical representation such as a flow diagram or Petri net. The formalism should be chosen in relation to what is being modeled, as well as to the goal of the communication. In designing user interfaces, there are several domains where models are needed. Designers need to understand some relevant aspects of the user, especially human information processing capabilities. Cognitive psychology provides such types of knowledge, which discusses the variants in human information processing. We need only be concerned with those aspects which are relevant to the designing of computers. Another type of model used for various types of design activity are external models that help designers to document their decisions, to bactrack when one design overrules another, and to communicate the result of one design phase to people responsible for another phase.

Some types of models, such as task knowledge models, refer to aspects that are both internal and external. Task knowledge is originally found in the memory of human beings, in documents about the work domain, and in the actual situation of the work environment. These are internal models that are applied while doing the work. Designers need to understand this knowledge and apply it as the base of their designs. Additionally, they also need to perform a task analysis and model the task knowledge. This, obviously then, is an external model for use in design.

Human Information Processing Model

The model of human information processing is an example of an external model. The focus is on human perception, memory, and the processing of information in relation to the input and output of the human in interaction with an outside system. Figure HCI-1 depicts this model. Psychology textbooks may have a similar diagram but with equations that allow calculations of the speed of processing, the effect of learning, etc.

Figure HCI-1: Human Information Processing Model

In modern cognitive psychology, perception (the input to human information processing) is considered to proceed through a number of phases:

1. Edge Detection: The large amount of unstructured information that bombards our senses is automatically and quickly structured.


• Hearing: Structured into phenomes by the human brain.


• Vision: '2.5D sketch' based on movement, color and location.


2. Gestalt Formation: A small number of understandable structures known as Gestalts (triangle, word, tactile shape) are formed.


Based on:
• Similarities detected in the 2.5D sketch.


• Spatial Relations


• Simplicity.


3. Combination: The gestalts are combined into groups of segments that seem to belong together:


• An object consisting of triangles, cubes and cylinders.


• A spoken utterance consisting of a series of words.


4. Recognition: The group of segments is recognized:


• A picture of a horse


• A spoken sentence.

The processing in the later phases is done less and less automatically. People are aware of Gestalt formation and combination when any problem arises, for example because of irregularity or exceptional situations such as impossible figures. Recognition leads to conscious perception.

The whole series of phases takes a fraction of a second. It takes more time when a problem occurs because of an unexpected or distorted stimulus. It takes less time when the type of stimulus is familiar. So we may train our computer users to perceive important signals quickly and we may design our signals for easy, quick detection and discrimination. Psychologists and ergonomists know when a signal is easy to detect, what color combinations are slow to be detected, and what sounds are easy to discriminate.

The output of human beings is movement. People make gestures, manipulate tools, speak, or use a combination of these. For computer use, manipulation of keys, mouse or touch-screen, and speaking into microphones are common examples of output from the human user perspective. According to modern psychology, all those types of output are monitored by a central processing mechanism in the human. This central executive decides on the meaning of the output but leaves the actual execution to motor processes that, in normal cases, are running unattended, or in other words the actual execution is not consciously controlled. Only in case of problems is attention needed. For example, if the location to be pointed to on the screen is in an awkard position, or if a key is not functioning properly, or if the room is so noisy that the person cannot even hear his own spoken command. So we should design for human movements and human measures. It pays to ask an ergonomist about the most ergomic design of buttons and dials.

The central executive unit of human information processing is modeled as an instance that performs productions of the form: if condition then action, where the condition in most cases relates to some perceived input or to some knowledge available from memory. The action is a command to the motor system, with attributes derived from working memory. The central executive unit has a very limited capacity:

• Only a very small number of processes can be performed simultaneously,


• women: it may be 2, 3 or 4, or more


• men: 0 or 1.


• Knowledge needed in testing the condition as well as knowledge needed in motor output has to be available in working memory.

Most of the time we may consider the limitations to be the result in the execution of one process at a time and, consequently, in causing competing processes to be scheduled for sequential execution based on perceived priority. For example, when a driver approaches a crossroads, the talk with his passenger will be temporarily interrupted and only resumed when the driving decisions have been made. The amount of avaialbe resources has to be taken into account when designing systems. For example, most humans cannot cope with several error messages each of which requires an immediate decision, especially if each requires complex error diagnosis to be performed before reaction is feasible.

Working memory is another relevant concept in the model. Modern psychology presumes there is only one memory structure, long-term memory, that contains knowledge that is permanently stored. Any stimulus that reaches the central executive unit leads to the activation of an element in long-term memory. The activated elements together form the current working memory. The capacity of the set of activated elements is very limited. The average capacity of the human information processor is 5-9 elements. If new elements are activated, other elements lose their activation status, and hence, are no longer immediately available to the central executive. In other words, they are not in working memory any more.

Long term memory is highly structured. One important type of relation concerns semantic relations between concepts, such as part-of, member of, and specialization-generalization. In fact, each piece of knowledge can be considered a concept defined by its relation to other elements. Such a piece of knowledge is often called a chunk. It is assumed that working memory has a capacity of 5-9 chunks. An expert in some task domain is someone who has available well-chosen chunks in that domain, so that he is able to expand any chunk into relevant relations, but only when needed for making decisions and deriving an answer to a problem. If not neeeded, an expert will not expand the chunk that has been triggered by the recognition phase of perception or by the production based on a stimulus. For example, the sequence of digits '85884' could occupy five entries in working memory. However, if it is your mother's telephone number, it is encoded as such and only occupies one entry. When the number has to be dialed, this single entry is expanded to a series of digits again. Entries in the working memory can thus be viewed as labels denoting some unit of information, such as a digit, your mother's telephone number, or the routine quicksort. In this way an expert can cope with a situation even with the restrictions on the capacity of working memory.

The structures in long term memory are the basis for solving problems in a domain and for expertise. When working with a system, people develop a suitable knowledge structure for performing their tasks. If they are able to understand the system as much as they need they may develop a coherent and useful structure in long-term memory. As far as this structure can be considered a model of the system, we consider it to be a mental model of the system.

Information System Mental Models

Mental models are structures in long term memory. They consist of elements and relations between those elements. They represent relevant knowledge structures analogous to physical, organizational, and procedural structures in the world. These mental models become instantiated when activated by an actual need arising to make a decision related to an element of this knowledge. The activated mental model is, to a certain extent, run or executed in order to predict how the structure in the world that is represented by the mental model would behave in relation to the current situation and in reaction to the possible actions of the person. In the terminology introduced before, mental models are internal models.

When working with complex systems, where part of the relevant structure and processes of the system cannot be perceived by a human being or cannot be completely monitored, a mental model is needed to behave optimally in relation to the system. Hence, people develop mental models of such situations and systems. If the system is a computer system, there are four functions of using the system that require the activation of a mental model:

1. When planning the use of the technolgy, users will apply their knowledge to find out for what part of their task the system could be used and the conditions for its use. Users will determine what they need to do beforehand, when they would like to perform actions and take decisions during the use of the system, and when they would abort execution or reconsider use.


• Suppose a user wants to use a library information system to search for literature on mental models. Before the user can search by author name, there must be one or more candidate authors.


2. During execution of a task with a system, there is a continuous need for the fine-tuning of user actions towards system events. The mental model is applied to monitoring the collaboration and reconsidering decisions taken during the planning stage.


• If the result of a search is unsatisfactory, the user may decide to look up alternative author names or switch to a keyword search. If the keywords used by the system are keywords listed in the titles of publications, there is quite a chance that relevant literature is not found by a search using the keywords mental model. The user may then consider other keywords as well, such as human-computer interaction.


3. If the system has performed some task for the user and produced output, there is the need to evaluate the results, to understand when to stop, and to understand the value of the results in relation to the intended task. The mental model of the system is needed to evaluate the system's actions and output, and to translate these to the goals and needs of the user.


• Continuing our example, suppose some of the literature sources found have titles indicating a relation between mental models and learning, while others relate mental models to personality factors. The user may decide to keep only titles that relate mental models to Human Computer Interaction, since the others are probably not relevant.


4. Modern computer systems are frequently not working in isolation and more processes may be going on than those initiated by current use. The user has to cope with unexpected system events, and needs to interpret the system's behavior in relation to the intended task as well as to the state of the system and the network of related systems. For this interpretation, users need an adequate mental model of the system and its relation to the current task.


• A user may accept a slow response to a query if he is aware that the answer is quite long and network traffic during office hours is heavy.

Mental models as developed by users of a system are always just models. They abstract from aspects the user considers not relevant and they have to be usable for a human information processor with his restricted resources and capacities. Consequently, we observe some general restrictions in the qualities of human mental models of computer systems. It has been shown that mental models of systems of the complexity of a computer application have the following general characteristics:

• Mental models are incomplete and users are generally aware of the fact they do not really know all of the details of the system, even if relevant. However, experts will know where the details can be found.


• Mental models can only be partly run because of the nature of human knowledge representation.


• A user may know how to replace a global replacement in a text editor (i.e. the start and end locations) without knowing how the intended effect is obtained.


• Mental models tend to be unstable. They change over time, both because users of complex computer systems may need to use different or upgraded systems which tends to spoil the knowledge of previously applied systems and because new experiences can also change the model.


• This can be the case even if the user is considered a guru on the system for the past ten years.


• Mental models also have vague boundaries. People tend to mix characteristics of their word processor with aspects of the operating system, and hence, are prone to occasionally make fatal errors based on well-prepared decisions.


• Mental models can also be parsimonious that is people like to maintain models that are not too complex and try to stick to enough basic knowledge to be able to apply the model for the majority of their tasks. If something uncommon has to be done they accept having to do some extra operations, even if they know there should be a simpler solution to those exceptions.


• And finally, mental models can have the characteristics of superstitions that help people feel comfortable in situations which they do not really understand. An example might be the experienced user who changes back to his root directory before switching off his machine or before loggin out. He knows perfectly well there is no real need for this, but he prefers to behave in a nice and systematic way and hopes the machine will behave nicely and systematically in return.

Designers of user interfaces should understand the types of mental structures users tend to develop. There are techniques for acquiring information about an individual user's mental models, as well as about the generic mental structures of groups of users. Psychological techniques can be applied and, in the design of new types of system, it is worthwhile to apply some expert help in assessing the knowledge structures that may be needed for the system, as well as those that may be expected to be developed by users. If the knowledge needed differs from the mental models developed in actual use, there is a problem and designers should ask for expert help before the design results in an implementation that does not accord with the user's models.

User Interface Design Conceptual Models

A central part of user interface design is the stepwise-refined specification of the system as far as it is relevant for the user. This includes the knowledge the user needs in order to operate the system, the definition of the dialog between user and system, and the functionality that the system provides to the user. All that is specified in the design process is obviously also explicitly modeled, in order to make sure implementation does not result in a system that differs from the one intended. In cognitive ergonomics, all that is modeled about the system as far as relevant to its different sets of users is called the conceptual model of the system.

Formal design modeling techniques have been developed in order to communicate in design teams, to document design decisions, to be able to backtrack on specifications, and to calculate the effects of design specifications. Some techniques use competence models which model the user's knowledge, or process models which focus on the interaction process, other modeling techniques use both competence models as well as process models. Reisner's Psychological Backus-Naur Form (BNF) is an example of a competence model. In this model, the set of valid user dialogs is defined using a context-free grammar. Process models may model time aspects of interaction, as in the Keystroke model which give performance predictions of low-level user actions. Task Action Grammar (TAG) is an example of a combined model. It allows the calculation of indexes for learning (the time needed to acquire knowledge) and for ease of use.

The conceptual model can be looked upon from 3 different viewpoints:

1. The psychological view considers the specification as the definition of all that a user should understand and know about the new system.


2. The linguistic view describes the interaction between human and system in all aspects that are relevant for both participants in the dialog.


3. The design view specifies all that needs to be decided about the system from the point of view of the user interface diagram.

Moran(1981) distinguishes six levels in the conceptual model, structured in three components. Each level details concepts from a higher level, from the specific point of view of the current level. The formalism that Moran proposes would now be replaced by more sophisticated notations, but the architectural concepts show the relevance of analyzing design decisions from different viewpoints and at the same time investigating the relationships between these viewpoints:

1. Conceptual Compenent: This component concerns design decisions at the level of functionality: what the system will do for the users:


1. Task Level: At this level we describe the task domain in relation to the system:


• Which tasks can be delegated to the machine.


• Which tasks must be done by the user.


• From the users perspective - Representations concern Tasks and Task Related objects:


2. Semantic Level: Semantics in the sense of Command Language Grammar concern the system's functionality in relation to the tasks
   • concerns system's functionality in relation to tasks that can be delegated.
   
   
   • Task delegation is specified in relation to the system.
   
   
   • System objects are described with their attributes and relevant states.
   
   
   • Operations on objects specified as a result of task delegation.
   
   
   • Example:
   
   
   • Object: Letter
   
   
   • Attribute: Print Date
   
   
   • Attribute: Storage Date
   
   
   • Attribute: File size
   
   
2. Communication Component: This component describes the dialog of the Seeheim model.


1. Syntax Level:


1. describes dialog style.


• Menus


• Form Filling


• Commands


• Answering Questions


2. Specifies Lexicographical Structure of user and System actions.


3. Store Letter Example:


• User Indicates letter to be stored.


• User Indicates Storage Location.


• User Initiates Storage Command.


• User Executes End of Command.


2. Keystroke Level: The physical actions of the user and the system are specified at this level:


• Clicking Mouse Buttons.


• Pointing


• Typing


• Dragging


• Blink cursor


• Audio Signals


3. Material Component: Domain of classical Ergonomics


• Presentation Aspect


• Interface Aspects


• Hardware Aspects


1. Spatial Layout Level: or screen design


• Character shape and color.


• icon shape color and placement.


• Button size and Layout


• Sound and tactile interface aspects.


2. Any other Relevant Hardware Aspects:


• button Shape


• Pressure needed to activate a button.

The Command Language Grammer outlined above provides a fairly complete template for the user interface. The layers and their relations are important, as well as how the user interface specification includes Conceptual Component, Communication Component and Material Component.

Interactive System Design

Traditional user interface design mainly concerns the situation of a single user and a monolithic system. In current applications, computers are mostly part of a network, and users are collaborating, or at least communicating with others through networks. Consequently, the User Interface should include all aspects of distributed computing and networking, as far as this is relevant to the user, such as access, structural and time aspects of remote sources of data and computing. For example, when using a web browser, it is relevant to understand mechanisms of caching and of refreshing and reloading a web page, both in terms of the content that may have changed since the previous loading operation and in terms of the time needed for operations to complete.

These newer types of applications bring another dimension of complexity into view. People are collaborating in various ways mediated by information technology. Collaboration via systems requires special aspects of functionality. It requires facilities for the integration of actions originating from different users on shared objects and environments, facilities to manage and coordinate the collaboration, and communication functionality. Such systems are often denoted as groupware. Modern user interface design techniques cater for both the situation of the classical single user system and groupware. This distinction is likely to disappear in the future.

There are several classes of stakeholder in system development. These include, the clients which are the people or organizations that pay for the design or acaquisition of systems, and the users which are the people that apply the systems as part of their daily work, often referred to as the end users. Throughout design these two classes of stakeholders have to be distinguished, since they may well have different goals for the system, different knowledge about the task domain, and different views on what is an optimal or at least acceptable system. This does not mean that in certain situations these classes will not overlap. But even if this is the case, individual people may well turn out to have contradictory views on the system they need. In many situations there will be additional classes of stakeholders to cater for, such as people who are involved in maintaining the system, and people who need traces or logs of the system to monitor cases of failure or abuse, such as lawyers (never a good thing when they get involved).

In relation to these different classes of stakeholders, designers are in a situation of potential political stress. Clients and users may have contradicitory inputs into the specification of the system. Moreover, the financial and temporal constraints on the amount of effort to be invested in designing the different aspects of the system, such as functionality, user interface specification and testing. These different aspects tend to counteract the designers' ambitions to sufficiently take care of the user's needs.

Making a distinction between classes of stakeholders does not solve the problem of user diversity. In complex systems design, we are confronted with different end users playing different roles, as well as end-user groups that have knowledge or a view on the task domain that need not be equivalent to the knowledge and views of each individual member.

Design as an Activity Structure

Viewing design as a structure of interrelated activities, we will need a process model. The model we will use will be a cyclical process with phases devoted to analysis, specification and evaluation. Figure IA-1 depicts this process model.

Figure IA-1: A process model for user interface design

Analysis: Since the system to be developed will feature a Task Level we will start with Task Analysis. We will further break the task level down to Task Level 1 which models the current task situation, and Task Level 2 which models the future task situation. The future task situation deals with where the system to be developed will be used, including changes in the organization of people and work procedures. The relationship between Task Levels 1 & 2, then, is how implementation of the system will change the structure and organization of the recipient of the system.

The development of Task Level 2 from Task Level 1 uses knowledge of current inadequacies and problems concerning the existing task situation, needs for change as articulated by the clients, and insight into current technological developments.

Specification: The specification of the system to be designed is based on Task Level 2. The system specification needs to be modeled in all details that are relevant to the users, including cooperation technology and user-relevant system structure and network characteristics. Differences between specification of the new system and Task Level 2 must be considered explicitly and lead to the proper design iteration.

Evaluation: The specification of the new system incurs many design decisions that have to be considered in relation to the system's prospective use. For some design decisions, guidelines and industry standards must be used as checklists. In other situations, formal evaluation may be applied, using formal modeling tools that provide an indication of the complexity of use or learning effort required by proposed users of the system. For other design decisions, however, proper evaluation will require intended user feedback with relevant aspects of the intended system. Some kind of prototyping is a good way to let the intended user evaluate the proposed solution. A prototype allows experimentation with selected elements or aspects of the final implemented user interface. The prototype enables imitation of the presentation interface, and the prototype allows the user to express himself in the interaction language, and can even be used to simulate aspects of the system's functionality including organizational and structural characteristics of the intended task structure.

Design as a multi-Disciplinary Collaboration

The main problem with the design activities discussed up to this point is that different methods may provide conflicting viewpoints and goals. A psychological focus on individual users and their capacities leads to neglecting the goals and methods of the task domain. On the other hand, sociological approaches towards groupware design tends to omit individual knowledge and needs. Nevertheless, consideration of both extremes can provide some unique contributions.

In order to design for people, we need to consider both sides of the coin: on one side are the individual users and clients, on the other side are the structure and organization of the group for which the system is intended. We need to know the individual's knowledge and views on the task, on applying the technology, and the relation between using the technology and the user's expertise, knowledge and skills. With respect to the group, we need to know the group's structure and dynamics, we will need to study the phenomenon of group knowledge, work practice and culture. These aspects are needed in order to acquire insight into both existing and projected task situations where new cooperation technology is introduced. Both types of insight, individual and group dynamics, are needed for the proper design decisions. Consequently, in prototyping and field-testing, we need insight into the acceptance and use by individuals and the effect of the new design on group processes and complex task dynamics.

For example, in a traditional bank setting, the client and the bank employee are on different sides of the counter. The bank employee is probably using a computer, but the client cannot see the screen, and does not know what the clerk is doing. In a service oriented bank setting, the clerk and client may both be looking at the screen together. They are searching for a solution to the client's question or needs. This overturns the existing culture of the bank, so an ethnographer may be asked to assess what this change in situation may bring about.

The general framework for our approach to user interface design is now depcited in Figure IA-2, which is a refinement of Figure IA-1:

Figure IA-2: Design Team Activities

Task Level 1 is based on knowledge of single users in terms of knowledge, skills, psychological variables, task-related skills. Task Level 1 is also based on complex phenomena in the task situation in terms of official procedures, variations in strategies, variations in the application of procedures and the different roles the users play. The integration of this insight into a model often does not provide a single or even the best decomposition of tasks. The model also may not provide a unique structure of relationships between people, activities and environments. However, the model often shows alternative ways to perform a certain task, role specific and situation specific methods and procedures, and a variety of alternative subtasks to people. for example the joint problem solving approach to the bank counter sketched out above, cannot be applied to the ATM. The ATM requires a different approach and a different user interface.

Between the restrictions imposed by Task Level 1 and because of client requirements, compromises often have to be made in defining Task Level 2, the new task situation for which the technology has been designed. This process includes the interpretation of problems in the current Task Level (Task Level 1), negotiating with the client regarding his requirements, and the resources available for design (including both time and financial constraints). Ultimately, decisions have to be made about complex aspects, such as re-arranging the balance of power and the possibilities for users in various roles to exercise control.

Task Analysis

Analyzing a complex system means analyzing the world in which the system functions, according to ISO 9241, 1966, the context of use comprises:

• the users


• the tasks


• the equipment (hardware, software, and materials)


• the social environment


• the physical environment

If we design systems for the context of use, we must take these different aspects of the task world into consideration. In traditional literature on task analysis from the Human Computer Interaction mainstream, the focus is mostly on users, tasks and software. On the other hand, design approaches for groupware and computer-supported collaborative work (CSCW) often focus on analyzing the world from the point of view of the physical and social environment. Recent approaches to task modeling include some aspects that belong to both categories, but one view needs to dominate.

Task Analysis in Human Computer Interaction Design

Classical Human Computer Interaction features a variety of concepts regarding class analysis. Task analysis can include the following activities:

• Analyzing a current task situation


• Envisioning a task situation for which information technology is to be designed


• Specifying the semantics of the information technology to be designed

Many Human Computer Interaction task analysis methods combine more than one of these activities and relate them to actual design stages. The design of a new system is triggered by an existing task situation. Either the current method of performing tasks is not considered to be optimal, or the availability of a new technology is expected to allow an improvement over current methods. A systematic analysis of the current situation may help formulate requirements and allow later evaluation of the design. In all cases, where a current version of the task exists, it pays to model this first. Task models of this type pretend to describe the situation as it can be found in real life, by asking or observing people who know the situation. Task Level 1 is often considered to be generic, indicating that expert users typically have similar skillsets.

Many design methods in Human Computer Interfacing that start with task modeling are structured in a number of phases. After describing a current situation in Task Level 1, the method requires a re-design of the task structure in order to include technological solutions for problems and technological answers to requirements. Task Level 2 is in general formulated and structured the same way as Task Level 1. However, it is not considered a descriptive model of user's knowledge, though in some cases it may be applied as a proscriptive model and expert user of the new technology should possess.

A third type of modeling activity focuses on the technology to be designed. This may be considered part of Task Level 2. However, some Human Computer Interaction approaches distinguish specific design activities which focus on the technology. This part of the design activity is focused on a detailed description of the system as far as it is of direct relevance to the end user. The design of the user interface should be separated from the design of Task Level 2. This is mainly because the User Interface models the solution in terms of technology, while Task Level 2 focuses on the task structure and work organization. In actual design, an iterative process occurs between these design models. This should be an explicit activity, making the implications of each obvious in its consequences for the other.

Human Computer Interaction task models represent a restricted point of view. All Human Computer Interaction task modeling is rather narrowly focused, mainly considering individual people's tasks. Most Human Computer Interaction approaches are based on cognitive psychology, which means they focus on the knowledge of experts in the task domain, whether this domain exists or still has to be restructured by introducing new technology.

As a consequence of their origin, Human Computer Interaction task models seldom provide an insight into complex organizational aspects, situational conditions for task performance, or complex relationships between tasks of individuals with different roles. Busines processes and business goals (such as the service focus of a modern bank counter) are seldom part of the knowledge of individual workers and consequently, are seldom related to the goals and processes found in Human Computer Interaction task modeling.

Task analysis assumes there are tasks to analyze. Many current Web applications, though, are information-centric. The value of such systems is in the information they provide, and to a much lesser extent in the tasks they offer to the user. The key operation is searching. The developers seek to make searching simpler by organizing the information in a logical way, or by providing navigational schemes. The challenge for such applications is to induce the users to provide ever more information.

Analysis Approaches for Collaborative Work

Computer-supported collaborative work (CSCW) stresses the importance of situational aspects, group phenomena and organizational structure and procedures. Shapiro (1996) even goes so far as to state that Human Computer Interaction has failed in the task analysis for cooperative work situations; since generic individual knowledge of the total complex task domain does not exist. The Computer-Supported Collaborative Work (CSCW) literature strongly advocates ethnographic methods.

Ethnographers study a task domain by becoming a participant-observer, if possible with the status of an apprentice. The ethnographer observes the world through the eyes of a newbie and at the same time is aware of his status as an outside observer whose final goal is to understand and describe for a certain purpose and to a certain audience (in the case of CSCW, a design project). Ethnographers start their observations purposely without a conceptual framework of task knowledge, but instead focus on activities, environments, people and objects. The choice of focus of ethnography itself comes from prior ethnographic observations, illustrating the bootstrapping character of knowledge elicitation in ethno-methodology. Methods of data collection currently start with video recordings of relevant phenomena followed by systematic transaction analysis, where inter-observer agreement serves to improve the reliability of interpretation. Knowledge of individual workers in the task domain may be collected as far as it seems to be relevant, but it not a priority, it is not considered the main source and is never considered indicative of generic task knowledge.

The ethnographic approach is unique in its attention to all relevant phenomena in the task domain that cannot be verbalized explicitly by all experts. The approach attends to knowledge and intentions:

• specific for some actors only.


• of conflicting goals.


• of cultural aspects not perceived by actors in the culture.


• of temporal changes in beliefs


• of situational factors that are triggers or conditions for strategies.


• of non-physical objects such as stories, signatures and symbols.

Ethno-methodology covers the methods of collecting information that might serve as a basis for developing Task Level 1, and no more than this since ethnomethodology only covers information on the current state of a task domain. However, the methodology for the collection of data and its structuring into a complete task domain description is often rather special and difficult to follow in detail. The general impression is that Computer Supported Collaborative Work design methods skip the explicit construction of Task Level 1 and Task Level 2 and, after collecting sufficient information on the community of practice, immediately embark on specifying the User Interface, based on deep knowledge of the current task situation that is not yet formalized. This may cause two types of problems:

1. The relationship between specifications for design and analysis of the current task world might depend more on intuition than on systematic design decisions.


2. Skipping the development of Task Level 2 may lead to conservatism with respect to organizational and structural aspects of the work for which a system is to be designed.

Knowledge Sources and Collection Methods

Collecting task knowledge to analyze the current situation of a complex system has to start by identifying the relevant knowledge sources. The relevant knowledge sources are depicted in the following diagram:

Figure KD-1: Knowledge Dimensions for Complex Task Domains

The two dimensions of this framework are Levels of Communication and Sources of Knowledge. These dimensions denote where knowledge resides and how it can be communicated. For example, A stands for the explicit knowledge of an individual while D stands for the implicit knowledge of a group.

This framework has been applied in actual design processes for large individual and government interactive systems. This framework provides a map of knowledge sources that help us to identify the different techniques that we might need in order to collect information and structure this information into a model of the task world.

The following techniques may be used to gather task knowledge:

The following techniques may be used to gather task knowledge:

• Cell A:


• Interviews


• Questionnaires


• Think-Aloud Protocols


• Single Person Observations


• Cell B:


• Task Behavior Observations


• Hermeneutic Methods to Interpret Mental Representations


• Cell C:


• Study of Procedures


• Study of Standards


• Study of Protocols


• Study of Requirements Documents


• Cell D:


• Interaction Analysis


• Cognitive Task Analysis


• Critical Decision Method Analysis


• Decompose, Network and Assess Method

It must be noted that knowledge from one cell may be in conflict with what may be learned from other sources. For example, explicit individual knowledge is abstract with respect to observable behavior and ignores the situation in which the task behavior is exhibited. Furthermore, explicit group knowledge expressed in official rules and time schedules is often in conflict with actual group behavior. In all cases of discrepancy between sources of task knowledge, ethnographic methods will reveal unique and relevant information that has to be represented explicitly in Task Level 1.

The allocation of methods to knowledge sources should not be taken too strictly. The knowledge sources may not be completely located in a single cell of the Knowledge Dimensions diagram above. The main conclusion is that we may need different methods in a complementary sense, just as we need information from different knowledge sources.

An Integrated Approach to Task Analysis

Groupware Task Analysis is an attempt to integrate the merits from the most important Human Computer Interaction approaches with the ethnographic methods applied for Computer Supported Collaborative Work. Groupware Task Analysis contains a collection of concepts and their relations that allows analysis and representation of all relevant concepts regarding human work and task situations.

The Groupware Task Analysis framework is intended to structure Task Levels 1 & 2, and hence, to guide the choice of techniques for information collection in the case of Task Level 1. For Task Level 2, design decisions have to be made, based on problems and conflicts that are present in Task Level 1 and the requirements specification.

Task models for complex situations are composed of different aspects. Each describes the task world from a different viewpoint and how each relates to the others. The three viewpoints are:

1. Agents: Often people, either individually or in groups. We make a distinction between agents and the roles they play.


2. Work: in both its structural and dynamic aspects. We take task as the basic concept. A task has a goal as an attribute. We make a distinction between tasks and actions. Tasks can be identified at various levels of complexity. The two task types we are concerned with are:


• Unit Task: The lowest task level that people want to considerin referring to their work.


• Basic Task: The unit level of a task delegation that is defined by the tool that it is used by.


• Example: A single command in a command-driven computer application.



Unit tasks are often role-related. Unit tasks and basic tasks may be decomposed further into user actions and system events. The proper frame of reference utilizing the corresponding task must be applied to really understand the unit tasks.

• Example: Hitting a return key has a differnt meaning depending on whether it ends a command or starts a newline in a text editor.



The task structure is often at least partially hierarchial. On the other hand, performance on certain tasks may influence the procedures for other tasks (possibly with other roles involved).

• Example: A secretary needing to deliver mail on time, likely will need to interrupt other tasks in order to get the mail delivered on time.



Therefore, we need to understand both task flow and data flow over time as well as the relationship with other related flows.

3. Situation: Analyzing a task world from the viewpoint of the situation means detecting and describing the environment and the objects in the environment. Each thing that is relevant to the work in a certain situation is an object of task analysis. Objects may be physical or conceptual:


• Messages


• Gestures


• Passwords


• Stories


• Signatures



The task environment is the current situation for the performance of a certain task. It includes actors with roles as well as conditions for the performance of a certain task. It includes actors with roles as well as conditions for task performance. The history of past relevant events is also part of the task environment.

User Interface Specification Details

Specifying details of the user interface means elaborating on all aspects that are relevant for the user, hence the concept of a User Virtual Machine. When there are several types of user for the system, we need to specify several separate User Virtual Machines, since each user role is defined by another subset of tasks. As suggested by the Seeheim Model, we need to consider the representation which is the dialog and the application, each in relation to the type of user on which we are focusing. The first aspect we need to consider in the specification is the functionality of the system.

The other activities in relation to the specification of the User Virtual Machine concern the interaction between the user and the system, with its two aspects:

1. Dialog: Corresponds to the dialog layer in the Seeheim model, and the syntax used in Command Language Grammar (CLG).


2. Presentation: Corresponds to the presentation layer in the Seeheim model as well as the spatial layout level.



The 3 components of the User Virtual Machine (dialog, presentation, functionality) lead to 3 specifications that are strongly related. The dialog and the presentation are two sides of the user-system interaction, and both express the semantics and functionality of the system.

User Interface Dialog

The dialog in modern user interfaces is frequently a combination of several dialog styles, such as command language, menu choice, answering of questions generated by the interface, fill-in-the-blanks forms, and direct manipulation of interface objects. The following list gives metaphors of dialog style implementation:

• In a command language (command prompt, python) or natural language (C/C++, Visual Basic, Pascal ), the user and the interface 'speak' to each other, well, at least communicate. The user feels in control as long as he understands the possibilities and remembers the right terminology. The user does not perform the task directly but is rather obliged to persuade the system to perform it.


• When the user has to choose from a menu, click on an icon, fill in labeled slots in a form on the screen, or answer questions from the interface, the interface provides a structure where the user is prompted to react by selecting an option. In this case, the user does not have to remember too many options, since they are right in front of him, on the screen. On the other hand, a more experienced user may not feel totally in control of the dialog.


• In a direct-manipulation-type interface, the user moves in a space, and by acting in the space, shows the computer what he needs. The space can be represented on a two-dimensional screen, where the user can drag and drop icons, a 3-D virtual reality environment, or something in-between (augmented-reality interface). The direct manipulation style provides the user with a sense of direct engagement. To the user, it feels as if there is direct contact with objects from the task domain.



It makes sense to relate the metaphor to the actual task domain. For example, a space can be specified as a desktop, or a virtual library, depending on whether the task domain concerns the management of documents or a search for objects.

In specifying the dialog further, the syntax of the interaction has to be specified in detail, including the list of possible user actions. For example, in both the Windows and Macintosh interface, the user has to indicate the object first and then the operation. The reverse is true in the Unix/Linux command language. Also, the lexicon (word(s) or expressions) has to be specified. The includes deciding on the verbal labels for objects and operations, the iconic representation:

or gestures: and sounds. The lexicon includes both the symbols that the user can use in communicating with the machine and the symbols that the machine will output to the user.

There are various modeling techniques for specifying the dialog. An example dialog specification technique is User Action Notation (UAN), which combines the specification of functionality with dialog details. User Action Notation (UAN) is relatively easy to use, especially since the actual formation may to a large extent be tailored to the designer's needs without losing the benefits of a formal model. User Action Notation (UAN) provides a representation of this specification of the User Virtual Machine (UVM) with separate columns to specify user actions, interface actions, interface states, and connections to underlying events. Each of these columns is filled in with the amount of detail the designer needs at a certain moment. Later on, any item may be refined.

User Virtual Machine Representation

The aspect of the User Virtual Machine contains the details of the interface that determine the perceptible aspects. The lexicon indicated in the previous section includes all interface elements. Their representation concerns their shape, color, contrast, pitch, loudness, size, etc. Representation also concerns all aspects of screen design, such as the size of the windows, the speed of movement displayed, and the timbre of generated sounds. The representation aspect may need the assistance of artists or at least specialists who understand the laws of graphical representation, sound representation, and animation.

This part of design is the one for which no general formal representation models have been developed. Sketches, video clips, and verbal descriptions of what one intends to have represented are commonly used, and frequent communication between the designer and the implementor is needed. Artists may have very useful ideas but may propose very convincing solutions that change the essence of the intended specification.

User Interface Design Evaluation

Evaluation is needed whenever decisions are made in the course fo the design process. This will be the case during both analysis and specification. Evaluating analysis decisions is mostly related to the phase where the future task world is envisioned. During the process of developing detail specifications of the User Virtual Machine, many decisions have to be made that relate the technical details to aspects of usability.

Evaluation does not solve problems, it merely points to them. A valid diagnosis often requires several complementary techniques as well as expertise. Evaluation is done by presenting design decisions such that questions can be answered. Sometimes the questions may be asked in an unbiased way by the designer himself, but in most cases there is a need to include others in the evaluation process, either specialists in design or humanities, or stakeholders in the design.

Analysis Decisions Evaluation

During the early development of Task Level 2 there is not enough detail to develop a prototype. However, scenarios can be developed regarding the future task world, and it can be modeled with the help of formalisms. Both scenarios and formalisms offer opportunities to apply evaluation techniques.

In scenario-based evaluation, scenarios will be developed concerning those parts of Task Level 2 that concern a change in comparison to the current situation. In this phase of the design, it is worthwhile to represent not only the new use of technology, but also the situational aspects of this use and the organizational consequences. Developing a scenario means describing a process of using the intended system in an actual situation and organizational setting. The scenario may be described verbally, but a video representation is often used because that may show more relevant situational details, even though the technology may be represented in a way that represents no details.

In order to evaluate a scenario, a claims analysis can be performed. Apart from the designer and the constructor of the scenario, future users, representatives of the client, and any other relevant stakeholders may participate in the claims analysis. It often makes sense not to choose only representatives who have a positive attitude towards the intended change but also to include people whar are afraid of the future developments or who have a pronounced opinion on the negative effects of future implementation. For each aspect of the task delegation to the new system that is an explicit change from the current system, the participants of the clamis analysis try to identify:

• to what extent the change is providing a solution in the intended direction, and


• what the positive and negative side effects of the change will be.



Formal Evaluation Techniques allow specific questions to be answered. For example, a Hierarchial Task Analysis describes the task domain as a hierarchial structure of tasks and subtasks with a complete representation of the procedural structures of the decomposition. Hierarchial Task Analysis formal evaluation techniques may consider the length of a sequence of subtasks, given the relevant conditions. The same type of formalism allows an evaluation of possible modular procedures. Modular procedures can be represented as subtrees that describe the tasks and subtasks that need to be performed. This can be a useful tool in determining the amount of learning that will be required to implement the new task domain.

User Virtual Machine Specification Evaluation

As soon as details of the User Virtual Machine have been specified, other representations may be used for evaluation, even though it may still be worthwhile to analyze scenarios. At this stage in the design, scenarios are often elaborated and executed with the help of mock-up representations or simulations of the intended system, especially if hardware aspects are expected to influence system operation. The simulations in this evaluation are concerned with the size, weight and perceptible aspects of the intended User Virtual Machine. In this stage of design, it is not necessary for all interactive aspects to have been specified. Many evaluation techniques from classical ergonomics may be applied as well.

Formal evaluation will certainly play an important role for the User Virtual Machine. Many formalisms have been developed with certain usability aspects in mind. For example, Task Action Grammar (TAG) allows us to measure ease of learning and use. This is done by indexing the rules and the number of features to be considered during each command to the machine. User Action Notation (UAN) representations allow us to evaluate the learning difficulty of the system as well.

Detailed specification of the User Virtual Machine permits other evaluation techniques as well, in particular heuristic evaluation and cognitive walkthrough. In both cases, it is useful to employ experts in user interface design who have not been part of the team that developed the specification. Experienced designers know the criteria applied in the techniques and will have considered them during their decisions. Only somebody for whom the design is new and can have an unbiased view concerning these criteria. A last type of evaluation technique, user testing and observation, requires future users to be involved with a working prototype. Obviously, the prototype can also be a basis for the heuristic evaluation and cognitive walkthrough.

Heuristic evaluation is based on some definition of usability. Usability is a complex of aspects such as the ease of use, ease of learning, ease of recall after not using the system for a while (re-learnability), help and likeability. Approaches toward heuristic evaluation provide checklists of the characteristics of the User Virtual Machine that describe the different usability aspects. This checklist may also be used as a guideline for making design decisions. Checklists exist in different forms, and often include a technique for calculationg usability indexes. Each item may be checked when applicable, and, each item that is diagnosed as non-optimum can be resubmitted for design changes. Example items that may be on the checklist are:

• Use a simple and natural dialog: Humans can only pay attention to a few things at a time. Therefore, dialogs should not contain information that is irrelevant or rarely needed. Information should appear in a logical and natural order to ease comprehension.


• Speak the user's language: A dialog is easier to understand if it contains concepts and phrases that are familiar to the user. The dialog should not be expressed in computer-oriented terms, but in terms of the task domain.


• Minimize memory load: The user should not have to remember information from one part of the dialog to another. Long command sequences, menus with a large number of entries, and uncertaintly about where the &%!! are we? hamper interaction. There must be easy, obvious ways to find out what to do next, how to retrace steps, and get instructions for use.


• Be consistent: Users should not have to wonder whether different words or actions mean the same thing. Metaphors need to be chosen carefully, so as not to confuse the user.


• Provide feedback: The system should keep the user informed of what is going on. If certain processes take a while, the user should not be left in the dark (metaphor). A moving widget informs the user that the system is doing something; a percentage done indicator can inform the user about the status of the progress towards the goal.


• Provide clearly marked exits: Users make mistakes, activate the wrong function, and even follow the wrong thread. There must be an easy way to leave an unwanted state.


• Provide shortcuts: Novice users may be presented with an extensive question-answer dialog. It gives them a safe feeling and helps them to learn the system. Experts are hindered by a tedious step-by-step dialog and the system should provide shortcuts to accomodate them.


• Give good/helpful error messages: Error messages should be explained in plain language. They should not refer to the internals of the system. Error messages should precisely state the problem, with recommended options and solutions.



Items from a checklist should be used with care. Design involves tradeoffs between conflicting goals. Consider the two function-key layouts:

Figure Eval-1: Two possible cursor key arrangements




Interface designers might prefer the STAR configuration on the left. It is a symmetric design, aesthetically pleasing and consistent with directional indicators on a compass. Yet, studies have shown that the Inverted-T on the right is the most useful configuration. With the index finger on the cursor-left and the ring finger on cursor-right, the middle finger can efficiently cover the cursor-up & cursor-down keys. Designers of computer games have been aware of this arrangement for some time.

Cognitive walkthroughs focus solely on the cognitive aspects of the dialog. The technique can be applied early in the specification phase and it helps to detect failures that would otherwise make the system incomprehensible for users. Like heuristic evaluation, cognitive walkthroughs require only a small number of expert colleagues who are confronted with the specification of the dialog and the content of the presentation. Screen design need not have finished yet, but the information representation (wording & icons) of the screen must be known. Cognitive walkthroughs require a specification of scenarios of faultless dialog examples. A scenario might for example read as follows: When checking in you have to put your credit card in the slot; the machine will ask you to enter your pin code; you type.... Based on a specification of this type, the start state is described to the evaluator, along with all the information on the task, as the user should know it, as well as the background of the user (education, training and system experience). The evaluator is asked to answer a small set of questions about the next move. After these have been answered, the next intended move is described with the resulting state of the interface, after which the evaluator is asked the next set of questions.

An example of such a set of questions is:

• What should the user do now?


• Based on what knowledge or information would the user do this?


• What would the user expect to be next step of the system?



Techniques such as this quickly expose any invalid expectations of user knowledge and understanding and any lack of relevant information at the interface. Additionally, they show inconsistencies in the interface information as well as inconsistencies in dialog conventions.

Prototype Evaluation

The proof of the pudding is in the eating. An evaluation should, at some point, be carried out with real users. As soon as a mock-up or simulation is available, users can be asked to perform tasks. We ar then asking users to play a scenario, even if we do not mention this explicitly. As soon as the details of the dialog have been established, a more interactive type of testing is possible.

Evaluation by the user is often based on a prototype implementation of the system. In early phases of detail design, the prototype will often not represent the total User Virtual Machine. A single aspect of the task to be delegated, one instance of the dialog and representation with hardly any real functionality available, may be enough to detect problems early on. Several types of user testing may be done. Users may be provided with tasks or asked to explore. Observations during the interaction, complemented with subsequent discussions with the user, may reveal problems and misunderstandings. In addition, user performance may be measured by the time to complete a task, or the frequency of errors. Users may be asked for their subjective understanding of the system. Standardized measurement techniques are commercially available, focusing on subjective learnability, ease of use, and mental load.

Summary

The user interface of a system is important. About half of the code of an interactive system is devoted to the user interface. Consequently, about half of the time and effort is spent on that interface as well. The quality of the user interface is a critical success factor. Good user interfaces increase the efficiency and productivity of their users, reduce errors and training time, and improve user acceptance.

In a technical sense, the user interface of a system often consists of two components or layers:

• A presentation layer that handles the perceptible aspects of the interface, including screen design and keyboard layout.


• A dialog layer that handles the syntax of the interaction, including symbol generation and interpretation as well as help functions and error messages.



As a result of this limited view of what a user interface is, it is often designed independently of the system's functionality. The design of the user interface is then seen as a separate activity, not in the mainstream requirements-engineering-design-implementation-testing phases. Chances are by then, it will not get the attention it deserves.

It would help to develop a much more user-friendly system if the design of the interface and the design of the functionality go hand in hand. An even more extreme, but perhaps even preferable stance could be: The User Interface is the System. There are two main reasons why to take this broader view of what a user interface is:

• The system, and hence its interface, exist to help the user perform certain tasks. The user interface should then reflect the structure of the task domain. The design of tasks and the design of the corresponding user interface influence each other and should be part of the same cyclical process. Like quality, the user interface is not a supplement.


• The dialog and representation alone do not provide sufficient information to the user. In order to be able to work with a system, the user sometimes needs to know: What is going on behind the screen?.



When thinking about user interface design, it is important to make a distinction between the User's Mental Model, User Virtual Machine, and the Conceptual model.

The Mental Model is a model in human memory. It is the user's model of the system he uses. It is based on education, knowledge of other systems, knowledge of the application domain, and general knowledge about the world. The mental model is used during interaction with the system, to plan actions and interpret system reactions. The mental model is often incomplete and inconsistent.

The User Virtual Machine (UVM) includes everything the user should know in order to use the system. The UVM includes aspects ranging from the physical outlook of the computer and connected devices to the style of interaction and the form and content of exchange.

the Conceptual Model is the explicit model of the system created by the designers and teachers. It is a consistent and complete representation of the system as far as relevant to the users. The conceptual model shows itself in the interface. If there is only one class of users, the user virtual machine and the conceptual model are the same. If there is more than one class of users, or the user uses more than one device, there is one UVM for each class, and the conceptual model is the union of all the different UVMs.

The central issue in Human Computer Interaction is to attune the user's mental model and the conceptual model as closely as possible. When this is achieved, the system becomes easier to use and understand. When the models conflict, the user gets confused and starts making errors. Good design starts with the derivation of a conceptual model from an analysis of users and their tasks. The conceptual model is then built into the system in such a way that it induces adequate mental models in the users.

The design of a user's interface involves different disciplines. Psychologists know how humans perceive, learn, remember, think and feel, how people work together and how work situations affect work. Cognitive psychology, anthropology, and ethnography provide techniques for collecting information on how people work. Artists know how to make an attractive design while providing effective functionality. Cognitive ergonomics is concerned with the characteristics of human information processing in interacting with information systems. Design teams for interactive systems may thus contain quite a variety of expertise alongside software engineering.