METHODS

Focus Group Testing

Focus Group testing entails presenting an issue or product to people gathered in a group and collecting their reaction. A skilled moderator poses questions and special scenarios to elicit discussion which is typically recorded so the opinions and perceptions can be analyzed later. Traditionally, the outcome is a subjective picture of the group’s responses. To achieve more actionable outcomes, Usability Associates augments the traditional method with other more objective techniques either during the group session or in an augmented session. The result is a rich combination of subjective and related objective data that can inform product design at various stages of development. See: Caplan, Stanley H., “Using Focus Group Methodology for Ergonomic Design”, Ergonomics, Vol. 33, No. 5, May, 1990

• When used: When user perspectives and feedback are sought and the interplay of the participants may bring out additional information than with one-on-one interviews; when time for user research is constrained, and when one-on-one interviews would be difficult because participants are not readily available;
• Pros/Cons: The interplay of the participants may bring out additional information than with one-on-one interviews; the skill of the facilitator is very key, to both ensure that all participants are contributing, to be mindful that the goals are being fulfilled (questions being answered), and to keep the session moving forward within the set meeting time.

Surveying

Surveys are a flexible tool for remote data collection.

• When to use:
• Pros/Cons:

Conjoint Analysis

Conjoint analysis is typically a marketing research technique used to determine the priority of product characteristics (i.e., features/attributes/price) preferred by prospective customers for the product. The output of the data collected is a relative numerical value for each characteristic that identifies the strength of its preference or perceived importance. The conjoint analysis method can be used for user experience analysis by including usability features, user interaction touchpoints, and customer touchpoints as the characteristics studied.

Self-Reporting Diary

A self-reporting diary is a useful tool when conducting ethnographic type, long term studies.

• When used:
• Pros/Cons:

Ethnographic study

An ethnographic study, in the purest sense, is a type of contextual study where the observer(s) is the proverbial “fly on the wall”. That is, the observer has no  interaction with the person(s) being studied, so that any influence that may affect the natural process is eliminated. An ethnographic study may be combined with a contextual inquiry component in order to probe on or validate observation notes, and thereby accelerate the study.

Lead User Analysis

Lead User Analysis is a research method making use of the knowledge and motivations of ‘lead users’ in a particular discipline or product user group who–

-face the needs that will be general in the market place, but face them months or years before the bulk of that marketplace encounters them

-They are positioned to benefit significantly by obtaining a solution to those needs

A challenge to this method is identifying and finding lead users. The underlying trends of use of a product can be clues to point to lead users. Lead users are also typically users of complementary or customizers of existing, typical products/interfaces. They demonstrate an interest in improving the product domaine.

Lead users are recruited to participate in the design phase of a new or reengineered product.

Pugh Controlled Convergence Analysis

A Pugh Concept Convergence Analysis (PCCA) is a structured technique for analyzing the design merits of multiple alternative concepts for a given product. The technique, which involves discussing and rating the attributes of each concept, can be employed by a team of subject matter experts or by people identified as target prospective customers/users. PCCA has been successfully used during focus group sessions with prospective users to give a rich combination of qualitative and quantitative data. Analysis of the data can identify a minimal set of concepts for evaluating non-design considerations or an entirely new design concept comprised of the best attributes of the original concepts studied in the PCAA process.

Product Attribute Mapping

In this method, separate subjective scales for importance and for satisfaction are used to collect perceptions of attributes of a product or process.  The subjective responses, collected from current or prospective users, are analyzed and plotted on a graph marked off in quadrants.  An attribute’s position on the graph can be an indicator of how worthwhile it is to “fix” it.  Attributes in the high importance/high satisfaction quadrant are already in the target quadrant and don’t necessarily need “fixing” but budget might be productively spent to achieve or maintain the attributes.  In the high importance/low satisfaction quadrant, spending funds to increase an attribute’s perceived satisfaction, e.g., through focused marketing, would leverage the already perceived importance of the attribute.  Attributes in the low importance/high satisfaction quadrant are candidates for reducing associated budget spending to maintain satisfaction because they aren’t very important. Finally, the low importance/low satisfaction attributes are candidates for being eliminated, especially if they absorb a significant amount of cost.

FAST diagramming

FAST or Function Analysis System Technique is a value engineering diagramming method that can be used to show the hierarchical relationship of functions embedded in a system. The system can be a product, technical process, or organizational process and it can be used to evaluate an existing system or construct a new one. In either case, the FAST diagram is typically constructed by a group of subject matter experts lead by a facilitator knowledgeable about the method. Basic ground rules for constructing the diagram are (1) functions must be stated with an action verb and a noun, (2) each function must answer the question “how” for an associated higher level function and (3) asking the question “why” for any function should be answered by an associated higher level function. The structure of the diagram is not based on time or on system features. Benefits of using this method derive not only from the utility of the final diagram, but also from the insights gained by the discussion among group members while constructing the diagram. See: Caplan, Stanley H., Suzanne Rodgers and Harry Rosenfeld, “A Novel Approach to Clarifying Organizational Roles”, Proceedings of the Human Factors Society, – 35th Annual Meeting, September, 1991.

Literature Research

Literature research is a useful starting point to evaluate the state of a specific field….

Heuristic Evaluation

A heuristic evaluation is performed by a small panel of usability experts who first individually assess product usability and then reconcile with each other any differences in their assessments. This method can be used on product prototypes or on already commercialized products. Often it is used to compare competing products. Typically, it is less expensive to perform than conducting usability testing. The scope of the evaluation can be broadened to cover the whole user experience.

Usability Testing

Usability testing is a fundamental method for evaluating a design in process.

• When used:
• Pros/Cons:

Wizard of Oz

Wizard of Oz is an usability testing method whereby the response to a participant’s actions with a prototype user interface are generated in real time by a co-moderator “behind the scenes” of the testing environment. This methodology can be a viable approach when the cost or time to build the necessary interactivity in a prototype is out of scope, or an unimplemented embedded user interface is to be tested. Ideally (or as much as possible) the participant is unaware of the co-moderator. At the same time, the co-moderator must be able to observe and hear the actions of the participant.

Advantages of this method include simpler prototype development and thus time and cost savings, and the potential ability to “react” to more unplanned for participant’s actions, and the ability to test more complex feedback technologies such as voice/spoken responses.

Ideation and Brainstorming

In the early stage of design the goal should be to ideate a broad and large set of ideas that aim to address the assessed user needs (collected from a previous step). This is done in a time-boxed brainstorming work session, ideally with a team with diverse backgrounds and experiences. Stakeholders should also participate. The aim is quantity over quality: to have a large variety of ideas that may not be complete or particularly applicable individually, but can be nuggets for more detailed concept development. Furthermore, refined ideas and concepts can be derived by cross-pollinating incomplete concept ideas.

Mr. Potato Head

Mr. Potato Head is a technique for rapidly prototyping a hardware design. As the name suggests, the underlying process models the popular kids toy whereby different versions of  ‘Mr. Potato Head’ can be constructed by selecting from options of eyes, ears, noses, etc.  In the case of designing a new product, for example a blood pressure cuff, a foam piece representing the product’s core is then enhanced with  foam-core pieces representing buttons and displays, etc., each pulled from a pre-made library of model control options. Designers, human factors engineers and even target users can collaborate in assembling multiple concept devices. The resulting concepts (prototypes) can be iteratively evaluated and adjusted.  In many respects, this method is analogous to paper prototyping – except in 3D.

Affinity Clustering

Affinity clustering is an iterative, often group activity for categorizing raw data captured from a process analysis, e.g., series of contextual inquires. Clusters/ groups initially created may be split or combined, the goal of the iterative process being to build a meaningful information architecture that then helps defined the user experience design.

Card Sorting

Card sorting is a technique used to to develop the information architecture of a site or application. The outcome of the underlying activity is typically a hierarchy of topics and related descriptors that can serve as menu and sub menu options.

• When used:
• Pros/Cons: