Strategic Planning Banner quail logo

Summary Report of Front End Evaluation
Fossil Mysteries
FOCUS GROUPS REPORT
Marianna Adams, Ed.D., Hava Contini, M.S.
Institute for Learning Innovation, Annapolis, MD
November 1, 2001

This material is based upon work supported by the National Science Foundation under Grant No. 0004253.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the
authors and do not necessarily reflect the views of the National Science Foundation.

Introduction

The San Diego Natural History Museum (SDNHM) is undertaking a major redesign and installation of its permanent exhibitions. These new displays are designed to promote understanding of the evolution and diversity of Southern and Baja California and to inspire in all people a respect for the environment. Throughout this process the staff is committed to integrating a grounded and systematic evaluation process that will build organizational capacity. In addition, evaluation will help keep the exhibition team focused and able to make informed decisions at critical junctures in the design process. To this end, the Institute for Learning Innovation (Institute), a non-profit learning research and development organization based in Annapolis, MD, had conducted a multi-tiered front-end evaluation study.

This study comprised two phases: Phase 1 sought to assess the level of visitor knowledge and interest in general topics of natural history as well as determine the delivery systems that visitors preferred to use and learn from while in museums (see Front-End Research Visitor Interviews Report: Exhibit Choices, Interest, and Learning Preferences); Phase 2 drew upon the findings of Phase 1 and focused in on the exhibition area that was a primary interest to most visitors - fossils and prehistoric change over time. The intent of this second phase was to test activity and exhibit ideas for the seven thematic areas of the fossils exhibition.

This document summarizes the focus group results.

Assessing Response to Specific Activities and Exhibits in Fossil Exhibition

Methodology

To assess responses to the exhibition ideas from a broad audience, a series of ten focus group interviews were conducted between October 11-24, 2001 in the boardroom of the SDNHM. Group 1 was comprised of docents of the museum; group 2 consisted of marketing and PR professionals in other museums, the zoo, and media; group 3 was made up of educators; group 4 was SDNHM staff; group 5 was comprised of senior citizens; group 6 was families with young children; group 7 was an all Spanish-speaking group of families and young singles; group 8 consisted of professionals and individuals who represent visitors with special physical, developmental, and learning needs, group 9 was the museum's Public Programs Advisory Committee, and group 10 were youth aged 13-19. In addition, a series of naturalistic interviews were conducted with children, ages 6-10, who were involved in a separate activity/discussion while their parents were in group 6. In each focus group a SDNHM staff used conceptual drawings to explain the seven thematic sections of the fossils exhibition and describe a variety of activities and experiences in each section. The Institute researcher then led discussion and guided participants to complete the written component of selecting their most and least favorite activities and sections. In the children's group SDNHM volunteers conducted the interviews. (See Appendix B for the focus group protocol and written worksheet for adults and children.)

Description of the Sample

A total of 84 people of the 101 who participated in the ten focus groups completed the written surveys; the participants in group 4 were not asked to complete the written survey as the questions were not appropriate for SDNHM staff. Twenty-one children were interviewed by SDNHM volunteers. Attendance numbers at each focus group was as follows.

    Group 1 - Docent   8
    Group 2 - Marketing and PR   8
    Group 3 - Educators   18
    Group 4 - SDNHM staff   16
    Group 5 - Senior citizens   8
    Group 6 - Families with young children   12
    Group 7 - All Spanish-speaking group of families and young singles   7
    Group 8 -Special physical, developmental, and learning needs   7
    Group 9 - Public Programs   6
    Group 10 - Youth/teens   11
    TOTAL 101

Almost three-fourths (70%, n=57) of those who completed the written demographic data were female and 30% (n=25) were male. Most of the participants were in their 40s (24%, n=19), with an almost equal percentage of people in their 30s (19%, n=15); 17% (n=13) were in their 50s, 14% (n=11) were teens, 14% (n=11) were in their 60s, 12% (n=9) were 70 and older, and 5% (n=4) were in their 20s. Almost all of the participants were Caucasian (66%, n=54) with 16% (n=13) Latino, 11% (n=9) mixed race, 5% (n=4) African American, and 2% (n=2) Asian.

Response of Focus Groups to Fossil Exhibition Ideas

Focus group participants were asked to respond to a list of five title ideas for the entire fossils exhibition. The titles were:

    1. Mysteries from the Past
    2. Fossil Detectives
    3. Prehistoric Puzzles
    4. Ancient Clues
    5. Fossil Mysteries

There was no clear pattern as to a favorite title choice either within or across focus groups. For every person who liked a specific title, there was someone who felt there were problems in message or meaning with it.

Summary Observations about Exhibition Title: Despite what seemed like equally conflicting opinions and responses across focus groups, there were some patterns in the way focus group participants talked about the title issue even if those patterns were that there was a split in opinion.

Many people noted that the word "ancient" made them think of ancient cultures such as ancient Egypt or Rome. They did not associate "ancient" with prehistoric times. Many people felt that the word "fossil" needed to be in the exhibition title. They felt it was a commonly understood word that expressed most accurately what the exhibition would be about. Many people believed that fossils are what people come to see in such an exhibition. There was mixed response to the words mystery, detective, and puzzle. Some people liked those words because it felt interactive to them - that there would be something for people to figure out. Other people disliked these words because it made them think of a theme park or felt titles with these words would be better for educational material or as sub-titles, than for an exhibition.

The Spanish-speaking focus group found some titles as translated awkward and confusing, reminding exhibit planners of the importance of cultural context and syntax in language. Titles and exhibit text will not be successful if translated literally.

Exhibition Sections

Each focus group was somewhat unique in the way they responded to the ideas for the fossil exhibition yet, there were clear patterns about why they liked or did not like an activity or exhibit idea. Table 2 provides an overview of the ways participants rated the various activities in each section on a three-point scale with 1 being low and 3 being high. In addition there is the rating for each individual section. The ratings were ranked "Hot," "Warm," and "Cool." A hot rating means that the activity was well received and there were very few concerns or problems with the idea. A warm rating means that many people liked it but almost as many had some serious concerns about it. A cool rating means that most people did not like it or had serious concerns or problems with it. It is important to remember that people were asked to make a forced choice of the most and least liked activities. Many people did not list any least liked activities or indicated that they only listed something in a least liked column because they had to.

Table 2: Summary of Participant Rating of Activities within Sections

SECTION 1 - UNEXPECTED DINOSAURS
B - skeleton of dinosaurs
F - video viewscope
D - climb nodosaur
E - nodosaur clues animation
A - life-size dinosaur scene
C - interactive maps
G - interpreter ammonite shell
 
2.9
2.8
2.5
2.5
2.2
1.9
1.6
 
HOT



WARM

COOL
SECTION 2 - GLOBAL FORCES
A - multi-media show
F - Shine sun
B - assemble earth
C - heat core
E - evolution whales
G - El Nino
H - Computer global warm
I - Seismograph
D - vote extinction theory
 
2.8
2.7
2.7
2.6
2.0
2.1
1.9
1.9
1.7
 
HOT



WARM
SECTION 3 - PICTURE PAST LIFE
A - Eocene scene
F - Compare strength/agility
C - Mechanical models
B - Compare fossils/skeletons
D - Microfossils
E -Computer construction
 
2.8
2.7
2.4
2.3
2.0
1.7
 
HOT

WARM
SECTION 4 - RACE FOR SURVIVAL
A - Eye-level scene
B - Compare vision, etc.
G - Scene change
D - Treadmill
F - Extinct volcanoes
C - Assemble skeleton
E - Computer balance
 
2.6
2.8
2.7
2.5
2.3
1.9
1.4
 
HOT



WARM

COOL
SECTION 5 - SHIFTING LANDSCAPES
B - 3-d map
D - Large shark
G - Aquarium
A - Computer model
F - Gulf fossils
C - Compare LA/SD
E - Desmostylus
 
2.8
2.8
2.6
2.4
2.0
1.8
1.6
 
HOT



WARM

COOL
SECTION 6 - KNOW YOUR NICHE
A - View skeleton/model
C - Match shark teeth
G - Food web game
H - Interpreter
E - Map ancient bay
B - Walrus
F - ID sea birds
I - Compare horse prints
D - Compare/construct
 
2.7
2.6
2.3
2.2
2.1
2.0
2.0
1.9
1.6
 
HOT

WARM
SECTION 7 - GOING, GOING, GONE
I - Latet fossils
C - Who remains?
A - Face off
B - Bite
D - Map of extinctions
F - Models of sea change
G - Ice Age relicts
H - Packrat midden
E - Explore theories
 
2.9
2.6
2.6
2.5
2.5
2.5
2.2
2.0
1.7
 
HOT





WARM
SECTION RATINGS
Section 2
Section 7
Section 1
Section 3
Section 4
Section 5
Section 6
 
2.6
25
2.5
2.5
2.0
1.8
1.3
 
HOT



WARM

COOL

When people gave reasons for why they liked or did not like an activity, the answers usually fell into a category of "general positive" or "general negative." This means that people just liked what they saw and heard, it appealed to their interests, and/or they liked the general way it was presented. It is not unusual for people to be general in their reasons. Many museum visitors do not know exactly why they do or do not like something, they just know IF they do or do not like something.

When people gave more specific reasons for why they liked or did not like an activity, their responses fell into seven basic categories as illustrated in Table 3.

Table 3: Categories of responses to activities within exhibition sections

Positive Responses
Minds-on: challenging, engaging, imaginative, open-ended, compare & contrast
Hands-on: interactive, touch, manipulate
Ah-ha: new, full perspective "see," overview, abstract to concrete, important information, informative, instructional
Wow!: new, unique, different, latest, curious, exciting, immersion experience
Appeals to Many Ages/Types: a multi-age experience, something for everyone
Negative Responses
Unclear: confusing, too hard, too much work
Passive: not interactive, no engagement
Not Interesting: Won't hold attention, not new or unique (been there, done that)
Concern about User-Friendliness: Takes too long, a bottleneck, maintenance problem
Doesn't Appeal to Many Ages/Types: Only for kids, or only for adults, not age appropriate

Table 4 provides a listing of the percent and number of times people in the focus groups gave one or more of the specific reasons for liking or disliking a section activity described in Table 3. It is encouraging to note that the positive specific reasons were almost twice as frequent as the negative specific reasons. Clearly, most people liked most activities suggesting that the exhibition design team is on the right track. Following the table is an interpretation of the data by each section.

Table 4: Percentage of response categories by section

POSITIVE

SECTION
Minds On
Hands On
Ah Ha
Wow!
Many Ages
1
38%; 21
38%; 21
16%; 9
9%; 5
34%; 19
2
27%; 15
29%; 16
21%; 12
9%; 5
34%; 12
3
42%; 22
31%; 16
14%; 7
15%; 8
17%; 9
4
33%; 20
20%; 12
23%; 14
20%; 12
15%; 9
5
30%; 14
19%; 9
13%; 6
12%; 10
13%; 6
6
42%; 18
5%; 2
5%; 2
2%; 1
5%; 2
7
26%; 10
15%; 6
26%; 10
13%; 5
5%; 2

NEGATIVE

SECTION
Unclear
Passive
Not Interesting
Not User-friendly
Not Multi Age
Depends/Other Form
1
14%; 8
20%; 11
14%; 8
5%; 3
16%; 9
4%; 2
2
13%; 7
7%; 4
8%; 7
14%; 4
10%; 5
2%; 1
3
4%; 2
6%; 3
8%; 4
14%; 7
10%; 5
6%; 3
4
5%; 3
7%; 4
5%; 3
10%; 6
12%; 7
13%; 8
5
9%; 4
2%; 1
11%; 5
-
2%; 1
11%; 5
6
9%; 4
7%; 3
9%; 4
2%; 1
5%; 2
12%; 5
7
5%; 2
3%; 1
8%; 3
-
3%; 1
15%; 6

Regional Location

Many people responded very favorably to the idea of locating themselves - their house, recognizable landmarks (e.g., SDNHM) - throughout the exhibition. The concept of such long stretches of time and the changes that occurred is very abstract to many people. Any way that the exhibition designers can help visitors personalize prehistory was well-received.

Discovery Learning & Making the Abstract Concrete

People responded very positively to the idea of discovery. They were intrigued with the idea of taking on the role of a paleontologist. There is great interest in understanding how scientists know what they know. Some people suggested having some mysteries or puzzles that thread through the entire exhibition. For example, one staff member suggested putting an oyster in every section to see if people notice that oysters change very little.

Again, the issue of abstract to concrete applies here. People do not always understand how scientists take what seems to the visitor to be a small shred of a fossil and identify it as a certain animal or plant in a certain time period. Whenever people felt like an exhibition activity or idea would help them grasp an abstract concept, they were very enthusiastic. People were also very interested in personal stories, e.g., who found the fossils, how did they find them.

Computers

People in all of the groups were very cool towards the idea of computers in the exhibition. There were several reasons. Many people were concerned that computers were singular experiences and not conducive to groups of people. This would inevitably result in a bottleneck of traffic and frustration of waiting in lines to use the devices. In addition, many people found that high-tech devices in other museums were often broken, creating an additional level of frustration. Parents and educators acknowledge that children were often interested in using computers but they said that was not the reason they brought children to the museum. They want children to get an experience they cannot get at home or school. Some parents and educators also pointed out that children often focus on how the make the game work rather than on the content of the game. There was a great deal of interest across all the groups in having the information and games planned for computer interactives on a web site. There would need to be efficient ways to inform visitors that the exhibition experience could be extended to the Internet.

How much do people want to know?

There was not time in all of the groups to directly ask people how much they wanted to know when viewing an exhibition. Even so, there were comments across groups that provided some clues. These findings are supported by other research conducted by the Institute. First of all, museums are not the best place to teach new factual information. People are not in the frame of mind to memorize facts. They are in the frame of mind to be intrigued, stimulated, and awed. In addition, people want to have a good time at a museum.

That being said then, how much information do they want? It varies. Some people want to "feel" or experience an exhibition and soak up whatever they happen to soak up. These people tend to want highlights but not much text or specific information. They would like to know how to find other information if they should want it. Some people do want specific factual information and they want it at their fingertips, but this seems to be a smaller group of people. Many people responded positively to the availability of random access audio devices. While this hampers group experiences to a degree, it seemed to strike people that this was a good solution to providing more information for those who want it without filling the exhibition with text panels.

Families were interested in having access to questions they could ask their children as they go through the exhibition. They were also interested in having interpreters who could answer questions directly. Many parents and educators expressed an interest in take-away materials. Some parents said they would be willing to purchase some of the more sophisticated materials.

Lessons from special needs visitors

Special needs group provided some very helpful information about how people with visual, physical, and processing problems respond in a museum. Prior research by the Institute suggests that this is not just an issue for special needs visitors. Many people are over-stimulated by visual and aural cacophony of a large interactive exhibition. As a result visitors start skimming or skipping around exhibits and have trouble focusing. It is a difficult balance to strike, however. The museum needs to feel like a place where much is going on, where there are a variety of things to see, do, and learn for a variety of ages and learning styles. However, if it is so active and intense that people are overwhelmed, then, learning will suffer. Certainly an exhibition cannot be totally accessible to all visitors with every type of special need, but the conversation with seniors and those representing visitors with special needs provided a number of suggestions for making the experience better for all visitors.

Conclusions

Clearly, people are very interested in the content of the fossil exhibition as they are in the broader issues central to a natural history museum. The reasons or criteria that people gave for why they liked the content or delivery approach of an exhibit could be used to help the exhibition design team. As the team revisits the exhibition activities and components, the findings from this study could be reconfigured into a checklist.

These criteria can be used by the exhibition design team as a series of tests to determine whether or not to include certain components. As themes and activities are developed, submit each idea to a rigorous test using the criteria. Certainly, not all exhibition components will achieve ultimate success in all of the criteria checkpoints but the exhibition team can make conscious choices about what and what not to include and work to achieve a balance across sections. This could be developed in the form of a matrix such as the model below with themes and topics rated along six criteria. The rating scale below in Table 5 is one suggestion. The design team and Institute researchers could work together to develop a more sophisticated rating system. In any case, if used as a discussion guide, such a matrix would help the design team make informed decisions about components as well as identify areas that need work and/or further testing with potential visitors.

A possible rating scale might be the use of words such as, "high," "medium," and "low." High could mean that there is much of this criteria in use in an activity, medium could mean that there is some degree of that criteria evident in the activity but there are limits, and low would mean that there is very little of that criteria evident. Or the matrix could be filled out using descriptive explanations such as was done for the last column "User Friendly" in Table 5. In any case, this approach is designed to stimulate discussion and to keep the team focused.

Table 5: Prototype for Exhibition Design Criteria Matrix

 Minds-OnHands-OnAh Ha!Wow!Appeals to Many Ages/StylesUser-Friendly, Low Maintenance
Section 1      
Activity AMediumHighHighMediumHighMight be long cues
Activity BHighHighMediumLowLowQuestion Reliability of technology
Activity CMediumHighHighHighHighEasy to use and repair

This front-end study was designed to provide ongoing feedback to the exhibition design team. The team made full use of the iterative character of this study and made adjustments to the design throughout the course of the data collection and analysis. As strong trends emerged, the team was attentive and flexible to the findings and immediately incorporated them into their overall design. The following table indicates specific details of those changes and accurately describes the current conceptual design of the fossils exhibition and new working titles.

Table 6: Description of changes to exhibition outline as a result of front-end study

Exhibit Sections Prior to Focus GroupsExhibit Sections After Focus Groups
1: Unexpected Dinosaurs1: Unexpected Dinosaurs
2: Global Forces2: An End and a Beginning
3: Dynamic Spheres
3: Picture Past Life4: Compare/Contrast--Present/Past
4: Ecological Arms Race5: Race for Survival
5: Shifting Landscapes6: Movers and Shakers
6: Know Your Niche/Players on a Stage7: Paleo Puzzle Pieces
7: Going, Going, Gone8: Going, Going, Gone

1 Nancy Owens Renner, Exhibit Developer, conducted all ten of the focus group presentations.
2 Sandra Spaulding, Ed.D. designed and supervised the children's interviews. Research interns Susanna Didrickson and Brad Allison from the University of California San Diego also conducted interviews.

Back to Findings

SP Home | The Process | Background | Calendar | FAQs | Findings
SDNHM Home

© 2013 San Diego Natural History Museum