An assessment of the effectiveness of educational programs in

relation to school curriculum needs - A Research Fellowship in Museum Practice

Segametsi Radise

Botswana National Museum

Smithsonian Institution Fellow in Museum Practice 2005

According to Jem Fraser in his summary of the Visitors' Studies Conference of 1999, “Museums are educational institutions and yet have the difficulty in demonstrating educational effectiveness”. As a museum educator, this statement came as a challenge to reflect on my practice and that of my fellow colleagues in our institution. My interest grew in finding out what is happening in other museums internationally.

The assessment was an attempt to raise museum educators' interest in exploring the effectiveness of museum programs in relation to school curriculum needs. The traditional form of assessments can imply judgement at the end of a task. This assessment was a process of collecting, analyzing and reporting data. The findings will be useful to both the educator and the teacher in monitoring and adjusting the program to meet specific standards of learning as well as the museum's educational goals.

There are two important experiences which ultimately prompted this research. In 2001, the education division of the Botswana National Museum , where I work as a museum educator, embarked on an impact assessment project for two educational programs. The first program was our mobile outreach program, a program which is now more than twenty-three years old) and the second was our teacher workshops' series. The assessments gave meaningful feedback to the museum on the benefits and shortcomings of our programs. One of the findings was that teachers want to be equal partners in concept development. The other important finding was that they did not conduct post visit lessons. They treat our museum programs as enrichment and not integral to the curriculum. The museum education department suspended teacher workshops and through collaboration with the ministry of education, they are going to be revised.

In 2003, I participated in a course called “Education and Exhibitions” as a tutorial fellow in the Programme for Museum Development in Africa 's Post Graduate Diploma in the Care and Management of Heritage and Museums Collections . Frequently, questions arose concerning the effectiveness of exhibitions designed by people who are not professional researchers in particular disciplines. The inference here was towards museum educators. Students in this class were given an assignment to conduct an evaluation of a travelling educational exhibition called Urban Origins of Eastern and Southern Africa at Jumba La Mtwana Site Museum. All the students evaluated the exhibition on the amount of knowledge the exhibition transmitted, the validity and reliability of the information, and its presentation, that is, its design and fabrication. Most notably, though, what the target audience learned from the exhibition was not explored.

Museum education came about because of the changing role of the museum in the society; the public became more aware of their needs and demanded more from museums. The ideology behind the development of museum education is that of bringing new knowledge and experience for the growth and development of the existing knowledge base of the visitor. Museum education can therefore become an integral part of the school curriculum as well as being a powerful resource for educating the public.

In his article “The constructivist museum” George E. Hein (in The Educational Role of the Museum , 1994) said that “an educational role consists of two major components: a theory of knowledge and a theory of learning. In order to consider how a museum is organized to facilitate learning, we need to address both what is to be learned and how it is to be learned”. The educational role of the museum is complex. There is a link between museum pedagogy and school curriculum.

Learning theories have influenced the conceptualization and development of school programs. Museum practitioners and teachers should collaborate and support one another to achieve their common goal of educating learners. Brett Dunlop in the article “Education and Learning”(in Museum Education and Public Programs, 1998) explains that, “teachers have the expertise in helping children to learn. Museum educators have transferred teaching expertise to the museum setting; usually without specific required training…they learn by participating in the museum community of learners”. Learning from objects has been incorporated and emphasized in the school curriculum. It is an efficient and effective way of enabling schools to meet educational requirements. Learning in museums is object-based inquiry , when it focuses on empowering students to discover and get insight about the processes taking place in the world and how they work. It is from this methodology, that museums can afford their learners the opportunity to bring in their psychological, physical and social interactions to the museum experience.

Why then is demonstrating educational effectiveness so difficult?

Falk and Dierking (1992) in their book, The Museum Experience aptly provide the explanation for the cause of the difficulty. They explain why many museum professionals doubt that a museum visit can be a learning experience. Available data show that few adults, and worst of all for children, demonstrate immediate recall of facts and concepts after visits to the museum. Many evaluation studies conducted on school programs have been limited to the transmission of knowledge, facts, concepts and values. They imply that if learning had taken place, the same should be recalled on demand. However learning as a process is a far more complex process than the simple retention of facts. Learning needs to be understood as the slow consolidation and incremental growth of existing ideas and information.

The Smithsonian Institution's National Museum of Natural History's Naturalist Center was selected for the study. The Naturalist Center is a facility that invites learners to “walk into the learning environment of the professional scientist, artist, or historian” (Smithsonian Institution, National Museum of Natural History Website) . This makes it, a very unique educational facility in the world!

To fulfill this goal, the Naturalist Center established a partnership with Loudoun County Public Schools in creating a Teacher Naturalist position. This is a qualified teacher, who is assigned from the county system to the Center for a period of one year. As the current teacher naturalist explains it, it is “an educator liaison at the museum”. The Teacher Naturalist is an important contact person between the Center and teachers in the county and region. As a trained teacher, the Teacher Naturalist has an understanding of how curriculum maps and the state's Standards of Learning work. The Teacher Naturalist therefore morphs into a curriculum developer for the Center by linking the Center's resources with the Standards of Learning.

For assessment, two Naturalist Center programs called Classification II and Unknowns were selected for assessment. Classification II is intended for middle school seventh grade life science LS.5 which states that “the student will investigate and understand how organisms can be classified”. The goals of the program are: “to determine characteristics of organisms, to categorize organisms by taxonomic levels and to classify organisms with a dichotomous key” (Classification II, 1985). It is out of this that a student centered lesson plan is drawn. Unknowns is a problem solving scientific methods program which helps visitors learn about some natural history topics (Unknowns, 1985). On a day's visit which usually lasts two and a half hours, students get engaged in both programs.

The Teacher Naturalist leads the programs and with the assistance of docents who are licensed teachers. This is one of the interesting aspects of the programs. It is done to bring the strength of trained teachers' expertise into program delivery. The program is delivered using objects from the center's collection and the activity sheets. The teacher draws observations and evidence for their conclusions during the lesson. The key questions for the assessing the program were:

The sample for the study was two hundred and fifty students and fifty teachers. Data was collected using two sets of self-administered open-ended questionnaires to gather information about the program as presented during the visit. The questionnaires were taken away to be filled out and returned through the “pony mail service ” that operates in Loudoun County schools. Open-ended questionnaires were used to make as exhaustive and inclusive a study to understand the effectiveness of the program. The responses were collated to discover patterns, which were channeled into categories in the compilation process.

FINDINGS FROM STUDENTS' RESPONSES

Programs that you participated in

All the students, that is 100% responded by writing down Classification II and Unknowns.

One thing I learned from this visit

85% responded with a similar definition of Classification as “how to classify things” or “how things are classified”. The remaining 15% responded with different definitions of classification.

I still want to know

45% wanted to know more about the differences in insects. 35% wanted to know how the specimens were conserved. 20% wanted to know about the different animal and fossil specimens that they saw at the Naturalist Center .

What I liked most about this visit was

All students, that is 100% responded by saying “being able to handle objects” or “being able to touch objects”. Of this 100%, 55% also responded that they “chatting” “talking” and “discussing” with their “friends”.

If I did this activity again

85% responded by saying they would “like to explore more objects”. 10% responded by saying they would “like to do more drawing” and 5% responded by saying they would “like to look around”.

Other things I could do at the Naturalist Center

88% responded by saying they could “do science projects”, 10% responded by saying they could “draw” and 2% responded by saying they could “look around”.

How did this visit change the way you view museums

75% responded by saying “museums can be fun when they are hands-on”, 20% responded by saying museums are “boring” and the Naturalist Center is different because “you can touch things”, “you don't walk around and listen”. 5% responded by saying “museums are boring because they are visual”, one student responded by saying “I am tired of cheesy tours”.

FINDINGS FROM TEACHERS' RESPONSES

What educational requirements were covered by this visit

All of the teachers, that is 100% wrote down the specific SOLs (Standards of Learning) that they were covering through the program.

What part of the trip went especially well and why

88% responded by saying “using Naturalist Center specimens to cover classroom work”, “using the Center to prepare for SOLs” and “using the Center to achieve SOLs”. 12% responded by saying “students being able to make a connection between class work and reality by using specimens at the Center”.

What changes would you make and why

10% responded by saying “increase in exploration time”, “increase exploration time and only concentrate on one part, i.e. classification because it is meant for SOLs”.

Do you plan to use the Center for personal use

55% responded by saying “yes, with my family” and “yes”.

  Do you plan to use the Center for future visits with students? If not, why not?

All teachers, that is 100% responded by saying “yes” and within the group, and 90% responded by saying they are using the Center “only because the program is an integral part of the curriculum” and 10% indicated that “the program is an integral part of the SOLs”.

In what way, if any, did this trip change your views of hands-on science

74% responded by saying “it increases students' understanding of science concepts”, 16% responded by saying “hands-on science gives more in-depth information” and 10% responded by saying “it peaks students' understanding of science”.

What continuing role do you see the Naturalist Center playing in regard to education

90% responded by saying it is “a hands-on interactive classroom” and 10% responded by saying it is “great resource for teaching science”.

How did this visit change the way you view museums

20% responded by saying “museums should be hands-on to be more fun”, 50% responded by saying “museums should be hands-on to be more fun and educational”.

Conclusion

Concerning effectiveness, the findings from teachers' responses show a great achievement of meeting school curriculum needs. This result seems to be attributed to the way the program was conceptualized and implemented. The leadership and guidance of the Teacher Naturalist has been a great strength for the process because of understanding the Standards of Learning and knowledge of the available resources at the Naturalist Center . From the findings of the study, teachers use the program as a hands-on interactive classroom to prepare their students for examinations and to achieve the required Standards of Learning. There is a direct link between the program and the curriculum.

The attitude level of the students about the program is highly positive. It is evident from this study that students prefer museum programs which are hands-on interactive.

It also seems fair to suggest that the conceptual understanding of students has been boosted. Based on the findings of this study, students were able to provide definitions of classification as well as wanting to know more about the distinguishing characteristics of insects which is an important aspect of classification.

Teachers seem to be greatly interested in engaging their students in programs that will assist students achieve good results in their examinations. Time taken out of the classroom should be spent on programs focusing on achieving the Standards of Learning. Based on the findings of this study, it was very interesting to note that, teachers use the program “only because it is an integral part of the curriculum”.

IMPLICATIONS FOR THE BOTSWANA NATIONAL MUSEUM

All school programs (those that deal with the curriculum) must be developed with the input of teachers with a thorough needs assessment conducted to make them effective. When teachers' knowledge and understanding of the curriculum is utilized in program development and implementation, like it is the case at The Naturalist Center, the program can be very effective. Based on the results of the 2002 Impact Assessment of Museum Outreach Programs, it is evident that teachers want to be consulted in program development.

Teachers are interested in the planning and development of programs that take their class-time. It is fair to suggest that, similarly, teachers in Botswana want school programs that will directly assist their students in achieving good results. Based on the results of the 2002 Impact Assessment of Museum Outreach Programs, teachers feel that the mobile museum program should meet the needs of the curriculum.

Given the large amount of resources (education division's annual budget) that go into the mobile museum program, the education division should revise the mobile outreach program and decide whether it remains complementary or if it can become an integral part of the curriculum. From the 2002 Impact Assessment of Museum Outreach Programs, teacher felt that the program should have students work in smaller groups. It is evident that, similar to the Naturalist Center programs, a smaller group, for example, Standard six could be targeted with a program integral to the curriculum. The program can also be museum-based to benefit Gaborone area schools or those who are able to make it to the museum.

It is also fair to suggest that complementary programs should not interfere with class-time and rather the education division should develop materials like teaching kits, videos, loan packages and posters to enrich the schools' curriculum. In the 2002 Impact Assessment of Museum Outreach Programs, teachers indicated that these complementary materials are useful for enrichment purposes.

 

REFERENCES CITED

Dunlop, B. Education and Learning. In Museum Education and Public Programs. Newsletter of the Education Group Museums Australia Inc, vol 1,1998.

Falk, J.H. and Dierking, L.D. Creating the Museum Experience. In The Museum Experience , Whalesback Books, Washington , D.C. 1992.

Fraser, J. Visitor Studies Conference Chicago 1999. In Journal of Education in Museums , vol.20, 1999.

Hein,G.E. The constructivist museum. In The Educational Role of the Museum , second edition, Eilean Hooper-Greenhill, Routledge, London ,1994.


NATURALIST CENTER TEACHER VISIT EVALUATION FORM

 

School:_____________________________________________________________________

Grade Level/Age: _______________________ Date of Visit: ___________________________

Length of Visit: _________________________ # of Students:___________________________

# of Teachers: _________________________ # of Chaperones:_________________________

Program (s): __________________________________________________________________

What educational requirements were covered by this visit?

What part of the trip went especially well? Why?

Do you plan to use the Center for personal use?

Do you plan to use the Center for future visits with students? Why?

In what way, if any, did this trip change your vies on hands-on science?

What continuing role do you see The Naturalist Center playing in regard to education?

How did this visit change the way you view museums?

Thank you for your cooperation.


NATURALIST CENTER STUDENT VISIT EVALUATION FORM

School: ______________________________________________________________________

Grade Level/Age: _______________________ Date of Visit: ____________________________

Programs that you participated in: ____________________________________________________________________________

One thing I learned from this visit is___________________________________________________________________________

I still want to know________________________________________________________________________

What I like most about this visit was_________________________________________________________________________

If I did this activity again, I would like to___________________________________________________________________________

Other things I could do at The Naturalist Center are ____________________________________________________________________________

How did this visit change the way you view museums?____________________________________________________________________

Thank you for your cooperation.