A-STOP
Wise as Serpents Curriculum
EVALUATION REPORT
RESEARCH RESULTS
Student Knowledge and Values Questionnaire
Methods
Five of the 34 churches that reported using the curriculum agreed to assist in the evaluation by asking student participants to complete a questionnaire addressing knowledge and faith issues. Sixty-four students completed this questionnaire after the course. It is unknown what percentage of students in these five churches filled out the questionnaire. The faith and knowledge questionnaire is shown in Appendix 1.
Survey Results
Student Faith
Students were asked to indicate how strongly they agree or disagree with statements about their own faith. Table 1 displays these responses.
Students agreed overwhelmingly that they had confidence in their own worth. Almost all young people agreed that they are valued children of God, and that God loves them even when they feel they are not worthy. All but five youth agreed that they would deserve help if they had been abused or were thinking of running away.
Many students also draw upon their Christian faith to help them make decisions. Ninety-one percent of students agreed that prayer is important for wisdom and guidance, and 84 percent said their faith in God influences their choices. All students agreed that they do not want to harm others and themselves by being a customer of commercial sexual exploitation.
A majority of students said they are aware of how adults might try to recruit young people into prostitution (82%), and they know who to go to for help and support with concerns in their life (79%).
One church asked students to complete this questionnaire both before and after the course. Because only ten students participated, caution must be used when attaching significance to these findings. However, it is interesting to note that there was a noticeable difference between the pre- and post- test scores on one of the above statements. Before the course, four students agreed, and six students strongly agreed that they were aware of how some adults might try to recruit young people into prostitution. After the course all ten students strongly agreed with this statement.
Student Knowledge
Students showed that they had basic knowledge about prostitution after participating in courses involving the Wise as Serpents curriculum. A large majority of students correctly answered questions about the sex industry. The role of strip clubs to prostitution recruitment and the relationship between sexual abuse and prostitution were the only items that less than 90 percent of students were able to correctly identify. Table 2 shows the questions and students' responses. Over 90 percent of students answered five of the seven questions correctly.
Of the 10 students who took this test before and after the course, there was a noticeable difference in correct scores on one knowledge question. Before the course half of students (correctly) agreed that strip clubs can be recruitment centers for teen prostitution. After the course all students agreed with this statement.
Whereas these 10 students answered the knowledge questions correctly 100 percent of the time in the post-test, they answered them correctly 90 percent of the time in the pre-test. This may indicate that a more difficult test would be a better indicator of knowledge gained.
Student Evaluation Questionnaire
Methods
Eight churches distributed evaluation questionnaires to students who had participated in some or all of the Wise as Serpents curriculum. Ninety-three students completed this questionnaire, which is given in Appendix 2.
Survey Results
Perceived Benefit of Curriculum
The questionnaire asked the open-ended question, "What did you learn from this course?". Figure 1 shows the most common responses.
Figure 1. Frequency of Student Reponses to the Open-ended Question "What Did You Learn from this Course?"
(N=96)
The comment given most often was that prostitution hurts the people involved. Below are some of these responses:
- That prostitution hurts everyone in some way.
- How scary prostitution is & how to protect yourself from it.
- That prostitution is a very horrible thing to get into and is not a good way to live your life.
- Not to get into prostitution, because it can hurt you and your family.
- That you hurt a lot of people with prostitution.
The second most common responses were techniques on how to avoid becoming a prostitute.
- To be careful when a person wants to meet you in a place by yourself.
- That not to meet someone if you don't know them.
- How pimps tricked people into prostitution.
- Things like pornography can lead to prostitution. Most girls are tricked into prostitution.
- People can be tricked into being in prostitution in many ways.
- (One teacher summarized student responses and wrote the following:) [Students] mentioned the warning signs and not to be scared to tell or ask for help.
The third most common thing students reported learning was that God loves them, no matter what. Anne Rasmusson found in a literature review of prostitution that one of the techniques pimps use to get young girls to be prostitutes, is to encourage the girl's negative self-image (see Appendix 10). Letting young people know that they are always loved by God helps to protect them from exploitation. The following are some examples of students expressing their inherent worth by writing that God loves them, regardless of what they do.
- That no matter what, God is with you and will help you through bad times.
- God wants you to save yourself for the right one and that no matter what, God loves you.
- I learned that no matter what you do wrong, God is there for you.
- To trust in God in everything I do and try to stay on the right path.
- That God loves you.
- The grace of God will always, always be with you.
- Nothing, really, except that no matter who you are and what you do, God can turn your life around.
- That prostitution is bad and I think it should be dealt with seriously. And that God will be there to help me.
There were also a number of comments noting that anyone can get tricked into prostitution.
There were a few comments about why one should not be a customer. One student responded that s/he learned "Not to be a pimp".
After students reported what they learned from the course, they were asked whether or not that knowledge will be helpful to them. Of the 88 students, 83 percent of students said "Yes", 3 percent said "Maybe", and 14 percent said "No".
In responding to the questionnaire, students' comments were very brief and their exact meaning was often unclear. For the actual comments, refer to Appendix 3. Students comments have been interpreted and summarized for the purposes of this report.
When asked how the course will be helpful to them, almost one third said they will know what to do in the future, possibly meaning in a situation of danger. Another fifth of students said they know how to avoid "it", most likely referring to situations where they could be at risk of becoming involved in prostitution. Other students commented that the course has informed them about the realities of prostitution, and they will not get involved in it. Still others said it has been helpful to realize that God will always be there to help them.
Figure 2. Frequency of Student Responses to the Open-ended Question "Do you think that what you learned will be helpful to you? If yes, how will it be helpful?"
(N=71)
Of the 12 students who did not think that what they learned will be helpful, eight said that they were already aware of the information, and four said they will not use it because they will never get involved in prostitution.
Parent Involvement
Another important goal of the curriculum is to encourage youth to talk with their parents about the dangers of people trying to trap them into prostitution. Included in every curriculum session were take-home worksheets entitled "Table Talk" that are designed to facilitate parent and child discussions. Over half (53%) of young people said they used "Table Talk" with a parent or other significant adult. Most of these students reported doing this only once. Table 3 shows how often they talked with their parents about the topics.
Satisfaction with Wise as Serpents Materials
Over half of the students who viewed the video rated it as a 4 or 5 (the highest ratings) on a 5-point scale. The stories were rated almost as high, with 47 percent of students giving them a rating of 4 or 5. Thirty-eight percent of students gave the activities a rating of 4 or 5, and 37 percent gave this rating to the overall course. Figure 3 displays these findings.
Figure 3. Student Satisfaction with Wise as Serpents Materials
(Video: N=60, Stories, Activities, Overall: N=91)
Researchers divided the churches into three groups. The first group used the video and a majority of students had done the "Table Talk" with parents (21 respondents were in these two classes). Three classes used the video while less than half of students had done the "Table Talk" (39 respondents). One church did neither the "Table Talk," nor did they show the video (31 respondents).
The following three figures display variance in satisfaction among students who received these different materials. The differences are statistically significant at p < 0.01. This means that there is less than a 1 in 100 chance that the differences between these three groups shown on the following pages are due to chance. Students in classes which used the video and in which most students did "Table Talk" were most satisfied with each of the materials and the overall quality of the course. The classes in which the video was shown but few students did "Table Talk" were less satisfied. The class which used neither the video nor "Table Talk" received the lowest scores from students.
It must be noted that in the third group which used neither video nor "Table Talk," the teachers used only sessions 1-4, and they expressed dissatisfaction that there was not enough material to continue with more sessions. It is not possible to know whether this last group's scores were a result of the students not seeing the video, not completing the curriculum, a result of the teachers' dissatisfaction with the course, or a combination of these factors.
In the graphs that follow ratings were based on a five-point scale where 1 represents "Really bad!" and 5 represents "Great!". In these figures "Great!" represents the two highest ratings, "Middle" represents the middle rating, and "Really Bad!" represents the lowest two ratings.
Figure 4. Student Ratings of Overall Course by Which Materials Used
(N=91)
Figure 5. Student Ratings of Course Activities by Which Materials Class Used
(N=91)
Figure 6. Student Ratings of Stories by Which Materials Class Used
(N=91)
What Students Liked Best about Wise as Serpents
The evaluation questionnaire asked students what they liked best about the course. Their responses varied somewhat by which materials their class had used. Students in classes where they viewed the video cited two aspects of the course in general and the video in particular which really had an impact on them. One aspect that seems to have had an impact on students is the people who told their stories. Students were most likely referring to the people telling their stories in the video. Some examples of comments which seem to refer to Heidi and Mark (in the video) are the following:
- I liked hearing the pimp and prostitute talk about how hard it was for them.
- I liked being able to hear of other people's experiences.
- Good information. Good people.
- I liked it because I thought those people were brave for doing that.
- How they changed their lives.
- They used real people that were involved in prostitution.
The second aspect that seems to have had a lot of impact on students who viewed the video in their course, is that the situations they heard about were not made up, but were real. Below are some of these responses which refer to using "real-life" people and their stories:
- You had real people instead of people acting it out so you really heard how it made you feel and what goes on.
- I liked watching the video. I like seeing real stories being told.
- Got the message across. One-on-one with the person; not actors or actress.
- The stories -- they were real examples and I learned a lot from them.
- True life stories.
- I liked the movie because it showed a real life example.
- The way it dealt with real life stories and not just a made up one.
- All people can get out of it if they want to. People were actually in the situation.
The video was very popular; it was mentioned as something students liked best about the curriculum in 27 of the 67 comments from students who had viewed it.
Conversely, in the one class which did not view the video, no students mentioned the course as being "real" to them. In fact one student commented that the course was not very realistic (a comment which was never given in any of the classes which had viewed the video). Again, this difference may be due to the fact that this class also didn't go through the entire curriculum, and their teacher was dissatisfied with the materials. However, the extent to which students felt that prostitution is something real that may have an effect on them may be attributed in large part to viewing the video.
Some comments indicated that students may not have enjoyed the movie very much and that it may have made them uncomfortable. However, in all such cases the students said they learned something from it. For example, one teacher summarized what the students liked best about the course as, "Video was scary, but seemed like it was told for a reason. Admired Heidi and Mark for telling their story." Here is a similar comment from a student:
- I didn't think the movie was that bad; yeah, it was a little boring, but I would rather find out about it now than get stuck in a situation later and not know what to do.
For a few students, the fact that people have been able to get out of prostitution made an impact on them. Below are some responses to what they liked best about the course.
- All people can get out of it if they want to. People were actually in the situation.
- How they changed their lives.
Student and Parent phone Interview
In order to maintain the anonymity of the participants in the phone interviews, we are not including this section of the report. Please refer to the Executive Summary for a summary of the results.
Teacher Curriculum Questionnaire
Methods
Pastors, Sunday School teachers, Confirmation teachers and youth workers used the Wise as Serpents curriculum in their religious education classes. These people are referred to as teachers in this report, although many of them are not licensed teachers. Twenty-one teachers filled out the self-administered questionnaires, three teachers responded to identical questions over the phone with Amy Hartman, and one teacher talked about her/his experience in general, and Amy Hartman matched the responses with similar questions. The Teacher Curriculum Questionnaire is shown in Appendix 6.
Survey Results
Teachers' responses to multiple-choice questions are given below in Table 4.
Almost two-thirds of teachers gave the highest rating to the curriculum's simplicity to use. Half of teachers gave the highest rating to the curriculum being well organized. At least 92 percent of teachers gave the curriculum's layout, organization, and usefulness the first or second highest rating on the five-point scale.
As shown in Table 5, almost all teachers gave the video, written materials, and curriculum as a whole the highest possible or next highest possible rating. The video received the highest ratings. Nine of 11 responses to open-ended questions were positive about the video.
Twenty-three of 25 teachers said they would recommend the curriculum to a colleague.
In the open-ended questions, teachers rarely mentioned either of the things which seemed most important to the students: the two people on the video, or that the situations and events seemed "real" to them. However, there was agreement among teachers in why they would recommend the curriculum to a colleague. Half of the teachers said they would recommend the curriculum because discussing the topic with youth was necessary. Here are some of these responses to why they would recommend this curriculum to a colleague:
- Needs to be talked about
- Awareness and talking about this issue are important.
- It is necessary to help our youth identify persons who might manipulate them.
- I think if is a subject that needs to be discussed.
- So important!! It can never be said enough that this message needs to be put across to everyone.
- People need awareness that this is an issue for kids. Kids are going off to college unprepared. It's a very positive thing, and we're going to continue with it.
- It's necessary for kids to know about it and how the view of sexuality has been skewed by our culture.
- It's the easiest, most complete resource that exists on this topic and doesn't shy away from an important issue.
- Because it is a high quality, easy to use curriculum dealing with serious subjects that many people are uncomfortable discussing.
There were a few other trends in teachers' open-ended responses. When asked to comment about the written materials, three teachers mentioned that they liked the choices available, and that they could choose which exercises seemed most appropriate. When asked what students seemed to like about the course, about equal numbers of teachers said the exercises (4), the discussion (4), and worksheets (5). Nine teachers mentioned the video. This coincides with the students' responses in their multiple-choice selection and comments that the video made the topic "real" to them. Three teachers mentioned that the students seemed to like the straight forward format, as shown in the comments below:
- When we were successful in getting them to talk, they really seemed to enjoy the openness of the discussions.
- The up forward verbiage of the video and the honesty was truly an asset.
When asked what students seemed to dislike, two teachers said the students were uncomfortable discussing the topic, although they both said this was understandable due to the ages of the youth and/or the group setting. Three teachers said they did not have time to cover all the material they wanted to in the six sessions, indicating that they should have allocated more time for the course. Two teachers simply said there was too much material.
Two teachers suggested that more "real-life" people be added, one teacher specifying a person from a rural area since their students still felt prostitution was predominately a city problem.
When asked about the strengths of the curriculum, five teachers mentioned the straight forward discussion format, four said the curriculum was easy to use, and two said the course was applicable to students' lives outside of class.
The questionnaire also asked teachers if they informed parents before using the curriculum. Twenty-two of 25 did so, with eight receiving a positive response, eight receiving no response, one finding parents to be concerned, and five not specifying the response they received from parents. Teachers were also asked if they informed anyone else of their intention to use the curriculum. Below are the groups that the churches informed:
- Eleven churches informed the Church Council.
- Five informed the Education Committee/Board of Education.
- Four churches informed parishioners via the church bulletin.
- Sermons were held on prostitution in three churches.
- The Youth Board was informed in one church, and the Director of Youth informed in another church.
Three teachers mentioned that the curriculum offered skills which were useful to students and that they would know what to do if approached. One of these teachers said that the worksheets were best at helping students with these skills. However, another teacher would have liked to have seen more skill development, for example in how to say "no" and how to handle angry people.
Table 6 gives an overview of teachers' comments.
Teacher Training Questionnaire
Methods
Sixty-nine teachers from 54 churches attended one of ten training workshops designed to help them use the Wise as Serpents curriculum. After the training 59 teachers completed a self-administered questionnaire. This questionnaire is shown in Appendix 8.
Survey Results
Seventy-two percent of teachers who attended the Wise as Serpents Training Workshop rated the Training as Excellent.
| 1: Poor |
2 | 3 | 4 | 5: Excellent |
Mean (Average) | |
|---|---|---|---|---|---|---|
| How would you rate this training as a whole? | - | - | - | 28% (16) |
72% (41) |
4.7 |
Teachers reported that workshop instructors were well-prepared, responded effectively to questions, and that the training was well-organized and provided useful tools and knowledge. On each of these measures, the mean (average) was over 6.0 on a seven-point scale. Results are given in Table 8.
* Endpoints on this five point scale are labeled. Middle ratings have no labels.
Table 8 shows teacher responses to benefits they took away from the Wise as Serpents training. Most teachers strongly agree that they understand the value of the curriculum for youth in their church, and they have a stronger commitment to preventing youth from being exploited or exploiting others. The training appears to be very successful at motivating teachers. Almost all teachers also agree, although not as strongly, that the course will help them do a better job of leading the course, including using the video and written materials, and that they clearly understand commercial sexual exploitation and how it intersects with the lives of their students. Teacher ratings of their ability to teach the curriculum are very high, although not as high as their motivation to teach the curriculum.
Teachers also described what they found most helpful about the training. There was variation in this depending on whether or not Heidi, one of the main characters in the video, was present at each training. The most common comments are given in Figure 7. For a full display of all comments refer to Appendix 9.
Figure 7. Frequencies of Teacher Responses to the Open-ended Question "What Did You Find Most Helpful About this Training?"*
* Many responses encompassed more than one comment, and comments mentioned only once are not included. Twenty teachers evaluated training workshops where Heidi participated, and 39 teachers evaluated training workshops where Heidi did not participate.
Three teachers mentioned the mock class as the most helpful aspect of the training, and the same number mentioned the mock class on the next question of what they found least helpful. Two teachers felt that too much time was spent on teachers' expectations, and two teachers felt that the workshop time or material could have been condensed. Comments which were made by only one teacher can be viewed in Appendix 9.
The last question on the training questionnaire asked for other comments or suggestions for improvement. Seventeen of the respondents said "Thank you" to the A-STOP staff for their work.
Several teachers mentioned that it would have been helpful to have had more discussion time on things such as the session layout, overview of the program, and its use and materials, and going through the course lessons. One teacher said the discussions were a very effective preparation for leading the programs.
Several teachers mentioned that A-STOP may want to work on how teachers can involve parents. One teacher suggested an "encapsulated parent overview video" would help to present this emotionally charged material. Another pointed out that the effectiveness of the program depends on the knowledge and understanding of parents.
Another concern teachers raised was how little many teachers know about these topics. One teacher suggested presenting an overview of the sex industry, and wanted to know how else teachers can help. Yet another teacher said that youth may know more about these subjects than teachers, so this information is important for teachers to be able to "discuss these issues boldly".
![[Link to Adults Saving Kids home page]](ASKLogoSmall.gif "Link to Adults Saving Kids home page")