Streams On Line Evaluation
by Lucia Wright
July, 1999
lucia@lrw.net
Streams OnLine (SOL) was conceived as a tool for improving student literacy. Initiated by Waterways and realized by Randy Wright, it is a computer program that uses a web based environment to encourage students to read and write and comment on each other's work. Streams OnLine was designed with the intent to enable collaboration between classes and schools and to allow for communication between remote sites.
Students submit original writings, commentary, and multimedia materials such a images, sound and video. Teachers approve student work before it is visible to participants in a collaboration. Work is not visible to anyone outside a defined collaboration.
The project began in late December 1998 and the tool was introduced into the computer rooms of two highschools by January 1999 for Alpha testing. By March 9, 1999 Streams Online was made available for Beta testing in classroom collaborations. Beta testing is a period during which selected participants try out a new piece of software with the purpose of getting feedback on what does and does not work. Aims and goals were outlined in terms of what we hoped would happen when SOL was introduced into the classroom setting.
Streams Online was introduced to two highschools in the New York City Area. Frederick Douglass Literacy Center (FDLC) is a NYC Board of Education site in Brooklyn that serves 17-21 year olds testing below 6.0 reading levels. Island Academy (IA) is a NYC Board of Education site on Rikers Island that serves incarcerated youth. The primary contacts were the computer instructors at each school: Madeline Slovenz at FDLC and Willie Bennett at IA. The Beta testing lasted until the end of the school year in mid June, 1999. During this period some teachers incorporated the tool into their classroom activities and individual students also came to use the tool of their own initiative. What follows is a report based on observation in the classroom, interviews with instructors, discussion on an electronic discussion list for SOL developers, instructors and administrators, and written material submitted to Streams OnLine by the students.
What did we hope will happen?
A Fun and Useful Tool
It was hoped that students would enjoy using Streams OnLine while improving their literacy and acquire skills useful in today's world.
Reading and Writing
It was hoped that Streams OnLine would encourage students to read and write and work on their poems, essays, written material, thereby exercising editing skills and learning to focus on what they have written.
Collaboration
Streams OnLine was designed with the intent to encourage teacher involvement and peer exchange. It was hoped that students would contribute and comment on each others' work. As a result, they would get experience in reading and writing and exercise critical thinking.
The program is designed to allow students and teachers to exchange email which provides another venue for a written exchange between students and teachers.
Streams OnLine was also created to allow for collaboration between schools that normally would not be able to collaborate. The hope was that this exchange would improve the literacy rate among the students.
The technologies employed for SOL were chosen and designed with the goal of allowing it to work in labs or classrooms equipped with older desktop computers as well as to replicate between remote sites. In particular, one of the goals was to allow a school without fulltime internet access to participate in a SOL collaboration.
Critical thinking
It was hoped that the tool would encourage students to read and comment upon each others work in a substantive manner. As a result, they would get experience in reading and writing and exercise critical thinking. The goal was for students to be encouraged to edit and improve their writing in response to the comments they receive. In addition, the students can categorize their works by theme which also encourages critical thinking.
Revision control
Streams OnLine was created with a revision control facility. This allows each student to revise his or her writing and retains a revision history of changes made to a document. The goal was to track the progress of a student's process and development and give both teacher and student easy access to previous versions of a written work. It was hoped that teachers would be able to spot difficulties and patterns more readily and have an easier time helping their students achieve literacy. It was also hoped that a revision history would provide data in evaluating the effect of SOL.
Technical skills
Streams OnLine provides an opportunity for students to exercise computer skills, writing in a web based environment. It was hoped that students would learn how to use a web browser, write HTML code, transfer files over a network, write email, and in general become familiar with using a computer.
Electronic Portfolios
It was hoped that being located on a computer, Streams OnLine would provide a solution to the problem often faced by teachers of where to store paper portfolios. Here, student work is entered into a computer, stored and retrieved at the terminal, easily shared in a classroom and between schools. Everyone within a collaboration, who has access to a computer terminal networked to a SOL server, can retrieve and view the work of others in the collaboration. There is always the option to print out their work if necessary.
Publication
Streams OnLine was designed with the goal of producing a product that is viewable to the world, such as an online publication called "Streams OnLine." Students and teachers would nominate some of their work for consideration by the editors who would be able to view the content remotely on a password protected website.
This evaluation is a report on the Beta Testing period of Streams Online. It is important to stress that during this period there was no formal launch of the tool. Teachers and students were informed that they were part of a beta test phase and they had the opportunity to explore this tool if they wished to do so. The developers of Streams OnLine had an opportunity to test the software, see if there was interest, and make preliminary observations regarding its impact on student literacy. While many software changes occurred during the test period, what follows is a report of observations in terms of the goals outlined above. So, How Did We Do?
Frederick Douglass Literacy Center is a NYC Board of Education site that serves 17-21 year olds testing below 6.0 reading levels. Madeline Slovenz, the computer instructor, first introduced the tool to individual students who enjoyed hanging out in the computer lab.
When SOL was ready for Beta testing in March 1999, Ms. Slovenz introduced the tool, along with a manual, to four teachers for use in the classroom. She explains that she chose to introduce SOL to classes at different reading levels because "I wanted to see how it would fly at the different levels." Mary-Ann Cornell's class was at level 1, with reading levels between 1.0 and 1.9 (mid- semester, Anasuya Isaacs took over this class). Sal Canale's class was at level 2, with reading levels between 2.0 and 3.9 . Gail Tuch's class was at level 3, between 4.0 - 5.9. Dolores Russo's class was the pre-GED class, with reading levels between 6.0 - 7.9. Teachers bring classes in to use the computer room once a week at FDLC.
All of the classrooms participated in the same collaboration which means that all of the participants could read and comment on each other's work. The teachers used the tool very differently over the course of the semester. Mr. Canale used SOL almost weekly with his class and then used it on his own once or twice a week in addition, to approve, view and edit student work. Ms. Isaacs initiated a creative project on SOL but did not follow through with it. Ms. Tuch used SOL once or twice early in the semester but then shifted to creating chapbooks on a word processor. Ms. Russo introduced her class to SOL at the beginning of the semester but had to shift her focus on preparing students for the GED. At the end of the semester she returned to SOL to pick up where they had left off. All in all, about 120 students at Frederick Douglass Literacy Center participated in Streams OnLine.
Island Academy is a NYC Board of Education site that serves incarcerated youth at the highschool level. Students may be in class for as short a period as two weeks and rarely longer than 6 months. Setting up SOL for Beta testing at IA followed shortly after it was set up at FDLC in March 1999. It was complicated by the difficulties involved in getting into the correctional facility.
Students at Island Academy were invited to try out the tool using a guest login and received a personal login if they showed special interest and frequent use of SOL. This was to minimize the "I lost my password" phenomenon. Willie Bennett, the computer instructor, introduced SOL to students who attended his computer classes. John Davis, an English teacher, introduced the tool to some of his students. Paul Auerbach, who works with students individually on reading and writing, introduced Streams OnLine to one of his students who showed an interest in poetry. Richard Spiegel introduced SOL to individual students during his visits to the school. At Island Academy, 11 students particpated in Streams OnLine.
Both Island Academy and Frederick Douglass started out with a local collaboration on SOL, limited to the students within that school. Collaboration between Frederick Douglass and Island Academy was anticipated to occur sometime during the spring semester, probably May 1999.
The following report is not an evaluation of a full scale use of Streams Online. It is a report of preliminary observations gathered during the beta test phase. It is intended to help plan for a formal launch of the tool this coming school year.
A Fun and Useful Tool
Participating in the Beta Testing for Streams Online at Frederick Douglass was like being in a private club. One student made a bulletin board in the hallway and posted poetry on it. In the last weeks of school, students from other classes began to hear about it, noticed the bulletin board, and came to Ms. Slovenz asking for logins.
Since Streams Online operates in a web-based environment it gives the feeling of being on the Internet, even when the collaboration is local to one school. One student at Island Academy wrote a description of himself that read like a personals ad, thinking that Streams OnLine was an Internet chat environment where he could meet a friend.
Madeline Slovenz, computer instructor at FDLC, reportedly saw more enjoyment and enthusiasm to get back to Streams Online than she observed in students using a word processor. She sensed an urgency in the students to complete their work because it would be shown to other students. The students knew that their work had a larger audience.
Ms. Tuch had four students that continued to use SOL on their own after she chose to redirect class activities to focus on developing chapbooks in a word processor.
Ms. Slovenz observed that Anasuya Isaacs' students were "highly motivated to get in there and plug away on that narrative. They were working on a project about "freedom stories," and typed their prose directly into SOL. It really brought the class together, compared to projects they had done before." Unfortunately, Ms. Isaacs did not approve any of the students' writings because she felt they were not "finished". As a result, a billion versions of half finished essays are piled up in the students' editing screens, waiting for approval. Ms. Issacs was overloaded and the cumbersomeness of the approval process added work. In addition, there is a reluctance on the part of some teachers to approve "unfinished" work which often means it contains spelling errors.
Students at FDLC have difficulty sustaining longterm interest in a project. SOL helped keep their interest by virtue of being a medium that feels "real, out there, alive, and worthy of their effort." It is different from a word processor because it is a communications medium.
Similar to the kind of focus exhibited by students who are preparing chapbooks, students feel they are "producing" work on Streams OnLine. Ms. Slovenz speculates that being a producer, and the process of production sustains interest in the long run.
Reading and Writing
At Frederick Douglass, all students who participated in SOL were able to enter text, read, and make comments.
Mr. Canale used Streams OnLine to help write the chapbooks for Waterways' end of the year hard copy publication of Streams. Students began the process on paper, with a poet in the classroom. They then took the poems they created in the classroom to the computer room, typed them up into a word processor, and cut and pasted them into Streams OnLine. By the end of the semester, students were writing poems on their own and typed them straight into SOL.
Mr. Canale observed that students began to use Streams OnLine instead of a word processor for all their writing: poems, paragraphs, and essays. Mr. Canale wondered whether students simply got used to using one program and then use it for everything. Typing straight into SOL was scarey for the students in the beginning because there is no spell checker. Over time, it appears they got used to typing into SOL and tolerated their own errors with the idea that they could go back and correct them. Mr. Canale suggests that this might a good thing. Typing without worrying about errors fosters writing in students since it encourages them not to worry about getting things right the first time around, but rather to get one's thoughts down.
Mr. Canale focused on the tool's ability to develop writing as an expressive experience with content, image, lines. Some students produced good pieces in terms of syntax, grammar, and spelling. For others it was a struggle to come up with some lines.
Lower functioning students are most handicapped when it comes to looking at a written piece in terms of grammar and structure. When the teacher would say to the students "go over it again," they would make simple corrections and have a friend help them. It was like doing a reading lesson. They helped each other out by pointing out "you need a capital letter there" or catching words that were mispelled. Spontaneous editing occurred when they worked together. Self correcting happened less. Corrections usually happened in collaboration with friends and classmates, or the teacher would give a quick read over the shoulder.
Collaborative editing between students does not always produce correct text. Consequently, Mr. Canale edits student work when he has to meet a deadline.
Ms. Slovenz saw no revision, in terms of working with ideas or writing style. She did observe some editing of grammer and spelling. She speculates that revision is just too difficult for most of the students, particularly on the lower reading levels. Sal Canale feels that it is not possible for students with 2nd level literacy skills to deal with revision. They can deal with editing, especially when paired up to help each other with spelling and grammar.
By the end of the semester, with the assistance Ms. Slovenz, two students advanced to incorporate graphic and sound files into their written works. Orlando Richards attached two images to his poem. The images were taken from an online collection that Ms. Slovenz had created for general use [see attachment "To Mom"]. Steve Guillard added a sound file to his poem. He created this file by making a recording of himself reading his poem [see attachment "What Do I Know"].
Ms. Slovenz demonstrated Guillard's audio poem to a student at the pre-K reading level, and observed how this student sat with it for a long time, playing the poem over and over again. The student was clearly reading along as she was listening.
Critical Thinking
Streams OnLine provides a venue for exercising students' critical thinking. Two features in particular foster this: the ability to comment on each other's work and the option to categorize one's work.
Commentary: A good example of how to use SOL to foster critical thinking is a lesson plan that Mr. Canale created [see attachment "Todays' Lesson"]. He entered two poems, one by Langston Hughes and another by a student in the class, Euvin Spencer. The assignment was to read the two poems and answer a question about how they differ. Mr. Canale printed out the lesson plan from SOL and handed it out in class to discuss before going into the computer room to type in their comments. All the students' comments become attached to the two poems. Euvin, the student whose poem was chosen, read all the comments he received and then later asked students verbally what they meant and a conversation followed.
Mr. Canale observed that when students write a poem expressing something it seems to come from a different place than when they make critical commentary. Poetically the students may produce powerful, expressive works while their comments are often undeveloped. Those who are proficient readers and writers gave better comments. By attaching comments to a written piece of work, SOL makes this difference in the two kinds of writing more visible. The way a teacher chooses to work with SOL can make a difference in how the students comment on each other's work.
In the last weeks of June, Ms. Russo's class returned to using SOL and were delighted to read the comments they had received during the semester. They in turn read and commented on other students' work. Being higher level readers, their first impulse was to comment on spelling and grammer. The teacher explained that "writing a comment is not correcting other's work. You may be correct but these are first level students and they are striving to express themselves and we do not comment on their spelling errors. We respond to what we get out of the poem." This helped students examine the task of commentary and they then wanted to revise their comments which Ms. Russo then approved. In their revised comments, students expressed emotional empathy, support, encouragement, and thanks for insights. This also illustrates how the teacher approval facility on SOL can be used as important opportunity to provide feedback.
Teachers at FDLC did not leave many comments. They were too busy working the room, helping students and approving their work. If the teachers had computers on their desks in the classrooms they would more easily be able to comment on student work in SOL. Ms. Tuch observes that when she stopped giving students feedback and comments on their work in SOL, most of them lost momentum and were not encouraged to continue. This suggests that receiving teacher feedback and comments is important to making Streams OnLine work. The email feature built into SOL also did not work, which provides another venue for teacher/student dialogue and feedback. It is important that teachers not be handicapped by the tool. Introducing a new piece of software into the classroom involves a learning curve and a period of getting adjusted to new work patterns.
At Island Academy, Mr. Bennett reported that he did not approve most of the student comments since they contained a lot of curse words and foul language. The teacher approval feature was created precisely for this purpose: to weed out obscenities. Since many students at Island Academy use a guest account, their comments were anonymous which may lower inhibitions and illicit more obscenity than if the comments were attributed.
For examples of student writing with commentary at FDLC, see attachements "Being Different", "The End of the World", "friendship", "Angels", "A love poem".
For examples of student writing with commentary at IA, see attachements "My Baby Girl!", "Just Listen to Her Voice", "My Life", "Black Knowledge", "How Do I tell her", "My time".
Categories: Students at both Island Academy and Frederick Douglass put their poems into categories.
Ms. Slovenz observed that choosing a category forces students to ask themselves: "what is my poem about? What's the topic? Is there another way of doing this?" This requires an ability to generalize and be concise. She observed that selecting a category gave the students pause. They would labor over choosing their categories. It is good that you can choose more than one. It would be good to let students create and define their own categories.
At Frederick Douglass, Mr. Canale did not request his students to categorize their work. He felt it was a bit too high for the level of his students and there was no need for it during this semester.
Collaboration
In the classroom: In the computer room there was alot of collaboration in front of the terminals as students were leaning over and pointing and helping each other. This may be related to the developmental reading level of the students. Just as likely it is due to the pleasure of socializing around computer terminals which gives students a chance to buddy and cluster. Perhaps it is difficult to use a new program and there is a spirit of helping each other out. Perhaps because the teacher is tied up, running around, the students help each other out. Students enjoy teaching each other. When someone knows how to do something, they want to share this knowledge.
Mr. Canale felt that Streams Online did lead to more collaboration in general. In a classroom with pencil and paper, students kept more to themselves. In the case of his lesson plan described above, the student whose poem was chosen, read all the comments he received and then later asked students verbally what they meant and a conversation would follow. This made the student feel more involved with his work.
Online: All the teachers reported that the students loved reading each other's writing and it was very important to receive comments. Receiving a comment was a mark of having been read. And the students want to be read.
Both Mr. Bennett and Ms. Slovenz observed that the students felt it was very real and present to have their writing "on the web". They had a heightened sense of audience when entering their work into SOL, as opposed to a regular text editor.
The Internet is a culture of reading and writing that these students care about. At Frederick Douglass, many of them spend time in chatrooms, which motivates them to invest some effort into learning to read and write quickly. Spelling errors are tolerated in this environment. As mentioned above, because SOL uses a web interface, it looks and feels like the Internet and gives the students a similar sense of ongoingness, community and presence. In fact, the technology of SOL is identical to the Internet, and that is what allows for collaboration between remote sites.
The collaboration between Frederick Douglass and Island Academy unfortunately did not occur in May as hoped for. Each school did produce a local collaboration among its own students. Due to technical complications, replication and transmission of data did not occur between the two sites. In late July, data finally began to transmit out of Island Academy.
Collaboration and feedback via email between student and teacher, did not occur at either Frederick Douglass or Island Academy. At FDLC it was due to the technical limitations of the computer lab configuration.
Revision Control
Most of the revisions on SOL are the result of minor editing and formatting changes. By the end of the semester there were so many different versions of the same poem, that it was cumbersome and confusing to the users, particularly lower level readers who may have difficulty identifying filenames.
Although Mr. Canale did not see much viewing of earlier versions, he did observe some students going back to look at an early version in the case of errors or to point out something such as "I changed that" and looked at an earlier version for confirmation. It appears that revision control is primarily useful for teachers and assessment. Users should be able to access earlier versions of their writing if they wish to, but only the latest version should be visible.
Technical Skills
Using Streams Online, or any new computer program, involves learning how to read instructions, menu bars, and typing in simple commands, userids and passwords.
At Island Academy, Mr. Bennett introduced Streams Online by first teaching how to use a web browser. He expressed satisfaction with this approach. Once students were comfortable with the web browser, they had no problem using SOL. A similar workshop may benefit teachers.
At Frederick Douglass, Ms. Slovenz saw no real difference in the way the classes of different reading levels used Streams Online at FDLC. The higher level readers had no problem at all. They sat in front of SOL and figured it out. The lower level readers needed more side by side help. In some cases it took a semester, but by the end there was universal independence with the tool.
Mr. Canale observed that facility with the computer improved while using Streams Online. Every application teaches something about computers and one's overall skills improve as a result. While two or three students in his class knew nothing about computers at the beginning of the semester, most of them started out knowing something about how to use a computer. Two students took advantage of the opportunity to learn how to add sound and images to their work, testing out the multi-media features of SOL.
For many students, Streams OnLine was their first experience with a web browser. For others, SOL represented a "readable version of the Internet." They would check themselves "online" to see if anyone had commented on their work. For a novice, using SOL is much easier than going to someplace on the Internet, where they might click around wildly from page to page without stopping to read anything because it is too advanced for them.
These observations suggests that there is a need for low level adult literacy resources on the web, with a chat room.
Electronic Portfolios
In general, the idea of students being able to publish online and read each other's work easily was enthusiastically received by teachers. It is good for the teacher to be able to read the student's work easily. All the work resides in one place. It is a convenient, tangible, public, and accessible venue. The material on SOL is easy to print out and bring into the classroom.
Publication
Students at both Island Academy and Fredeick Douglass were informed that material submitted to SOL may eventually be included in an online publication. At Island Academy this was important to one student who came into the lab repeatedly to submit work into SOL because he heard it might get published.
As noted above, SOL looks, feels and acts like the Internet because it uses a web interface. As a result, participants are aware of the possibility of a larger audience.
A domain name has been obtained for any future publication of Streams OnLine.
For the time being, this web url contains a demo of SOL for public testing, and password protected access to replications of the Frederick Douglass and Island Academy local collaborations.
Future Plans
All of the teachers at FDLC and Island Academy who tried out SOL were enthusiastic about the potential for Streams Online and said they plan to use it next year. They like the idea of students being able to collaborate online and having easy access a large body of work, their poems and others'. Online experience, access, collaboration, and fostering the creative process are the values of this tool.
SOL is still cumbersome for the teacher to work with although it was improved during the test period. Some teachers would like to be able to delete work and request super-user status. Teachers will not bother with the tool if it is inconvenient.
Teachers requested a formal presentation of the tool that explains, "this is what we're doing, why are we doing it, and what do we want to achieve in the end?". They need more preparation and understanding of Streams OnLine. Paul Auerbach and John Davis expressed the need for some training to understand how to use SOL and become more confident with it. Both Ms. Russo and Mr. Auerbach expressed anxiety over presenting a tool to students without knowing how to use it themselves. Mr. Auerbach even suggested a workshop for both teachers and students together. A training workshop would help "sell" this new tool to the teachers. Ms. Slovenz plans to make an event to formally launch Streams OnLine among the students at FDLC in the fall of 1999, perhaps with a contest.
All the teachers interviewed express the hope for an interschool collaboration.
Streams OnLine is licensed under the GNU General Public License (GPL) which is an open source license [For a technical description of Streams Online, see attachment "Streams OnLine: A Technological Description"]. Developers plan to make it available for downloading from the Internet some time in fall 1999. A todo list and a bug list will be maintained and worked on as labor is available. The manual also needs to be updated. It is hoped that other schools will show an interest and deploy their own Streams OnLine operation. It is hoped that additional developers will collaborate with us.
This evaluation was prepared for Waterways of Ten Penny Players who received funding from the NY State Council on the Arts, Empire State Partnership and NYC Annenberg Challenge for Arts Education to develop Streams On Line. Contact: lucia@lrw.net