ESE Model Student and Staff Feedback Surveys Posted:

The Department of Elementary & Secondary Education has recently posted student and staff 
model feedback surveys at http://www.doe.mass.edu/edeval/feedback/

Districts may adopt or adapt these surveys, and/or choose to use other feedback instruments.

ESE has also published guidance which includes: information about how to identify appropriate feedback instruments, recommendations on the use of student and staff feedback in the five-step evaluation cycle, and a review of alternative methods for collecting feedback from students and educators.

Posted Model Surveys and Administration Protocols include:

• Student Surveys - Gr. 3-5
• Student Surveys - Gr. 6-12
• Staff Surveys
• Discussion Prompts for Students in Gr. K-2

Using Student Growth Percentiles in Educator Evaluation

The folks at ESE recently released several Implementation Briefs focused on timely questions regarding District Determined Measures and Student Impact Ratings. One of these briefs focuses on the question, "How do districts use median Student Growth Percentile in the evaluation of teachers and administrators?" Many of the questions we have been asking about this use of SGP are addressed in this brief, including clarification about determining which teachers (and administrators) should use SGP as part of their Student Impact Rating determination.

While the median growth score for a group of students is the measure used, it is also very valuable to for an educator to examine the distribution of their students' MCAS growth and performance scores. In Edwin Analytics, we can look at a scatter plot of growth and performance by grade level, but no such report exists at this time for a classroom set of students. Creating this type of plot by hand can be very valuable in helping an educator explore classroom-level growth data in a reflective way.

Here's a link back to the activity (originally posted in August 2012, so the references are to EDW rather than Edwin Analytics) which gives some suggestions for facilitation along with a blank scatter plot and other resources: http://southeastdata.blogspot.com/2012/08/exploring-classroom-growth-data.html. I believe this is a valuable exercise as we talk about SGP in the context of educator evaluation - grounding the overall median score in reflection and explanation!

Educator Evaluation Update, District-Determined Measures

The Commissioner recently released a memo with an update for the timelines and expectations for implementing District-Determined Measures (DDMs) and the Student Impact Rating of the educator evaluation system. The DDM update can be found here http://www.doe.mass.edu/news/news.aspx?id=7640

District Determined Measures: Technical Guide A

From the May 17 Commissioner's Weekly Update:

Technical Guide A, a supplement to Part VII of the Mass. Model Educator Evaluation System, is now available at http://www.doe.mass.edu/edeval/ddm/TechnicalGuide.pdf. The guide describes key assessment concepts that district teams should consider when selecting district-determined measures. It also provides guidance on how to select or modify existing assessments and how to build a new assessment.

Educator Evaluation Spring Convening 5/29/2013

Educator Evaluation Spring Convening:
Connecting Policy, Practice, and Practitioners
Wednesday, May 29th, 7:45am-4:30pm

Sturbridge Host Hotel & Conference Center
366 Main Street, Sturbridge, Massachusetts 01566

The Department of Elementary and Secondary Education is excited to bring together public school districts, associations and educator preparation programs for a day of celebration, learning and reflection about promising and emerging practices in the new Educator Evaluation framework. This new framework will be implemented statewide in the 2013-2014 school year; however, many districts have already begun to engage in this work. This is an opportunity for districts currently implementing the framework to share their successes and challenges and indentify new and actionable strategies for improving implementation in the coming year. It is also a chance for newly implementing districts to learn from and connect with colleagues who have been engaged in this work over the past two years. ESE is pleased to bring together districts, associations, and educator preparation programs to build connections and relationships across the educator pipeline in Massachusetts.

More information and registration (by 5/15) is available at:
http://registration.airprojects.org/MDESEEducatorEvaluationDistrict/

ESE Assessment Literacy Webinar Series

ESE is offering an 8-part webinar series which will run from now through December, focused on assessment literacy in the context of district-determined measures (DDM's) for educator evaluation. These sessions will run 60-90 minutes and are designed for district and/or school teams who will be involved in the identification and selection of district-determined measures. The webinars and related materials will be archived and posted online along with the schedule at http://www.doe.mass.edu/edeval/ddm/webinar.html.

Webinar Session 1, "Introduction: District-Determined Measures and Assessment Literacy" was offered last Thursday, March 14, but will run a second time on Wednesday, March 27, at 4:00 pm. The registration is available online (the registration page also includes the handouts/PowerPoint for Session 1) through this link.

Also, stay tuned to the ESE site for "Technical Guide A" which will be coming this spring to accompany Part VII: Rating Educator Impact on Student Learning Using District-Determined Measures of Student Learning.

Additional Guidance on District-Determined Measures

As part of the Commonwealth's new educator evaluation system, districts will be required to identify and administer a district-wide set of student performance measures for each grade/subject that "permit a comparison of student learning gains." Where MCAS student growth percentile (SGP) exists, regulations state that it must be used as one of these measures, but it will be up to districts to identify/develop additional measures of student learning gains for many grades and subjects.

(You may recall a previous blog post about an activity designed to help teachers engage with and reflect on classroom-level growth data...I continue to use this activity with very positive responses from teacher teams!)

ESE has recently released a Quick Reference Guide on District-Determined Measures (along with QRG's on other EdEval topics) which summarizes the regulations and timeline related to the implementation of DDM's for all districts in Massachusetts. Additional guidance on the identification and selection of DDM's and related topics is scheduled for release later this Spring.

Data Warehouse Updates

The following information was shared during last week's “EDW User Group” webinar. See the bottom of this post to learn how you too can be part of the user group and hear this information first-hand!

Additional Data Now Available

With the addition of the 2011-12 End-of-Year Student Course Schedule data and EPIMS data:

1. The Classroom reports now have two years of data available. These reports currently include:

• Classroom Item Analysis Roster (IT516)                         
• Classroom Results by Standard (PE506)
• Performance and Growth by Teacher (PE520)

These reports are only available to users who have been assigned the "DW-Evaluator" role. ESE has developed guidance around assigning this role, available on the "Home" tab within the EDW. 

2. The Student Profile report (PR600) now displays two years of course data for the selected student(s).

Additional Functionality

1. The ability to view your data either by the  “old” MA2004 standards or by the new Common Core standards has been added to the Student Assessment Detail report (PE606) and the Student Item Analysis Roster report (IT616), bringing the total to six EDW reports with this option (the Test Item Analysis Summary reports (IT301/IT401) and the Results by Strand and Topic reports (CU306/CU406) already had this option). This option will soon be added to the Classroom Item Analysis Roster (IT516) and the Classroom Results by Standard report (PE506).

2. The Student Assessment Detail report (PE606), which previously printed one page per student, now has the option for a roster display (list of many students on one page).

3. The Growth Distribution reports (GR302/GR402) now have an option to display multiple years of data for a particular grade level for trend analysis.

Reminders

1. The EDW Team at DESE holds webinars twice a month (1^st Wed at 11am, 3^rd Wed at 2pm) to share information regarding new features and data in the EDW. Participants have an opportunity to share comments, questions and/or suggestions. To join the listserve and receive reminders about these webinars, please see the bottom of the EDW home page.

2. The supported browsers for the EDW are Internet Explorer and Firefox. You may experience issues with other browsers. It is expected that at some point in the future Safari and Chrome will also be supported.

3. The old EDW (v2.6) will be put out to pasture after this month (i.e. no longer available). Is anyone still using the old version??? 

(Thank you to Paula O'Sullivan, Data Specialist for the Greater Boston DSAC, for sharing this summary with me so that I didn't have to re-invent the wheel.)

Exploring Classroom Growth Data

A favorite EDW report of many folks is the GR601 report, the Student Growth Scatter Plot. It displays a dot for each student showing his or her SGP (x-axis) and scaled score (y-axis) for a given MCAS test for an entire grade level. Wouldn't it be great to have this same scatter plot for a single classroom of students? There isn't currently an EDW report like this, but creating one by hand is a fantastic exercise for engaging with student data, particularly since student growth data is a piece of Massachusetts' new Educator Evaluation model.

You'll need a student data set (scaled score and SGP for each student for a given test) and a blank copy of the scatterplot document. I've written out some more detailed guidance for those that are interested, but this could be as formal or informal an activity as you like, as long as the participants have a working knowledge of SGP and how it is determined. Creating this type of graph by hand provides an opportunity for teachers to really engage with the data, student by student, and to reflect on the story the data tells for both individual students as well as groups or classrooms.

The activity I am sharing has evolved over time, and I would like to acknowledge Paula O'Sullivan (Data Specialist, Greater Boston DSAC), Mary Fredrickson (Assessment Specialist, Quincy Public Schools), and the educators at the Marguerite E. Small School in Yarmouth for their contributions to the development and refinement of this activity and related materials. I would love to hear from you (laura.tilton@dsacma.org) if you give this activity a try - feedback is always welcome!

Growth and EDW 3.0 Workshops August 2012

The Southeast DSAC (Laura Tilton, Data Specialist) will be offering the following no-cost workshops on Thursday, August 16 at Pembroke High School. Workshops require a minimum of 12 participants (maximum of 24) and priority will be given to participants from regional Level 3 districts. (Customized in-district workshops are also available upon request for Level 3 districts.)

Getting to Know Data Warehouse 3.0 (2.5 hours, 9:00-11:30, August 16)
The workshop will provide a structured environment in which to explore data using the pre-defined reports in the new Education Data Warehouse, including the Growth Model reports and NEW classroom-level reports. Participants will access their own district or school’s data, and will have time to run and save reports for use in planning for the 2012-2013 school year. This workshop is appropriate for both new and previous users of the Education Data Warehouse; all attendees must have been assigned appropriate Security Portal access to the EDW by their district's Directory Administrator.

Exploring MCAS Growth Data (2 hours, 12:00-2:00, August 16)
This workshop (suggested as an extension to the previous workshop) will go beyond an overview of the Growth Model and focus on the use of MCAS growth data and reports at multiple levels (district, school, classroom/group, and student). Participants will deepen their understanding of the growth model and how to interpret growth reports (with connections to the new Educator Evaluation System), engage with school/district growth data following a data analysis protocol, and learn what growth reports are available and where to find them.

Please register via this form to reserve your spot. Thank you!

Growth and EDW 3.0 Workshop Offerings Summer 2012

The Southeast DSAC (Laura Tilton, Data Specialist) has set aside July 24 and August 16 to offer the following set of workshops. Workshops require a minimum of 12 participants (maximum of 24) and priority will be given to participants from regional Level 3 districts. However, we anticipate that we will able to accommodate educators from all interested Southeast districts.


Getting to Know Data Warehouse 3.0 (2.5 hours, 9:00-11:30, July 24 or August 16)
The workshop will provide a structured environment in which to explore data using the pre-defined reports in the new Education Data Warehouse, including the Growth Model reports and classroom-level reports. Participants will access their own district or school’s data, and will have time to run and save reports for use in planning for the 2012-2013 school year. This workshop is appropriate for both new and previous users of the Education Data Warehouse; all attendees must have been assigned appropriate Security Portal access to the EDW by their district's Directory Administrator.

Exploring MCAS Growth Data (2 hours, 12:00-2:00, July 24 or August 16)
This workshop (suggested as an extension to the previous workshop) will go beyond an overview of the Growth Model and focus on the use of MCAS growth data and reports at multiple levels (district, school, classroom/group, and student). Participants will deepen their understanding of the growth model and how to interpret growth reports (with connections to the new Educator Evaluation System), engage with school/district growth data following a data analysis protocol, and learn what growth reports are available and where to find them.

Please register via this form and let us know which workshops interest you and whether your district may be able to host the workshop. Thank you!

Commissioner's Update: February 16, 2012

An update from the Commissioner is now available on the ESE Web site. Topics include the Model System for Educator Evaluation, the TELL Mass survey, DART Detail: ELLs, a Race to the Top Newsletter, EPP plans and spring 2012 assessments, and more. There sure is a lot happening in Massachusetts schools!

1/13/2012 Commissioner's Update

The recent update from the Commissioner shared information about the materials for the Educator Evaluation Model System that were released last week. As I digest these new materials I will share relevant resources on the blog, as there are many references to the use of data in this new system.

In addition, the Commissioner announced that the 2012 Early Warning Indicator System (EWIS) rosters are now available in the Security Portal (Accountability 2011 Data Dropbox) for those who have been assigned the district accountability security role.

A Holiday Gift for You

I have been working with a fellow data specialist to create a how-to guide on Creating Student Growth Rosters for Instructional Groups. If you're interested in generating growth/performance reports in the EDW for a specific classroom, intervention group, team, etc., this is the process to follow. Basic knowledge about the use of the Education Data Warehouse and Microsoft Excel are required to complete this process.

The document focuses on the G-602 Student Growth Roster (in the older EDW) which provides three years of MCAS performance data (scaled score) and the most recent student growth percentile (SGP) for each student for both Mathematics and ELA. The report also provides a median SGP for the group, if there are 20 or more students with SGPs in the group.

Please refer to my previous blog post on Examining MCAS Growth Data by Classroom for more thoughts about the opportunity to engage staff in discussions about growth data now as we move toward the new educator evaluation system.

I shared this process with a district colleague earlier in the week, and she sent me an email the next day saying, "I think I love you!!!!...Our secretary was able to get 3 years instructional groups for me with state IDs. The reports are soooooo easy to do and save!!! You saved me tons of time. I am very excited." So I hope there are other readers out there who will have the same reaction.

Happy Holidays to all!

Educator Evaluation Resources Coming from ESE

The December 8 Commissioner's Update included (among many other important bits of information) several resources under development related to the state's new educator evaluation system. The Model System will be released in January, accompanied by a webinar and regional rollout meetings for Race to the Top districts. Later this spring, there will be a set of training materials and other resources available to support districts as they implement this new system.

Southeast Data Coach Network

The Southeast DSAC Data Coach Network is an opportunity for school-based data coaches to participate in a regional professional learning community focused on building structures of collaborative inquiry. Network sessions will focus on sharing best practices and building upon skills and knowledge gained in the Data Coaching course from Research for Better Teaching.

The Network is intended for participants from Level 3 districts and "legacy" Level 3 districts who have taken the Data Coaching course from RBT. These districts were provided with a previous opportunity to register, and we have determined that there are openings for a few additional participants. The first meeting is scheduled for this Friday, December 2, from 12:30 pm to 3:00 pm in Plymouth, MA.

The highlighted topic for this Friday's meeting will be "Using the Data-Driven Dialogue Process to Explore Student Growth Data." The RBT course highlights a process of drilling down to strand/item/student work, but did not touch on exploring Student Growth Data. This measure will be a piece of the new Educator Evaluation Framework from ESE (where applicable) so reflective conversations about this measure now will certainly benefit schools in the long term. We will also take some time to explore selected resources from the Data Coach materials that we did not spend time on during the course this summer and plan for future network meetings.

I apologize for the last-minute announcement of openings, but would be happy to hear from additional district or school-based data coaches (or those that serve in a data coach role) that would be interested in participating. You may register at this link: https://docs.google.com/a/dsacma.org/spreadsheet/viewform?hl=en_US&formkey=dGs4TDZWOVhDckprQ3JhQTRDQXZqSWc6MA#gid=0

If you or others in your district have taken the RBT Data Coaching course, but are not able to attend this first meeting, please don't hesitate to contact me to share your interest. (laura.tilton@dsacma.org)

Examining MCAS Growth Data by Classroom

One message about the new educator evaluation system that comes through loud and clear is that student growth data will be an important measure considered (where available). This school year is a great opportunity to provide more information to staff about MCAS growth data in order to have folks become more comfortable with this measure and what it tells us.

As a reminder, the median student growth percentile for any group is the midpoint of student growth percentiles in the group (which must be >20 students), where half of the students have student growth percentiles higher than the median; half have lower. (It is not appropriate to use the average or “mean” when comparing percentiles.)

Last week, I visited an upper elementary school and presented an overview of the growth model to the staff. After sharing some of the basics, I asked the staff to share with me ways that they would like to look at growth data for their students. Teachers responded that they would like look at median growth for particular groups of students (subgroups, interventions, programs) but also that they wanted to see the median growth for their own classrooms over time. (And yes, I'll be working with the principal to help get this information to the teachers!)

I've been hearing people ask for a while now how they can more easily calculate the median SGP by group, especially by classroom. The Student Growth Roster (G-602) report in the older version of the EDW (2.6) will display a median SGP for a custom roster of students as long as at least 20 students are selected. In the next few weeks I'll be putting together some revised instructions on how to do this (students can be selected using a list, filters, or SASIDs) so stay tuned for more resources on this front!

MassCUE Conference Happenings

Wednesday I attended Day 1 of the 2011 MassCUE conference at Gillette Stadium, and it was exciting as usual! Lots of great presenters, vendors, and opportunities to share ideas with colleagues made for a productive and engaging day.

The Spotlight Session entitled, "MA New Educator Evaluation Regulations: Opportunity and Challenges" presented by Karla Brooks Baehr provided an opportunity to learn more about the details in the developing story of the new educator evaluation regulations (with specific discussions about positions such as technology integration specialists that work with both staff and students). Two major takeaways from this session were that (1) both teachers and administrators should be taking time to become VERY familiar with the state's growth model, since this is one of the measures that will be used (where applicable) for educator evaluation; and (2) the Commonwealth's Model System is on target to be available for review on January 10, 2012. A variety of resources, including a version of the PowerPoint that was presented at the conference, are available on the ESE's Educator Evaluation web page.

Associate Commissioner Bob Bickerton and Director of Information Services Rob Curtin presented "Next Generation Supports for Teaching, Learning, and Student Success" which highlighted a variety of initiatives within ESE, including the new Curriculum Frameworks, PARCC, and many RTTT projects, especially the Teaching and Learning System which is currently under discussion. The ESE Showcase area provided an opportunity for conference attendees to learn more about initatives including the Education Data Warehouse, Thinkfinity Partnership, PBS LearningMedia Partnership, and the Teaching and Learning System.

The final session of the day entitled, "The Paperless Classroom: Tips and Tricks for the 21st Century Educator" was fascinating and introduced attendees to a wide variety of Web 2.0 tools I had not seen before. While these tools are not data-specific, I'd like to share some them here for those of you with an interest in staying current with educational technology. The presenter, Melissa Panchuck from the Fessenden School, was incredibly passionate and engaging about her use of Web 2.0 tools with her 5th grade students, including:

• Glogster - virtual posters
• Google Sketchup -
• Pixton - online comics
• Blabberize - talking pictures
• Go!Animate and XtraNormal - animations
• Screenr - instant screencasts
• diigo - link collections and highlighting
• issuu - digital publishing

 So many new and exciting tools (both inside and outside the classroom!) that can be used to improve teaching and learning!