Looking at Student Roster Reports

In Edwin Analytics, the "Students, Staff & Classroom" tab contains a number of student-level MCAS reports. Some of these reports are "roster" reports which means that they display data for a given list of students in a tabular format, usually one line of data per student. Student reports that are not in "roster" format generally display a page or two of data for each student.

The students included in a report can be selected in one of two ways - by using filters (such as "all ELL students" or "all students who scored Advanced") OR by selecting students from a list (such as a teacher's class list from last year, from their class coming in this year, etc.) There is a document in the Edwin Analytics handouts folder that provides further instructions.

Some of these roster reports include:

• IT616 Test Item Analysis Roster - provides test item data as well as achievement level, scaled score, raw score, SGP, % correct, and CD status for each student in a list. One challenge with this report is that the font is VERY small if it is printed on a single sheet of paper. I have found that if you save your report as a PDF file, Adobe Reader has a "poster" option that will allow you to print the file larger across multiple pages. This report is single-subject (ELA, math OR science.)

• PE617 Student MCAS Roster - provides a list of students and their scaled scores, achivement levels, and SGPs for all tested subjects (ELA, math, AND science) on a single report. This report is very useful if you would like to look at performance on all 3 tests for a given group of students.

• GR602 Student Growth Roster - provides MCAS scaled score information for 3 years of ELA and math performance, plus the most recent year's SGP for each student. This is very useful for examining individual student performance over time in a compact way.

And here is one more "roster" report that I had not seen until a few days ago when a colleague shared it with me. If it is new to you as well you may want to check it out, particularly if you're interested in a roster report of student performance by question type or standard.

• PE606 (Student Assessment Detail) report has traditionally been one of those reports that was a page or two per student. However, there is now a "roster" option on the initial prompt page. If you choose this, you'll get a report similar to IT616, except it will give information by question type and standard rather than by item. 

Folks that have access to the Classroom-level reports (available only to those with the DW-Evaluator role) should note that the IT516 and PE506 reports are similar to the 600-level counterparts listed above, but that they can be run for a selected classroom.


  
 

Exploring Classroom Growth Data

A favorite EDW report of many folks is the GR601 report, the Student Growth Scatter Plot. It displays a dot for each student showing his or her SGP (x-axis) and scaled score (y-axis) for a given MCAS test for an entire grade level. Wouldn't it be great to have this same scatter plot for a single classroom of students? There isn't currently an EDW report like this, but creating one by hand is a fantastic exercise for engaging with student data, particularly since student growth data is a piece of Massachusetts' new Educator Evaluation model.

You'll need a student data set (scaled score and SGP for each student for a given test) and a blank copy of the scatterplot document. I've written out some more detailed guidance for those that are interested, but this could be as formal or informal an activity as you like, as long as the participants have a working knowledge of SGP and how it is determined. Creating this type of graph by hand provides an opportunity for teachers to really engage with the data, student by student, and to reflect on the story the data tells for both individual students as well as groups or classrooms.

The activity I am sharing has evolved over time, and I would like to acknowledge Paula O'Sullivan (Data Specialist, Greater Boston DSAC), Mary Fredrickson (Assessment Specialist, Quincy Public Schools), and the educators at the Marguerite E. Small School in Yarmouth for their contributions to the development and refinement of this activity and related materials. I would love to hear from you (laura.tilton@dsacma.org) if you give this activity a try - feedback is always welcome!

MCAS and Accountability Teleconference Highlights

I know that many readers of this blog attended this week's teleconference related to the release of the preliminary 2012 MCAS results and related accountability reports but I thought I would provide some of the essential information for those that were not able to attend.

MCAS Results and Discrepancy Reporting
Earlier this week, the preliminary MCAS data files were posted in the ESE Security Portal for review and discrepancy reporting. Some preliminary EDW reports are expected to be available in the Education Data Warehouse by this Friday, August 10 (ELA long compositions can be downloaded from the MCAS Service Center.)

As a reminder, the preliminary data is embargoed - Commissioner Chester has given the following guidelines for the use of preliminary data:  “I want to encourage schools and districts to use the preliminary… student data for educational planning purposes. Preliminary student results can and should be shared with teachers, parents and instructional leaders who are making decisions about Individual Education Plans, summer programs or decisions about the scope of services needed for English Language Learners.” The folks from ESE expect that the 2012 MCAS and accountability results will be released publicly on or about September 19, 2012. 

Accountability Reporting

In June, districts received their schools' provisional PPI data and accountability and assistance levels, which are based on data from 2009, 2010, and 2011. During the week of August 13, districts will receive their preliminary school-and district-level PPI data and accountability and assistance levels based on four years of data (2009, 2010, 2011 and 2012) for review (also embargoed). The official results will be released in mid-September.

Education Data Warehouse Updates

There are lots of excellent new updates in the Education Data Warehouse just in time for the busy season! Here are some highlights:

• The EDW Home page is now standard for all EDW users without needing to be configured as previously discussed. It contains information about recent updates and improvements as well as what's coming down the pike. 
• All of the reports from version 2.6 have been migrated to the new EDW. Please note that although the link is still available from the landing page, EDW 2.6 is no longer being updated and will be retired in October.
• The Test Item Analysis reports now have the option to map the test items to EITHER the old or new curriculum frameworks for Math and ELA. 
• There is a "Super" student profile report which provides SIMS, attendance, course, MCAS, and CD status information for any selected student.
• There are several new classroom-level reports which utilize SCS data for a given instructional group (classroom, course section, etc.) These reports are only available to users who have been assigned the "DW-Evaluator" role. ESE has developed guidance around assigning this role to EDW users which they will post on the EDW home page shortly. (Watch for a more detailed blog post once this guidance document is available online.)

Growth and EDW 3.0 Workshops August 2012

The Southeast DSAC (Laura Tilton, Data Specialist) will be offering the following no-cost workshops on Thursday, August 16 at Pembroke High School. Workshops require a minimum of 12 participants (maximum of 24) and priority will be given to participants from regional Level 3 districts. (Customized in-district workshops are also available upon request for Level 3 districts.)

Getting to Know Data Warehouse 3.0 (2.5 hours, 9:00-11:30, August 16)
The workshop will provide a structured environment in which to explore data using the pre-defined reports in the new Education Data Warehouse, including the Growth Model reports and NEW classroom-level reports. Participants will access their own district or school’s data, and will have time to run and save reports for use in planning for the 2012-2013 school year. This workshop is appropriate for both new and previous users of the Education Data Warehouse; all attendees must have been assigned appropriate Security Portal access to the EDW by their district's Directory Administrator.

Exploring MCAS Growth Data (2 hours, 12:00-2:00, August 16)
This workshop (suggested as an extension to the previous workshop) will go beyond an overview of the Growth Model and focus on the use of MCAS growth data and reports at multiple levels (district, school, classroom/group, and student). Participants will deepen their understanding of the growth model and how to interpret growth reports (with connections to the new Educator Evaluation System), engage with school/district growth data following a data analysis protocol, and learn what growth reports are available and where to find them.

Please register via this form to reserve your spot. Thank you!

Examining MCAS Growth Data by Classroom

One message about the new educator evaluation system that comes through loud and clear is that student growth data will be an important measure considered (where available). This school year is a great opportunity to provide more information to staff about MCAS growth data in order to have folks become more comfortable with this measure and what it tells us.

As a reminder, the median student growth percentile for any group is the midpoint of student growth percentiles in the group (which must be >20 students), where half of the students have student growth percentiles higher than the median; half have lower. (It is not appropriate to use the average or “mean” when comparing percentiles.)

Last week, I visited an upper elementary school and presented an overview of the growth model to the staff. After sharing some of the basics, I asked the staff to share with me ways that they would like to look at growth data for their students. Teachers responded that they would like look at median growth for particular groups of students (subgroups, interventions, programs) but also that they wanted to see the median growth for their own classrooms over time. (And yes, I'll be working with the principal to help get this information to the teachers!)

I've been hearing people ask for a while now how they can more easily calculate the median SGP by group, especially by classroom. The Student Growth Roster (G-602) report in the older version of the EDW (2.6) will display a median SGP for a custom roster of students as long as at least 20 students are selected. In the next few weeks I'll be putting together some revised instructions on how to do this (students can be selected using a list, filters, or SASIDs) so stay tuned for more resources on this front!

Creating Customized Student Roster Reports, September 2011

I know there are many folks who would like to create customized MCAS student roster reports "by classroom" or a variety of other groups - intervention group, performance level, subgroup, etc. This type of reporting is not currently available in the Data Warehouse. However, there are a few different options you can look into if you'd like to run reports this way.

The first option is to use the R-616 report (only available at this time in EDW 2.6, not EDW 3.0) to create a custom report either using filters or by choosing students from a list. This can be a bit tedious if you're creating reports by classroom but it's probably the quickest option at this time.

The second option is to start with the roster spreadsheet report from the Dropbox, and use Excel to add in a column for teacher or intervention group. The Filter option in Excel can then be used to create lists based on various criteria AND then Excel could be used to calculate group statistics such as median SGP (>20 students). However, entering the information into the teacher/group column could take a while.

For more advanced Excel users only: There is a function in Excel called VLOOKUP which would allow the automation of the addition of teacher/group data to a column in the roster spreadsheet. In order to do this, you'd need your Dropbox roster spreadsheet PLUS a spreadsheet that lists students (by SASID) by teacher/group. The SASID field will serve as a common data point between the two spreadsheets and allow you to bring data from one spreadsheet into another. Someday I'll write a tutorial for this, but for now, here's a link to a great VLOOKUP tutorial online for anyone would like a reference.