Exploring Classroom Growth Data

The "Exploring Classroom Growth Data" activity I posted last fall invites educators to engage with student growth and performance data by creating a scatter plot for a given group of students. I've used this activity extensively with teachers throughout the year and I know others out there have done the same. I wanted to post a link back to the activity and accompanying instructions for those who would like to use it this fall to help educators explore and engage with student growth data at the classroom level.

http://southeastdata.blogspot.com/2012/08/exploring-classroom-growth-data.html 

MCAS Data Investigation Activity Ideas

Last week I posted information about an activity focused on the exploration of classroom growth data through the paper-and-pencil creation of a scatter plot of growth and scaled score for a given instructional group. One colleague asked me, "I wonder if people will be able to see how cool the activity is just by reading about it...or maybe it's only people like us who get excited by scatterplots??"

So, I'm posting this follow-up because I bet there are folks out there who missed the post last week, but I know you'll agree that the activity is worth investigating! Hope you'll check it out once you come up for air during this busy time of year. I also wanted to link back to some previous years' blog posts with additional MCAS data investigation activities...some ideas for digging in to the data during fall staff meetings or PLC's!

• Exploring Classroom Growth Data (August 2012)
• MCAS Item Analysis and the Standards for Mathematical Practice (August 2011)
• Engaging Staff in MCAS Data Exploration (September 2010)

Happy first days of school!

Growth and EDW 3.0 Workshop Offerings Summer 2012

The Southeast DSAC (Laura Tilton, Data Specialist) has set aside July 24 and August 16 to offer the following set of workshops. Workshops require a minimum of 12 participants (maximum of 24) and priority will be given to participants from regional Level 3 districts. However, we anticipate that we will able to accommodate educators from all interested Southeast districts.


Getting to Know Data Warehouse 3.0 (2.5 hours, 9:00-11:30, July 24 or August 16)
The workshop will provide a structured environment in which to explore data using the pre-defined reports in the new Education Data Warehouse, including the Growth Model reports and classroom-level reports. Participants will access their own district or school’s data, and will have time to run and save reports for use in planning for the 2012-2013 school year. This workshop is appropriate for both new and previous users of the Education Data Warehouse; all attendees must have been assigned appropriate Security Portal access to the EDW by their district's Directory Administrator.

Exploring MCAS Growth Data (2 hours, 12:00-2:00, July 24 or August 16)
This workshop (suggested as an extension to the previous workshop) will go beyond an overview of the Growth Model and focus on the use of MCAS growth data and reports at multiple levels (district, school, classroom/group, and student). Participants will deepen their understanding of the growth model and how to interpret growth reports (with connections to the new Educator Evaluation System), engage with school/district growth data following a data analysis protocol, and learn what growth reports are available and where to find them.

Please register via this form and let us know which workshops interest you and whether your district may be able to host the workshop. Thank you!

Farewell, AYP....hello, PPI!

I had the opportunity to take part in both a webinar and a face-to-face session in the past week to learn more about the changes to the state's accountability system given the the flexibility that has been granted to Massachusetts under the NCLB waiver.

The waiver has given Massachusetts the flexibility to implement a system that unifies both federal and state accountability requirements, rather than separate systems for each. The major changes, as outlined in the presentations, are as follows:

• The NCLB goal of 100% proficiency has been replaced by a new goal: reducing proficiency gaps by half by 2017. This goal is the same for all districts, schools, and groups...but the targets are different since the starting points will be different.
• The NCLB accountability status labels (Corrective Action, Restructuring, etc.) have been eliminated. The "Level" designation will now be used to identify both federal and state accountability status (and corresponding assistance levels) for schools and districts.
• AYP will be replaced with a new performance measure, known as PPI (see below) that incorporates student growth and other indicators.
• There is an enhanced focus on subgroups, including a new "high needs" group (students who fall into one or more of the following categories: special education, low income, or English language learner/former English language learner.) While all subgroups (minimum 30 students) will be reported, only the "All Students" and "High Needs" groups will be used for classification for the majority of districts.
• SES and school choice requirements for federal funds are replaced with options for flexibility to address identified needs.

So what is PPI? It stands for Progress and Performance Index. Each district, school, and measurable subgroup will  have two different types of PPI; an annual PPI as well as a four-year PPI. For elementary and middle schools, the PPI incorporates five elements:

• Progress on CPI gap-closing in ELA
• Progress on CPI gap-closing in math
• Progress on CPI gap-closing in science
• Student Growth in ELA
• Student Growth in Math

PPI measures for high schools will include these five elements, plus graduation rates and annual drop-out rates. In addition, it is possible to improve a PPI rating through decreases in %W/F and increases in %Adv.

This is just the tip of the iceberg! The ESE presenters have indicated materials from the workshops will be available online in the near future. I will post the link here when I have it.

Update 5/9/2012: Materials now available, see blog post here.

Southeast Data Coach Network 1/27/2012

The Southeast DSAC Data Coach Network is an opportunity for school-based data coaches (and others who support collaborative inquiry) to participate in a regional professional learning community focused on building structures of collaborative inquiry. Monthly network sessions focus on sharing best practices and building upon skills and knowledge gained in the Data Coaching course from Research for Better Teaching.

The Network is intended for participants from Level 3 districts and "legacy" Level 3 districts who have taken the Data Coaching course from RBT. These districts were provided with a previous opportunity to register, and we have determined that there are openings for a few additional participants. The next meeting is scheduled for this Friday, January 27, from 12:30 pm to 3:00 pm in Plymouth, MA.

I apologize for the last-minute announcement of openings, but would be happy to hear from additional district or school-based data coaches (or those that serve in a data coach role) that would be interested in participating. You may register at this link. Thank you!

A Holiday Gift for You

I have been working with a fellow data specialist to create a how-to guide on Creating Student Growth Rosters for Instructional Groups. If you're interested in generating growth/performance reports in the EDW for a specific classroom, intervention group, team, etc., this is the process to follow. Basic knowledge about the use of the Education Data Warehouse and Microsoft Excel are required to complete this process.

The document focuses on the G-602 Student Growth Roster (in the older EDW) which provides three years of MCAS performance data (scaled score) and the most recent student growth percentile (SGP) for each student for both Mathematics and ELA. The report also provides a median SGP for the group, if there are 20 or more students with SGPs in the group.

Please refer to my previous blog post on Examining MCAS Growth Data by Classroom for more thoughts about the opportunity to engage staff in discussions about growth data now as we move toward the new educator evaluation system.

I shared this process with a district colleague earlier in the week, and she sent me an email the next day saying, "I think I love you!!!!...Our secretary was able to get 3 years instructional groups for me with state IDs. The reports are soooooo easy to do and save!!! You saved me tons of time. I am very excited." So I hope there are other readers out there who will have the same reaction.

Happy Holidays to all!

Southeast Data Coach Network

The Southeast DSAC Data Coach Network is an opportunity for school-based data coaches to participate in a regional professional learning community focused on building structures of collaborative inquiry. Network sessions will focus on sharing best practices and building upon skills and knowledge gained in the Data Coaching course from Research for Better Teaching.

The Network is intended for participants from Level 3 districts and "legacy" Level 3 districts who have taken the Data Coaching course from RBT. These districts were provided with a previous opportunity to register, and we have determined that there are openings for a few additional participants. The first meeting is scheduled for this Friday, December 2, from 12:30 pm to 3:00 pm in Plymouth, MA.

The highlighted topic for this Friday's meeting will be "Using the Data-Driven Dialogue Process to Explore Student Growth Data." The RBT course highlights a process of drilling down to strand/item/student work, but did not touch on exploring Student Growth Data. This measure will be a piece of the new Educator Evaluation Framework from ESE (where applicable) so reflective conversations about this measure now will certainly benefit schools in the long term. We will also take some time to explore selected resources from the Data Coach materials that we did not spend time on during the course this summer and plan for future network meetings.

I apologize for the last-minute announcement of openings, but would be happy to hear from additional district or school-based data coaches (or those that serve in a data coach role) that would be interested in participating. You may register at this link: https://docs.google.com/a/dsacma.org/spreadsheet/viewform?hl=en_US&formkey=dGs4TDZWOVhDckprQ3JhQTRDQXZqSWc6MA#gid=0

If you or others in your district have taken the RBT Data Coaching course, but are not able to attend this first meeting, please don't hesitate to contact me to share your interest. (laura.tilton@dsacma.org)

MassCUE Conference Happenings

Wednesday I attended Day 1 of the 2011 MassCUE conference at Gillette Stadium, and it was exciting as usual! Lots of great presenters, vendors, and opportunities to share ideas with colleagues made for a productive and engaging day.

The Spotlight Session entitled, "MA New Educator Evaluation Regulations: Opportunity and Challenges" presented by Karla Brooks Baehr provided an opportunity to learn more about the details in the developing story of the new educator evaluation regulations (with specific discussions about positions such as technology integration specialists that work with both staff and students). Two major takeaways from this session were that (1) both teachers and administrators should be taking time to become VERY familiar with the state's growth model, since this is one of the measures that will be used (where applicable) for educator evaluation; and (2) the Commonwealth's Model System is on target to be available for review on January 10, 2012. A variety of resources, including a version of the PowerPoint that was presented at the conference, are available on the ESE's Educator Evaluation web page.

Associate Commissioner Bob Bickerton and Director of Information Services Rob Curtin presented "Next Generation Supports for Teaching, Learning, and Student Success" which highlighted a variety of initiatives within ESE, including the new Curriculum Frameworks, PARCC, and many RTTT projects, especially the Teaching and Learning System which is currently under discussion. The ESE Showcase area provided an opportunity for conference attendees to learn more about initatives including the Education Data Warehouse, Thinkfinity Partnership, PBS LearningMedia Partnership, and the Teaching and Learning System.

The final session of the day entitled, "The Paperless Classroom: Tips and Tricks for the 21st Century Educator" was fascinating and introduced attendees to a wide variety of Web 2.0 tools I had not seen before. While these tools are not data-specific, I'd like to share some them here for those of you with an interest in staying current with educational technology. The presenter, Melissa Panchuck from the Fessenden School, was incredibly passionate and engaging about her use of Web 2.0 tools with her 5th grade students, including:

• Glogster - virtual posters
• Google Sketchup -
• Pixton - online comics
• Blabberize - talking pictures
• Go!Animate and XtraNormal - animations
• Screenr - instant screencasts
• diigo - link collections and highlighting
• issuu - digital publishing

 So many new and exciting tools (both inside and outside the classroom!) that can be used to improve teaching and learning!

Data Update from the Curriculum Summit

This week, I had the opportunity to attend the state's Third Annual Summit on Curriculum, Instruction, and Assessment in Marlborough, so I thought I'd share what I heard and learned here on the blog.

Commissioner Mitchell Chester kicked off the Summit with an overview of successes and challenges in Massachusetts' public schools. He spoke with excitement about the opportunities that lie ahead for us in terms of funding (RTTT and other sources), next-generation assessments (the multi-state PARCC group), and new ways of looking at student performance (the growth model).

I attended a morning session focused on the Massachusetts System of Tiered Instruction - "a data driven prevention, early detection, and support system with the aim of providing high-quality core educational experiences for all students and targeted interventions to struggling students who experience learning and/or behavioral challenges." We heard from two school districts (Pittsfield and Winchendon) who have implemented this model for math and literacy, respectively. Even in these "best-practice" examples, the ongoing challenges related to data were evident - WHAT data to examine, HOW to utilize it (and by WHOM), and WHERE to keep it. But the more we can share and collaborate on these challenges, the closer we can come to addressing them.

Our lunch speaker was Julia Phelps, Executive Director for Curriculum and Instruction at the Department of Elementary and Secondary Education. She spoke about the new Common Core State Standards and engaged the audience in thinking about the process for transitioning to these new standards over the next few years. In terms of changes to MCAS, her main points were that the spring 2011 MCAS will be based on the existing Frameworks, and the spring 2012 MCAS will be based primarily on the intersection between the existing Frameworks and the new Common Core. It is clear that MUCH more information will be coming out to support districts during this transition, so stay tuned.

The final session I attended was Creating Formative Assessments for Mathematics using Released MCAS Items which reminded us of the wealth of test questions available through ESE, and provided some ideas for using these as a "dipstick" for measuring student understanding.

Please note that the PowerPoint presentations and other materials from the Summit will be posted on the ESE's Summit Web page in the upcoming weeks.

More Growth Resources from the State

As you may know, there is now a new Data Warehouse course which focuses on the Growth Model (DW203 - Using MCAS Student Growth Percentiles). As with the other DW courses, all the materials can be downloaded from the "Training Materials" link on the Welcome page of the Data Warehouse.

As part of the development of this course, there are several brand-new video tutorials which help stakeholders understand and interpret some of the new growth reports. These videos may be useful for you to use with stakeholders (administrators, teachers, school committees, parents, etc.) as you share information about the state's new growth model.

These tutorials are different from the "growth model basics" and "parent report" video tutorials that I have posted about previously; both the new and older video tutorials can be downloaded from the main Growth Model page on the ESE Web site. Scroll to the bottom of the Growth page for both the Flash videos and static Word documents containing the information for the following:

• Introduction to Student Growth Percentiles
  
• Using MCAS Growth Reports: Scatter Plots
  
• Using MCAS Growth Reports: Stacked Bar Charts
  

As a side note, I will be teaching the DW203 half-day course through the Bi-County Collaborative at Norton Middle School on Friday, October 29 from 9-12. Spots are still available (no cost); please contact Joni Larsen at jlarsen@bicounty.org.

October Workshop Spots Available

The following workshops are being offered at no cost through the Cape Cod Collaborative (taught by Laura Tilton, Data Specialist, Southeast DSAC). Spots are still available for all three workshops, which will be held at Bourne High School. You may register online; for questions please contact Sheila Murray at s.murray.ccc@comcast.net.

Thursday, October 14, 8:30-2:30, Data Warehouse Basics and More
This workshop will combine the essential elements of the DW101 and DW201 courses from the ESE, including the cycle of inquiry and use of the pre-defined reports. The new Student Growth Model and related reports will also be explored. While sample data will be used for the introductory activities, participants will have the opportunity to begin to investigate data for their own schools/districts.

Thursday, October 21, 8:30-11:30, Using Student Growth Percentiles (DW203)
This NEW course for district/school leaders and teachers will present an overview of the Growth Model and focus on the use of MCAS growth data and reports. In the course, participants will learn about growth data, identify district and school uses for growth data, and apply the inquiry process through hands-on activities using MCAS growth reports. Requirement: DW101, Introduction to the Data Warehouse. (Note: October 14 participants will have already had much of this material.)

Thursday, October 21, 12:00-3:00, Multi-Dimensional Analysis (DW202)
Participants will learn how to access and use the Cognos® PowerPlay tool to engage in multidimensional analysis through cubes, pre-defined data tables that make on-the-fly data manipulation possible. Participants in this workshop will use training accounts containing sample data but will be provided an opportunity to work with their own district's data as well. Requirement: DW101, Introduction to the Data Warehouse

Data-Related Updates from ESE

MCAS Release Schedule Update:
The MCAS and AYP Data Review and Release schedule has recently changed, with public release of data (in the School and District Profiles area on the ESE Web site) scheduled for next Tuesday, September 14. Updated reports in the Data Warehouse are scheduled to be available on September 16. The statewide MCAS summary report (overview) for 2010 is now available online.

New Growth Model Tutorial:
As you know, this year's MCAS parent reports will now include growth and performance information for students, schools, and districts. Since the growth model is a new concept, the state has provided a tutorial specifically geared towards parents. Additional resources to help learn more about the growth model are available on the Growth page on the ESE Web site.

Sharing Information about the Growth Model

Massachusetts has developed a statistically valid method of measuring growth in student, group, school, and district performance over time. The growth model complements the MCAS year-by-year test scores, since it reports change over time rather than grade-level performance results in any one year.

Many folks have asked what resources are available in order to help educate staff, parents, and others on this new way of reporting on student achievement. The folks at ESE have provided a letter to educators from the Commissioner which provides additional details about the growth model. A similar letter for parents will be included in the student MCAS reports this fall.

There is an entire page on the ESE Web site devoted to the Growth Model, including a variety of documents and videos. Additional resources from the ESE will be posted on the Growth page as they are made available.

R-616 Test Item Analysis Roster: A Universally Useful Report

I'm often asked, "What reports do you think are most useful?" Of course, the answer depends on the question you are trying to answer with data!

However, one report that is extremely useful and efficient is the R-616 Test Item Analysis Roster report. This report provides a roster of students with information about performance level, scaled score, and the student's individual test item performance (including strand and question type) for a particular test. For 2010, this report also includes the SGP (student growth percentile) for each student.

Hyperlinks within the report allow you to click through to the test item iself as well as additional student detail. If you save the report as PDF, those links will remain in place (although a person would need to have data warehouse access to click through to the student detail reports.)

Note that while other 600-level (student) reports often result in a single page (or more) per student, this report is unique in that it provides efficient access to student data in a roster format.

Also note that even though the data warehouse doesn't provide reports at the classroom level, the roster of students for R-616 can be customized to reflect a single classroom of students. Simply use your mouse to select the students you'd like to include in the report (you'll see this in the second step of running the report) to customize the report for a classroom, intervention group, or other subgroup.

Spots available in Data Warehouse Refresher course 8/26 at READS Collaborative

Thursday, August 26
Data Warehouse Refresher
READS Collaborative (8:30-11:30), Middleborough
Registration Contact: Mary Keeler @ READS Collaborative, mkeeler@readscollab.org

This session is intended for staff who have had previous Data Warehouse training (or have explored it independently) but need additional support in the use of this tool. The workshop will provide a structured environment in which to explore data using the pre-defined reports in the Data Warehouse, including the new Growth Model reports. Participants will access their own district or school’s data, get time-saving tips, and will have an opportunity to run and save reports for use in planning for the 2010-2011 school year.

Summer Data Warehouse Workshops and Refresher Courses

The following Data Warehouse courses are being offered (at no or nominal cost) in the Southeast region over the summer, and seats are limited. All workshops will be taught by Laura Tilton, Southeast DSAC Data Specialist. Participants must have been granted access to the Data Warehouse via the DESE security portal prior to arrival at these workshops.

Tuesday, July 13
Data Warehouse Refresher
Duxbury High School (8:30-11:30 with optional afternoon support)
Registration Contact: Laura Tilton, laura.tilton@dsacma.org
This session is intended for staff who have had previous Data Warehouse training (or have explored it independently) but need additional support in the use of this tool. The workshop will provide a structured environment in which to explore data using the pre-defined reports in the Data Warehouse, including the new Growth Model reports. Participants will access their own district or school’s data, and will have time to run and save reports for use in planning for the 2010-2011 school year.

Wednesday, August 11
Data Warehouse Basics and More (8:00 am - 1:00 pm with snack break)
sponsored by the Bi-County Collaborative (Franklin, MA) location TBA
Registration Contact: Joni Larsen @ Bi-County Collaborative, jlarsen@bicounty.org
This workshop will combine the essential elements of the DW101 and DW201 courses from the ESE, including the cycle of inquiry and use of the pre-defined reports. The new Student Growth Model reports will also be explored. While sample data will be used for the introductory activities, participants will have the opportunity to begin to investigate data for their own schools/districts.

Monday, August 16
Data Warehouse Refresher (see description above)
Bourne High School (8:30-11:30)
Registration Contact: Sheila Murray @ Cape Cod Collaborative, s.murray.ccc@comcast.net

Thursday, August 26
Data Warehouse Refresher (see description above)
READS Collaborative (8:30-11:30), Middleborough
Registration Contact: Mary Keeler @ READS Collaborative, mkeeler@readscollab.org

New Data Warehouse Report Summary

I wanted to share a document I put together (Data Warehouse Report Summary.pdf, 4 pages, updated 10/20/2010) which summarizes all the reports that are currently available in the Data Warehouse. It includes all the reports that are currently available, including the new growth model reports, and briefly describes the uses for each report.

It also indicates which pieces of information are also available in the Profiles area of the ESE Web site (so that you don't have to log into the Data Warehouse when it's not necessary.) Please note that the Profiles area will not be updated with 2010 data until this fall.

The ESE's Glossary of Predefined Reports (17 pages) is also a useful document, and contains much of the same information, including screen shots of each report. However, at this time, this document does not include information about the growth model reports and several other new reports (and it's quite lengthy).

In summary, there are a lot of different reports and it is easy to fall into the "analysis paralysis" trap. The goal is that these documents will help you structure your inquiry within the Data Warehouse, in order to save time and frustration. We expect some of the 2010 data reports to be available in the Data Warehouse as early as Thursday, June 24!

MCAS Reporting Information, Spring 2010

For those who were not able to participate in the MCAS Reporting teleconference yesterday, I thought I'd post the exciting highlights.

• Preliminary student results for ELA (including scaled scores and performance levels) as well as Math (multiple choice results only) are NOW available through DropBox Central in the Security Portal. Reports will begin to be available in the Data Warehouse on June 24, with ELA growth reports available on June 25.
• Preliminary math and STE data will be available in early August (DropBox), with Data Warehouse reports available starting August 13 (August 17 for growth data)
• Official MCAS results will be accessible via DropBox by districts on September 8, schools on September 9, with public release of data following shortly thereafter.
• The ELA long composition data and MEPA reports will continue to be posted at the MCAS Service Center for now.
• The folks at the ESE would like to know ASAP if you are missing any data, or if there are major issues with the data you have received. The electronic discrepancy reporting period (through www.mcasservicecenter.com) will be from August 6 through August 12.
• Please note that students may appear twice in your data file, if there are SASID issues, but these will be resolved over the summer and do not need to be reported at this time.
• This year, scaled scores will be reported for Grade 3 tests.
• This year, some students who took MCAS-Alt and scored at the "Progressing" level will receive 75 CPI points rather than 100, due to changes in federal guidelines.
• This year's parent reports will include information about student growth, which will likely highlight longitudinal (3 year) student data along with growth numbers for the student, the school, and the district (for both ELA and Math). More information/guidance will be distributed to districts prior to the release of the data in September.

Please remember that there is an embargo in place for the preliminary data. While it may be used for instructional planning, it may not be discussed with the public or press.