PARCC Field Tests and the MCAS

As you may know, many questions have surfaced recently regarding the PARCC Field Test and the Spring 2014 MCAS. ESE has heard these questions and has put together resources related to field-test logistics, including a timeline and information related to future accountability determinations.

Additional details are contained within a recent memo to superintendents at http://www.doe.mass.edu/news/news.aspx?id=7849. Please note that the conference call registration and exemption surveys links in the memo are intended for superintendent use only. Other leaders who are not superintendents should coordinate with their superintendent as ESE has requested one phone line per district for the conference calls.

June 2013 Release of Preliminary Results

As many of you already know, preliminary MCAS results were made available to authorized users in DropBox Central (in the security portal) on Tuesday. This release includes full preliminary ELA results and preliminary multiple-choice results for grades 3–8 and 10 Math and grades 5 and 8 Science.

There is an interpretive guide to accompany this release (currently available in DropBox Central, to be posted to the 2013 MCAS Web page shortly) which provides information about various ways that the preliminary data (along with the 2013 released items) can be reviewed and utilized. The raw-to-scaled score conversion tables for ELA have also been posted.

In addition, the latest Student Assessment Update notes that preliminary MCAS-Alt results have been made available through DropBox Central, and ELA long compositions are posted online at www.mcasservicecenter.com.

On Friday, June 21, the same preliminary results will be made available through selected reports in Edwin Analytics.

Per the Interpretive Guide: "All data released prior to the official release of school and district results in mid-September are embargoed, which means the data cannot be released to the public or discussed in public meetings.... Preliminary MCAS data, including the Mathematics and STE multiple-choice results, can and should be used for educational planning purposes. Whenever preliminary results are printed for planning purposes, they should be clearly dated and labeled “preliminary,” with the understanding that some results may change with the official release in September."

As a reminder, I will be offering three regional Edwin Analytics workshops in August after the remainder of the preliminary data has been released, but in the meantime, please do not hesitate to contact me with questions.

Release of Spring 2013 MCAS Items

The Massachusetts Department of Elementary and Secondary Education has posted the released items for the Spring 2013 tests by grade and content area at: http://www.doe.mass.edu/mcas/2013/release/default.html.

MCAS Participation and the Need for Data Quality

As schools gear up to administer the Math and Science MCAS exams next month, I thought it would be a good opportunity for a reminder of the importance of data quality, particularly when it comes to MCAS participation data.

As you may know, one way that a school can be classified as “Level 3” is by having less than 90% MCAS participation for the aggregate or any subgroup. (Schools can similarly be classified as “Level 2” by having less than 95% MCAS participation for the aggregate or any subgroup; see page 5 of the School Leaders’ Guide for additional information.)

Last year, 32 schools (27 of which were high schools) were identified for low MCAS participation statewide - 9 in the "very low / <90%" (Level 3) category and 23 in the "low / <95%" (Level 2) category. In some cases, the classification of a single student brought the participation percentage of a subgroup below 90%, thus designating the school as Level 3. In addition, several schools were preliminarily identified as Level 3 for low participation, but after these schools looked further at their data, they identified errors they had initially missed which brought their participation above the 90% mark. School-wide participation in achievement assessments is important, and it's essential that every student is counted correctly. It’s clear that quality data collection, roster verification, and proper  classification can prevent confusion and misidentification.

Additional guidance related to MCAS participation requirements, including information specific to high school students, can be found in Part II: Student Participation in the the Principal’s Administration Manual. Appendix D of this document also provides information about reporting various types of student non-participation, and how this information is used for MCAS reporting and accountability reporting.

For schools interested in looking back at their participation history (for both aggregate and subgroups), this data can be found in two places on the Profiles pages for that school. On the "Assessment" tab for each year, there is a report entitled "Participation Report." On the "Accountability" tab, clicking "View Detailed 2012 Data" and scrolling to the bottom of the page will bring you to a table of participation data by subgroup.

2012 MCAS Data Now in the EDW Cube

The Education Data Warehouse "cubes" were updated this week to include 2012 data. (Cubes are predefined data tables in the EDW that make on-the-fly data manipulation possible.)

The cubes can be accessed from either the District tab or the School tab in the EDW, near the bottom on the left side of the page, under the header Data Analysis Cube. Clicking the link ESE Cubes will open a folder with both the cubes and supporting documentation (a handbook and an up-to-date FAQ). The MCAS Default View cube allows users to start from scratch to analyze data for a particular school or district. The folder Starting Points (Cube Views) provides cube starting points that have already been manipulated and saved to present the data in a particular way. The currently available cube views include:

• MCAS % Correct by Strand and Topic Trend
• MCAS CPI Trends by Grade Level
• MCAS Demographic Trends
• MCAS Performance by Grade Level
• MCAS Performance by Year
• MCAS Performance by Gender by Subject
• MCAS Special Education Performance Gap Trends
• MCAS Subject, Strand, Topic, Question Type, % Correct compared to State by Subgroup (SPED)
• MCAS Weighting by Strand and Topic Level

The State MCAS Default View cube is available for training purposes and contains only state-level data.

To save a cube view, click the Save As icon in the lower right corner and save it to the My Folders tab or your District Folder. (Don't worry, you can't accidentally overwrite the public cube views.) This is similar to saving Report Views. To convert the cube view to PDF for printing/saving outside of the EDW, click the File button in the toolbar at the bottom of the screen (looks like a page) and choose Export PDF. The File button will also allow you to export the data into Excel.

I will offer a regional workshop on using the Data Warehouse cubes if there is enough interest - please email me at laura.tilton@dsacma.org if you're interested in a workshop.

2012 MCAS and Accountability Data Available in Profiles

The 2012 MCAS and Accountability information for Massachusetts public schools and districts is now available through the School and District Profiles page on the ESE Web site. (http://profiles.doe.mass.edu)

The easiest way to find a school or district is by using the "Quick Search" box in the upper left corner of the Profiles page. After clicking the name of the desired school or district, the "Assessment" tab will provide information about MCAS performance and growth for 2012. The "Accountability" tab will provide information about the school/district accountability/assistance level and the data behind the PPI calculations.

For detailed information about the calculation of PPI and the new accountability system, please refer to the About the Data page linked to the accountability reports as well as the Interpretive Materials, including the School Leader's Guide to the 2012 Accountability Determinations. Of particular value in the Guide are the Level explanations on page 5, and the PPI breakdown on page 10.

I am more than happy to assist you in finding answers to questions about the new data so please don't hesitate to contact me at laura.tilton@dsacma.org.

MCAS Data Investigation Activity Ideas

Last week I posted information about an activity focused on the exploration of classroom growth data through the paper-and-pencil creation of a scatter plot of growth and scaled score for a given instructional group. One colleague asked me, "I wonder if people will be able to see how cool the activity is just by reading about it...or maybe it's only people like us who get excited by scatterplots??"

So, I'm posting this follow-up because I bet there are folks out there who missed the post last week, but I know you'll agree that the activity is worth investigating! Hope you'll check it out once you come up for air during this busy time of year. I also wanted to link back to some previous years' blog posts with additional MCAS data investigation activities...some ideas for digging in to the data during fall staff meetings or PLC's!

• Exploring Classroom Growth Data (August 2012)
• MCAS Item Analysis and the Standards for Mathematical Practice (August 2011)
• Engaging Staff in MCAS Data Exploration (September 2010)

Happy first days of school!

2012 MCAS and Accountability Data Release

The recent Commissioner's Update contained an announcement regarding the release of the 2012 MCAS and accountability data, including a release schedule, MCAS discrepancy reporting information, review of provisional and preliminary accountability data, EDW updates, and reporting teleconferences on August 6 and 7.

The full announcement is online here: http://www.doe.mass.edu/news/news.aspx?id=6928.

Preliminary MCAS 2012 in EDW 3.0

The following EDW 3.0 reports are now available with preliminary MCAS 2012 data:

District-Level Reports

• MCAS District Results by Subgroup (PE304)
• MCAS District Results by Strand and Topic (CU306)
• MCAS District Test Item Analysis Summary (IT301)
• MCAS District Test Item Analysis Graph (IT302)

School-Level Reports

• MCAS School Results by Subgroup (PE404)
• MCAS Detailed School Performance (PE408) and its drillthrough
• MCAS School Results by Strand and Topic (CU406)
• MCAS School Test Item Analysis Summary (IT401)
• MCAS School Test Item Analysis Graph (IT402) 

Student-Level Reports

• Assessment Detail (Individual Student)(PE606)
• Assessment History Summary (Individual Student)(PE613)
• MCAS Student Item Analysis Roster (IT616)
• MCAS Student Roster (All Subjects)( PE617)
• MCAS Student Growth Scatter Plot (GR601) (ELA only)
• MCAS Student Growth Roster (GR602) (ELA only)

Please remember that there is an embargo in place for preliminary MCAS data. While this data may be used for instructional planning, it may not be discussed with the public or press.

See the bottom of the “EDW Home” tab within the EDW for additional features and functionality that have been added to the EDW 3.0. The "EDW Home" tab can be added as the default home page in your EDW account by following the steps outlined in the May 17 blog post.

Upcoming MCAS and MEPA Data Releases

A previous blog post provided some of the preliminary release dates for MCAS results, but I wanted to provide links to two ESE memos containing additional details.

Preliminary 2012 MCAS Data:

• June 12th - ELA Composition images posted electronically for principals
• June 15th - Portfolio Feedback Forms for MCAS-Alt posted electronically for principals
• June 22nd - Excel data files and school roster reports with preliminary student results to Dropbox Central.
• June 29th - Preliminary school and district reports to Dropbox Central.
• June 29th - Selected EDW reports posted.

See the Commissioner's May 21st memo for more details.

Spring 2012 MEPA Data:

• June 7th - Parent/Guardian reports received by schools via UPS.
• June 7th - Various data reports posted to http://www.mcasservicecenter.com/.
• June 25th - Student data file with 2011 and 2012 MEPA and MCAS scores will be posted at the Department's Security Portal.

More details on the MEPA release can be found here.

2012 Preliminary MCAS Release and EDW Updates

Yesterday's regularly scheduled Webinar on the Education Data Warehouse shared an excellent new feature - an EDW "Home Page" which will keep us updated about new features, data releases, and additional resources.This home page is now available, but does not currently appear automatically as a tab at the top of the screen after you log in. You can follow the steps below to add this tab as the default so that you will see the latest information when logging into EDW 3.0.

To add the EDW Home Page tab:

• In the upper left corner of the screen, under the “Data Warehouse” banner, click the “Tab Menu” button, and choose “Add Tabs.”
• Under “Available Entries,” navigate to Cognos > Public Folders > Pages.
• Click the checkbox next to “EDW Home” and then click the green arrow in the middle of the page so that “EDW Home” moves from “Available Entries” to “Selected Entries.”
• Click “OK” at the bottom of the page.

To make the EDW Home Page your default tab:

• In the upper left corner of the screen, under the “Data Warehouse” banner, click the “Tab Menu” button, and choose “Modify the Sequence of Tabs.”
• Select “EDW Home” and then click “To top.”
• Click “OK.”

One very important piece of information that was provided during the Webinar is that the preliminary 2012 MCAS data is expected to be released via the Dropbox on June 22, and then in the EDW on June 29.


The EDW Webinars are held on the first Wednesday of the month at 11 am and the third Wednesday of the month at 2 pm. Please email datawarehouse@doe.mass.edu if you would like to be placed on the notification/reminder list for these calls.

DART Detail: English Language Learners

The ESE is has introduced DART Detail: English Language Learners, the second of a series of dynamic Excel-based data investigation tools. This ELL-specific tool includes data such as enrollment, primary languages, countries of origin, MEPA performance, achievement gap information, MCAS performance, student indicators, staffing data, and Title III allocations. (Note: this version of the tool has been updated since the pilot release of DART Detail: ELLs in December and a User's Guide is now available.)

Re-release of Grade 10 MCAS Math Results

On Friday, September 30, the Commissioner’s office at DESE posted a press release regarding the re-release of 2011 grade 10 MCAS math scores. Nearly 3,300 students statewide who took the Spring 2011 grade 10 MCAS Math test and passed have earned higher performance levels than initially indicated.

School and district results in grade 10 Math have been updated in the Profiles section of the ESE Web site. The new data is scheduled to be available in the Education Data Warehouse no later than Thursday, October 6, 2011.  
 
Press Release: http://www.doe.mass.edu/news/news.aspx?id=6421

Public 2011 MCAS Data Available

The 2011 MCAS data for all Massachusetts public schools/districts is now available in several places:

The Education Data Warehouse (in the Security Portal; https://www4.doemass.org/)

School and District Profiles (http://profiles.doe.mass.edu/)

• From the right hand side of the main Profiles page, you can access a variety of statewide reports which can be exported into Excel for further analysis and investigation.
• The easiest way to find school/district information in Profiles: use the Search box to locate a school or district's profile. The "Assessment" tab in a school or district's Profile will allow you to access a variety of reports, including annual comparisons, growth, and subgroup information.

Adequate Yearly Progress Information (http://www.doe.mass.edu/sda/ayp/2011/default.html)

• Under "Lists and Analyses" there is a document entitled, "2011 Lists of Massachusetts Schools and Districts by NCLB Accountability Status and Accountability and Assistance Level." This document provides a summary of 2011 accountability data, including lists of districts and schools by Accountability and Assistance Level and schools commended for narrowing proficiency gaps, high growth, and/or exiting NCLB Accountability Status from 2010.

I'm always interested in knowing different ways in which people are using - or would like to use - this data! If you have ideas for data displays or different ways this data could be analyzed, please don't hesitate to contact me with your ideas. I may be able to help you out with an Excel tool that could be used by others as well.

2011 MCAS Results - Statewide Overview

The ESE published a press release today with an overview of the statewide results of the 2011 MCAS exams. Highlights: The percentage of 5th graders scoring Proficient or higher in ELA and math increased by 4 points since last year, and 10th graders improved their ELA performance by six percentage points a year after a slight decline.

Press Release: Gains Made by 5th & 10th Graders Highlight 2011 MCAS Results

Full Statewide Report: Summary of 2011 MCAS State Results (PDF)

Public release of school and district MCAS, AYP, School and District Levels, and Commendations schools is anticipated to take place the week of September 19.

Correction on "High Needs" Subgroup Post

Yesterday's post summarized the definition of the new "high needs" subgroup and how it is being used. One small correction needs to be made to the content of the post as follows.

The information is correct except for the last sentence which describes two ways the subgroup information is being used.  The second item, "measuring progress for districts who are working to exit Level 4 status” should read “measuring progress for schools who are working to exit Level 4 status."

Apologies for any confusion!

"High Needs" Subgroup - What Is It?

The 2011 MCAS reports include a new subgroup designation called "High Needs." The Data Warehouse FAQ describes this group as "a new subgroup category that includes all students in one or more of the subgroups: English language learner (ELL)/former English language learner students, low income students, or students with disabilities."

A number of people have asked for more information about this subgroup - in particular, if/how it is being used for accountability/assistance determination.

The simple answer from the accountability group at ESE is that this "new" subgroup is NOT being used to determine AYP or state accountability/assistance. (The individual subgroups that make up High Needs, of course, are a continuing piece of this calculation.) The High Needs subgroup is currently being used in two ways: (1) determining schools commended for narrowing proficiency gaps, and (2) measuring progress for schools who are working to exit Level 4 status.

post edited 9/8/2011

A New Spin on Item Analysis

I'm guessing that most readers of this blog have taken part in traditional "item analysis" activities using MCAS item reports and a packet of released test items. The Standards for Mathematical Practice (page 15 in the new Massachusetts Curriculum Frameworks for Mathematics document) provide an opportunity make connections with these process standards (such as, "Attend to precision" or "Look for and make use of structure") to this traditional type of data analysis.

Our regional Math Specialist, Madeline Roy, and I have documented some ideas for an activity (piloted this summer) to engage staff in exploring MCAS test item data while making connections to the Standards for Mathematical Practice. More information can be found in Facilitator's Notes and Recording Sheet documents which we have put together for your use. As always, we look forward to your feedback!

In addition, I'm sharing a link back to a post from fall 2010 entitled "Engaging Staff in MCAS Data Exploration" with additional ideas for sharing MCAS data with staff.

Best wishes to you all for a smooth start to the school year!

2011 MCAS and Accountability Data

DESE recently posted important information regarding 201l MCAS preliminary data and accountability data, and for discrepancy reporting. Please see the Commissioner's Memo and printable Release Schedule for complete details. Some upcoming dates are listed below:

August 8: District/school leaders can begin to access full preliminary (embargoed) 2011 MCAS results through the Security Portal (first in DropBox, then in EDW)

August 9: District/school leaders can access to preliminary (embargoed) school and district accountability data through the Security Portal.

August 8 (2:30 pm) OR August 9 (10:00 am): MCAS and Accountability Reporting Teleconference (register at www.mcasservicecenter.com)

The Commissioner's Memo also mentions the updates and improvements that will take place in the Education Data Warehouse later in August. I'll certainly be sharing additional information as it becomes available!

It's that time of year!

As the school year winds down, it's also a busy time of year data-wise! The MEPA data were released earlier this month, and the MCAS raw data files and roster files (embargoed!) were placed in DropBoxes at the end of last week. The end of this week will bring new preliminary data in the Data Warehouse, the MEPA-MCAS crosswalks in DropBoxes. If you missed the webinar last week on MCAS Reporting, the PowerPoint presentation can be found in the MCAS Reporting section of the MCAS Service Center.

Last week's conference call on EDW improvements provided of an insight into the improvements being made to the system. (PowerPoint to be posted soon on the ESE Web site.) Some features we can look forward to include:

• Faster report generation due to back-end improvements


• Interface enhancements including reports in a separate window, prompts directly on the report itself, and links to pre-run reports in DropBoxes


• MEPA and CD status data will be in EDW


• Faster access to appropriate prompts (for example, less scrolling to get to the district's name)


• On-screen previews of sample reports so users know what the report looks like before it is run

I don't have details about what features will be available when (I believe the goal is late summer), but I do know that we will see a few of the improvements in this week's EDW release. In addition, the new system and the old system will co-exist for a while as folks get used to the new system.

As a reminder, I am offering a Data Warehouse "Getting Started" workshop on June 29 and a "Refresher" course on August 22 (in Bourne). The refresher course will include any information about the "new" system that is available at that time. Spots are still available and you can sign up through the Cape Cod Collaborative.