Wednesday, September 1, 2010

Education Data Warehouse Alert

The following message was posted in the security portal for Data Warehouse users, but I thought I'd share it here for those that had not logged in for a few days. More details on this topic were also included in the August 31 "On the Desktop" memo from Commissioner Chester.

Certain reports in the Education Data Warehouse (EDW) are running very slowly during periods of high use. We apologize for this inconvenience. Additional options for getting the reports you need as well as a brief explanation follow.

Additional options to obtain reports:

Drop Box Central: Currently includes preliminary rosters and reports. Will add detailed corrected students rosters with MCAS results (including Student Growth Percentiles) in PDF, DAT and EXCEL formats on September 9th.

Profiles: Will be updated on September 9th to include most aggregate reports that would otherwise be available in the EDW.

Brief Explanation:
While we remain excited about the capabilities of the EDW we purchased, it is clear that some components were not adequately designed for a statewide implementation; e.g., when several users simultaneously request the more complicated reports, the system slows down and becomes unresponsive. ESE submitted a Longitudinal Data Systems grant to the U.S. Department of Education last November in order to re-engineer these aspects of the EDW—preferably before the start of the 2010/2011 school year. The “good news” is that we won a grant in the amount of $13M with the major portion of these funds targeted to just this purpose. Unfortunately, the funding was not made available to ESE until late last week (8/26). Improvements to the EDW will be made in stages, with some available in weeks and others over the coming months.

If you need to run a custom query in the EDW during peak school hours we recommend running using the scheduling tool to run the reports during off-peak hours.


Laura's Note: Please follow this link for a recent related blog post entitled "Save Time! Running Reports in the Background".