Powerschool SIS - Hosting Performance Degradation
Incident Report for PowerSchool Products
Resolved
The PowerSchool teams made improvements to the PowerSchool Network infrastructure to ensure a successful PowerSchool experience. This included building out additional network infrastructure to split the volume of traffic we see from the East Coast of the US. Today, we confirmed with districts, internal testing, and monitoring tools that the performance degradation experienced from 11/29/21 – 12/3/21 is now resolved.

If you are still experiencing issues, please open a new Support case so our teams can troubleshoot the specific problem.
Posted Dec 06, 2021 - 15:04 PST
Update
We have returned the PowerSchool Mobile App to full functionality.

We have not seen the delays this morning, and we are continuing to monitor performance across our environments.
Posted Dec 06, 2021 - 06:20 PST
Update
The team is continuing to monitor the performance of our US East Region Customers. To improve site stability, we are temporarily limiting the following features in this region:
- Access to new sign-ins for the PowerSchool Mobile App.
- The ability to refresh information within the PowerSchool Mobile App.

Note: Push Notification from your SIS to a user already signed into the PowerSchool Mobile App should continue to flow. Access to the parent and student web pages continues to be accessible.

We apologize for the inconvenience and will restore full functionality as soon as possible.

Note these limitations will only be seen by customers in the following US States: Arkansas, Connecticut, Washington DC, Florida, Georgia, Iowa, Idaho, Illinois, Indiana, Kansas, Kentucky, Louisiana, Massachusetts, Maryland, Maine, Michigan, Minnesota, Missouri, Mississippi, North Dakota, Nebraska, New Hampshire, New Jersey, New York, Ohio, Oklahoma, Pennsylvania, Rhode Island, South Carolina, South Dakota, Tennessee, Texas, Virginia, Vermont, Wisconsin, West Virginia.
Posted Dec 06, 2021 - 05:42 PST
Monitoring
We performed emergency maintenance last night on the existing setup to address the delays to provide relief. Last night after that maintenance, we continued to be able to reproduce the delays seen during our morning peak traffic times. So, we prepared other options to mitigate the disruption this morning. Even with the additional changes within the current setup and attempts to reroute the traffic we saw minimal relief.

While working on the changes to the current setup, we have also been working on more significant changes. We are implementing changes to split a substantial portion of our east coast traffic while also addressing the existing path where we see delays in our infrastructure. We expect to complete this split over the weekend to have better performance for Monday morning.
Posted Dec 03, 2021 - 12:11 PST
Identified
Our Team Monitoring the situation has seen some continued delays for some users. We are actively working to make additional corrections in our environment presently. We will continue to provide updates on this statuspage here.

The issues we have seen here are solely related to networking infrastructure and connections to your applications hosted on our servers and not with the servers themselves. Our PowerSchool application servers and their data have been operating as expected.

This performance degradation is also only impacting customers in one of our service regions which include the following US states: Arkansas, Connecticut, Washington DC, Florida, Georgia, Iowa, Idaho, Illinois, Indiana, Kansas, Kentucky, Louisiana, Massachusetts, Maryland, Maine, Michigan, Minnesota, Missouri, Mississippi, North Dakota, Nebraska, New Hampshire, New Jersey, New York, Ohio, Oklahoma, Pennsylvania, Rhode Island, South Carolina, South Dakota, Tennessee, Texas, Virginia, Vermont, Wisconsin, West Virginia.
Posted Dec 03, 2021 - 05:54 PST
Update
Last night we put in place additional diagnostic tools that allowed us to identify intermittent issues with one of our network appliances during peak load periods at the start of the day. This evening, we are taking emergency maintenance to resolve the problems seen with that network appliance. We are also working on other opportunities identified to improve performance and increase our resiliency. We are also working to synthetically recreate the peak load conditions in off-hours through the weekend to validate changes and continue to make improvements.

The issues we have seen here are solely related to networking infrastructure and connections to your applications hosted on our servers and not with the servers themselves. Our PowerSchool application servers and their data have been operating as expected.

This performance degradation is also only impacting customers in one of our service regions which include the following US States: Arkansas, Connecticut, Washington DC, Florida, Georgia, Iowa, Idaho, Illinois, Indiana, Kansas, Kentucky, Louisiana, Massachusetts, Maryland, Maine, Michigan, Minnesota, Missouri, Mississippi, North Dakota, Nebraska, New Hampshire, New Jersey, New York, Ohio, Oklahoma, Pennsylvania, Rhode Island, South Carolina, South Dakota, Tennessee, Texas, Virginia, Vermont, Wisconsin, West Virginia.
Posted Dec 02, 2021 - 12:06 PST
Update
We are aware of a service disruption/degraded performance with Powerschool SIS November 29, 30 & December 1st, 2nd. We are working to assess the situation and will provide you with regular updates as they become available. We appreciate your patience while we work to resolve this issue.
Posted Dec 02, 2021 - 07:23 PST
Update
We are aware of a service disruption/degraded performance with Powerschool SIS November 29, 30 & December 1st. We are working to assess the situation and will provide you with regular updates as they become available. We appreciate your patience while we work to resolve this issue.
Posted Dec 01, 2021 - 06:08 PST
Update
We are continuing to monitor/investigate this issue as additional reports of improved performance continue. Please reach out to PowerSchool Support via phone, web, or chat through our PowerSchool Community if you continue to experience any issues.
Posted Nov 30, 2021 - 15:56 PST
Monitoring
While we continue to monitor/investigate this issue. We are getting positive reports that the Performance is improving. Please reach out to PowerSchool Support via phone, web, or chat through our PowerSchool Community if you continue to experience any issues.
Posted Nov 30, 2021 - 09:52 PST
Update
We are aware of a service disruption/degraded performance with Powerschool SIS November 29 & 30th. We are working to assess the situation and will provide you with regular updates as they become available. We appreciate your patience while we work to resolve this issue.
Posted Nov 30, 2021 - 06:15 PST
Update
The issue appears resolved, but we are still investigating to ensure long term stability. Please continue to check this status page for ongoing updates.
Posted Nov 29, 2021 - 17:15 PST
Update
While we continue to monitor this issue, we are getting positive reports that the Performance is not longer an issue. Please reach out to PowerSchool Support via phone, web, or chat through our PowerSchool Community if you continue to experience any issues.
Posted Nov 29, 2021 - 12:44 PST
Update
We are aware of a service disruption/degraded performance with Powerschool SIS.  We continue to assess the situation and will provide you with regular updates as they become available. We appreciate your patience while we work to resolve this issue.
Posted Nov 29, 2021 - 08:02 PST
Investigating
We are aware of a service disruption/degraded performance with Powerschool SIS. We are working to assess the situation and will provide you with regular updates as they become available. We appreciate your patience while we work to resolve this issue.
Posted Nov 29, 2021 - 05:59 PST
This incident affected: PowerSchool SIS.