The other day one of my customers mentioned that their SCOM Web Portal has been broken for a while now. As I like digging into web application issues I took a quick look. Here’s what I came up with. It seems that the portal itself was loading fine, but viewing All Alerts or Active Alerts showed a Service Unavailable (“HTTP Error 503: The service is unavailable”).
One of the things about IIS based errors is that in most cases the Event Log on the web server can help you a great deal. In the System Event Log I found the following:
A process serving application pool 'OperationsManagerMonitoringView' reported a failure trying to read configuration during startup. The process id was '6352'. Please check the Application Event Log for further event messages logged by the worker process on the specific error. The data field contains the error number.
Checking the IIS Management Console I could indeed see that the Application Pool was stopped. Starting it succeeded, but viewing the page made it crash again. Looking a bit further I found the following in the Application Event Log:
The worker process for application pool 'OperationsManagerMonitoringView' encountered an error 'Configuration file is not well-formed XML
' trying to read configuration data from file '\\?\C:\Windows\Microsoft.NET\Framework64\v2.0.50727\CONFIG\web.config', line number '14'. The data field contains the error code.
Now that seems pretty descriptive! Using notepad I checked the contents of the file and tried to see why the XML was not well-formed. I checked the XML tags and the closings and such but I couldn’t find anything at first sight. Looking a bit longer I saw that the quotes (“) were different from the other quotes in the file. Here’s a screenshot of the bad line and the fixed line. You can simply erase and retype the “ on that line and you should be good to go.
Personally I like taking a backup copy before I perform manual fixes. After saving the file I did an IISReset just to be sure. And after that we were able to successfully view our alerts through the Web Portal again!