Troubleshooting IIS Web Errors
IIS Errors And GlobalSearch
IIS Communicates error messages through web errors – some of which you might be familiar with (IE. 401, 404, etc) These are commonly seen when using the web client, but can be detected using the thick client with the use of a web debugger such as Fiddler or Wireshark.
400 Bad Request
These are normally SQL Errors. They can occur in other ways such as not being able to connect to the databases or even files not being brought correctly into GlobalSearch LAN
Most SQL errors can be narrowed down by going to the following site. This is known as “Browsing to REST”
http://(servername/IP/FQDN):port/square9rest/documents/db
Often times a 400 Error will come in the form of a DB connectivity error. If Smart Search crashes or hangs during the upgrade portion when launching, the error message will appear behind the launcher and the launcher would have to be moved in order to see it.
In these instances, you should try and browse to REST (above) and then ensure your SQL instance is on and not in recovery or single-user mode.
401 Unauthorized
GlobalSearch uses 401s regularly as part of it's security and authentication protocol. It's very common to see 401s accompanied by 200s in Fiddler. However, a 401 is not ALWAYS benign and can be an indication of some malfunctioning component. Checking the error log for 401 errors will help tease out the actual problems from the normal operating procedure of the application. These errors can be found in both the Smart Search error log as well as in the browser if you attempt to access a page from the restproxy or web services (GetSmart website). It indicates that the server refused the authentication attempt provided.
401 - Unauthorized: "Access is denied due to invalid credentials. You do not have permission to view this directory or page using the credentials that you supplied.
Causes and Solutions:
Causes | Solutions |
---|---|
SPN* issues. | SPN issues are name resolution issues. These can be caused when a server's name and IP address have issues resolving. An easy resolution here would be to change the configuration files to use machine name instead of IP address or vice versa. Or use the setspn command to update the spn settings. |
SSL bug in windows | There is a bug in windows when using SSL that may cause a 401.1 unauthorized. Follow the first method listed here to resolve the issue by adding the FQDN into the registry |
Bad Connection String | The user passed through a connection string does not have proper permissions to a database. If you're using a connection string with explicit security, ensure the user being passed has access to the database specified. |
*SPN = Server Principle Name
Please Be Advised
GlobalSearch uses 401 webcalls very frequently. They are one of the most common webcalls our application makes. It can be very easy to mistake a normal, healthy 401 for a 401 that's causing issues. Please consult with a Square 9 Tech if you think you're seeing 401 calls erroneously.
403 Forbidden
If a user is absolutely forbidden from a web resource or page, expect to see a 403. These can come from security issues with said resources, related to permissions set in the wwwroot file.
Explained more eloquently, directly from Wikipedia:
A web server may return an HTTP 403 Forbidden status in response to a request from a client for a web page or it may indicate that the server can be reached and process the request but refuses to take any further action. HTTP status code 403 responses are the result of the web server being configured to deny access to the requested resource by the client.
A common request that may result in a 403 Forbidden response is a HTTP GET request for a web page performed by a web browser to retrieve the page for display to a user in a browser window. The web server may return a 403 Forbidden status for other types of requests as well.
404 Not Found
Perhaps one of the most well-known errors across the entire internet, a 404 error occurs when a resource or web page is not at the target of an invocation (webcall). As opposed to a 403 or 401 error, which can indicate a properly functioning website that has been blocked off from a user for a particular reason, a 404 error is almost always accidental, whether it be a bad invocation or a missing file.
If you get a 404 error on a page in GlobalSearch, it likely means that there are files missing from the virtual directory of that website in IIS. The web console may give you more detailed information on exactly what components are missing.
405 Method Not Allowed
A 405 error points to a bad or disallowed HTTP Verb. HTTP Verbs describe how a Web API can interact with data pushed through a URL. Common HTTP verbs include:
GET | This method is used to retrieve data from a URI, and it is probably the most-used HTTP method. |
HEAD | This method is much like the GET method, except that it doesn’t actually retrieve the data from the request URI – it simply retrieves the HTTP status. |
POST | This method is typically used to send new data to the URI; POST is often used to submit form data. |
PUT | This method is typically used to raw data to the URI; PUT is often used to submit JSON or XML data to Web API applications. |
For the purposes of GlobalSearch, the most important thing to know is that our web application uses these verbs to function – any call that pulls or transmits data is using these HTTP verbs. The most common cause of 405s is when a client has the WebDAV publishing module installed on their IIS server. If this application is on the server, it should be removed. If for whatever reason, it can’t be removed, support should be contacted. 405s can also be a result of a firewall port being closed. Opening both incoming and outgoing ports for your application (default port 80) may remedy this issue.
500 Internal Server Error
Internal Server errors are one of the more varied and complex types of IIS Errors. These errors will not show any actual error-handling data on devices other than the server. 500 errors can contain all sorts of information and can be varied in appearance and nature. Some might be filled with code, while others will appear like a more standard IIS Error message. Depending on what has gone wrong, we may have to dive a little deeper into the issue with a closer look. An accompanying ProcMon diagnosis might yield more information. Due to the varied nature of these types of errors, there is absolutely no prescribed solution – though clearing the server’s .NET Temp files for all Square 9 related temporary files is a good first step. Sometimes, old configuration data will become cached in the .NET temp directory. Clearing these files is harmless and will only result in a slightly longer initial load time on the next bootup of the Server-side application.
Due to the varied nature of these types of errors, there is absolutely no prescribed solution – though clearing the server’s .NET Temp files for all Square 9 related temporary files is a good first step. Sometimes, old configuration data will become cached in the .NET temp directory. Clearing these files is harmless and will only result in a slightly longer initial load time on the next bootup of the Server-side application.
You can clear these files by going to the following directory:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files
AND (as of GC 2.0.1.160)
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files
You can safely clear out any directories with “Square9” or “Getsmart” in the title.
Doing this will require you to stop the GS/GC App pools and websites, either manually or by an IISRESET/Stop
503 Service Unavailable
A 503 error is returned if the server is “busy” or otherwise not accepting incoming connections on the requested URL. This can happen during server maintenance or in the event of an overwhelming amount of traffic. Service Unavailable, in terms of GlobalSearch, usually means a stopping of the App Pool (creating an unavailable state for the server to accept traffic towards authentication or traffic stemming from other applications related to GS). There can be a number of reasons the App Pools might stop, including the following:
Causes | Solutions |
---|---|
Application pool identity is invalid. | If the application pool users' username and password combination are correct, you need to grant permission for the service account to logon as a batch process. |
Application pool identity does not have batch login rights. | To change this please check the Settings tab of these two GPOs in Group Policy Management Consult. Make sure your account is not added to “Computer Configuration–>Policies–>Windows Settings–>Security Settings–>Local Policies–>User Rights Assignment–>Deny log on as a batch job” and have correctly configured in “log on as a batch Job”. This should NOT be done by Square 9 techs. Once the group policy is updated you will have to run gpudate /force on the server to pull down the new group policy. |
Application pool has stopped running due to an execution error. | See Event Viewer for more information on the specific cause stopping the application pool. |