Lately I’ve been doing some publishing on ISA & TMG and here are some of the errors I came accross with a solution. I’m not saying they will also work for you as these error messages are perhaps generic, but for me they did. Some of them I stumbled upon trying to publish an AD FS 2.0 enabled web application, some of them when publishing an application using two factor authentication on ISA.

When an application is AD FS enabled, you no longer authenticate to the application server. In fact on the first visit you are redirected to your security token service server (AD FS). You authenticate against the STS, get a SAML token (claim) and present that to the application. One of the possibilities to publish this on ISA is to use two publishing rules with the same listener. Configure the listener for FBA (or something else) and configure a SSO domain. One rule publishes app.contoso.com, the other adfs.contoso.com. Visiting app.contoso.com will show you the FBA, you get to the application and are immediately redirected to adfs.contoso.com. Because of the ISA SSO on the listener you don’t have to authenticate a second time. Here are the errors I came across during this process. I believe they are specific to how the AD FS service expects it’s communication:

• Error Code: 500 Internal Server Error. The request was rejected by the HTTP filter. Contact the server administrator. (12217)

For some reason ISA is complaining, I found a KB article with some explanation: KB837865: You receive a "The request was rejected by the HTTP Security filter" error message when you try to... and also a blog post: Error Code: 500 Internal Server Error. The request was rejected by the HTTP filter. Contact the server administrator. (12217)

Bottom line: all you have to do is right click your rule, choose configure HTTP and deselect verify normalization

Another problem I came across during the AD FS publishing:

• ID6018: Digest verification failed for reference  ‘#_long_id’

This can be solved by unchecking “apply link translation to this rule” on the web publishing rule. It struck me as odd as there was no really translation configured (internal name was the same as external), but it' did seem to do the trick.

An error I come across now and then is “Error Code: 403 Forbidden. The server denied the specified Uniform Resource Locator (URL). Contact the server administrator. (12202)”. One of the possible reasons is a Web Publishing Rule configured with Kerberos Constrained Delegation (KCD) and you forgot to trust to ISA computer object for delegation in Active Directory.

And last but not least a fun fact about 2 factor authentication on ISA 2006 SP1 and TMG: you can configure a listener for Form Based Authentication (FBA) and configure the advanced option “require client certificate”. However there is a subtle difference when you choose either LDAP or AD to connect your Active Directory environment:

• When configure as LDAP (or RADIUS, RADIUS OTP and SecurID) ISA will only verify if the presented client certificate is trusted by ISA. There is no need for a link between the user credentials and the certificate. So you can present a trusted certificate, and use any user to authenticate.
• When configured as Active Directory, ISA will try to find a mapping to a user for the presented certificate in Active Directory and afterwards verify if the credentials entered match the user found by mapping the certificate.

This can cause an issue when you want to hand out a certificate to a user, but allow him to log on use multiple accounts. If the user enters credentials other than those of the user the certificate is mapped to, or when ISA doesn't find a mapping in AD, you’ll get the following error: ERROR : 403: forbidden Authentication failed. The client certificate used to establish the SSL connection with the forefront TMG computer doesn’t match with the user credentials that you entered (12323 )

Now how can a user map to a certificate?

1. When you enable advanced options in ADUC, you can right-click a user and choose “name mappings”. Then you can upload a certificate.
2. When the certificate contains an UPN in the SAN part of the certificate. Of course the UPN needs to map to a user in AD.

Now how can we alter the ISA behavior? KB 953684  How to change the default behavior for client certificate mapping when you use forms-based authentication with Active Directory in ISA Server 2006 Service Pack 1 of course you have to decide for yourself whether this is desired or not.