Form-based authentication is suited to applications that present a login form instead of a dialog box. This sort of authentication configuration makes sense when the application itself is form-based.
Form-based Authentication makes most sense when used with SSO. The typical configuration uses a SuccessRule parameter, which is a means for NetScaler to detect if the SSO was successful or not. If the SuccessRule criterion is met, NetScaler presents the final page, such the User's mailbox, when the application is OWA. If it is not met, NetScaler passes the login page as is to the User so that they can manually enter the credentials – say, when the credentials for SSO are not the same as for NetScaler authentication.
Please take a look at CTX128197 (http://support.citrix.com/article/CTX128197) for an article which shows you how to configure form-based SSO for exchange 2010 through NetScaler.
Authentication flow
Let's take a look at the flow as it should happen:
The client sends the GET for a resource; in the case of OWA it will be /owa.
NetScaler checks to see if authentication is configured and if an Authentication Cookie (NSC_TMAA or NSC_TMAS if secured) is already present in the request. If it is, the User doesn't need to authenticate again and is taken straight to the service.
Since the User is connecting for the first time, they won't have the Cookie. NetScaler then looks at the configuration to see if form-based authentication is configured.
Since the configuration contains form-based authentication, NetScaler will respond with a script. This script will contain the details of the AAA VIP to go to, to authenticate. Following is what the script looks like. It contains the details of the AAA vServer that the User needs to go to:
The result of executing the preceding script will be a redirection to a page where they can log in (/cgi/login):
Once on this page, the User types in their credentials and the authentication vServer authenticates the User. As part of the authentication, if SSO is configured it also tries to authenticate to the backend server, such as OWA, to see if the credentials are accepted, and if they are, looks to see if the response contains the success criteria. Reviewing CTX128197, we know that the criterion is for cadata cookie to be larger than 70 characters:
When this condition is met, NetScaler obtains the OWA mail homepage itself and presents it to the User. In the process it also increments the SSO counters as successful.
If SSO is not configured, the User will see the OWA login page instead.
The following steps detail how to troubleshoot form-based authentication:
DNS needs to work correctly for the redirect to the AAA URL to work, and so do certificates, on the LB vServer, but also on the authentication vServer. So, check to see if the users can resolve and access the LB vServer and the AAA vServer without any errors.
If you are still unable to get to the AAA page, try adding the site to the list of trusted sites.
If SSO is failing (as earlier, use nsconmsg –g sso –d current from shell to check if it is):
Check first if the SSO policy is getting hits.
Take a trace and check if cookies TMAA/TMAS are getting inserted.
Is rewrite enabled and are the policies getting hits? Also, check that the PBack cookie for OWA is getting inserted.
Is the success rule correct? This will be different for different apps. The correct way to determine what that should be for your custom application is it to take a trace and identify a cookie that gets consistently set following a successful login.
Did you set a high enough value for responsesize? Look at a successful response's content-length to determine the right value.