Testing, debugging, and error handling
This chapter describes the various aspects of testing and debugging APIs. It includes an overview of testing and debugging, a description of generic best practices for API error handling, with guidance of how to do unit testing ad debugging to do problem determination and analysis.
This chapter contains the following topics:
7.1 Overview of API testing and debugging
API testing is an important discussion. Application developers who use APIs to build mobile or web applications can be from internal, external, or partner organizations. Extensive API testing is the effective way to ensure that APIs are consumable by applications that use them. API testing needs to be even more rigorous when you are providing APIs for external consumption where the user applications that invoke them may be in the order of millions.
Debugging APIs is a big part of ensuring that the API function is maintained intact. Most often, developers spend time debugging the request messages and also the response from the back-end systems. Effective debugging techniques enable developers to determine the root cause of issues in the API function. Inspection of various items such as HTTP headers, response codes, user log entries are basic fundamental concepts that help proper problem determination and analysis.
Advanced techniques of debugging include use of packet analyzers (such as Wireshark, Fiddler, and others) to understand the exchange of information between your API and application, or between the API and the back-end service provider. Most browsers provide debugging tools that are built-in and are helpful while investigating issues in the API.
Good APIs always implement error handling. Exiting gracefully from an erroneous API invocation is important. An important a best practice is to mask back-end errors. Error-handling routines must catch specific errors, or all errors, and convey standard errors back to the calling application.
7.1.1 Testing APIs
As a developer who defines and implements APIs on a regular basis, an important question to ask yourself is “What do we need to test?” Several key factors apply when you are testing APIs:
•Methods for unit testing
•End-to-end integration or system testing
•Performance and load testing
•Regression testing
This chapter focuses on helping you understand the tasks required to carry out unit testing in detail. It also briefly covers the significance of the other factors listed above.
Unit testing of APIs
API developers must unit-test APIs to ensure that the business function of the API is working as designed in isolation. Ensuring that both positive and negative API test cases are captured and tested as a part of the unit test is important.
The following questions are important to ask when unit testing APIs:
•Is it a valid JSON/XML request message?
•Was a valid HTTP verb used for the request?
•If the API uses security, is the authentication working as expected?
•Does the API function as expected for a variety of input parameters (particularly edge cases)?
•Were valid HTTP headers used for the request?
•Did the API respond with valid and expected HTTP headers?
•Is the API displayed as expected in the Developer Portal?
•Is the API adequately documented in the Developer Portal?
•Did the API respond with the valid and expected business response?
•Did the API respond with the valid and expected HTTP response code?
A best practice is to create a unit test plan that contains test cases that cover all kinds of valid requests and corresponding expected response messages and response codes.
The next section shows how to unit-test API resources in IBM API Management.
Apart from ready-to-use tools provided by IBM API Management solutions, developers can also use several open source tools to carry out unit testing:
•curl
•JMeter
•SoapUI
•Browser-based tools, such as REST Console from Chrome and Mozilla Poster applications
End-to-end integration testing
Although successful unit testing proves that the API function is complete, one must carry out integration testing for all resources in the API. Integration testing of APIs is carried out in an environment that contains all the components that are a part of a full business transaction. For example, APIs that will be consumed by mobile applications trigger the business transaction that will invoke the API as a part of the business transaction.
Consider these important factors when you test in the integration environment:
•Ensure that the API is available and online for calling applications to invoke.
•Ensure connectivity can be established between the API resource and LDAP or authentication URLs if they are required for authentication.
•Ensure connectivity can be established between the API resource and the back-end enterprise services.
•Pay attention to the SSL connectivity, and installation of appropriate PKI resources.
Performance and load testing
Production-worthy APIs are reliable APIs when they have to serve the expected peak loads. Commonly, you can hear about a million calls per hour, especially, when you are exposing APIs for external consumption. In large enterprises, internal APIs might also receive high volume traffic, although not as much as external, Internet traffic.
Extensive performance and load testing are the best ways to ensure that APIs can scale in production. This also allows the API developer and the API operations team to determine the thresholds or breaking point of the API.
Several factors are crucial in performance and load testing:
•Ensure you understand and test with messages that are expected and projected in production.
•Are the non-production (or preproduction) environments the same as production, so the performance or load tests in preproduction environments are valid?
•What is the expected concurrency of connected applications?
•What is the back-end latency? This factor is important because the API layer or component must retain the request messages in memory until the message processing is completed at the back end.
•What is the expected end-to-end turnaround time for your API?
Several tools in the industry are capable of configuring and carrying out performance and load testing. Some of the popular tools are SoapUI, JMeter, and LoadUI.
7.2 Testing APIs with IBM API Management 2.0
With IBM API Management, you can carry out unit testing from within the API Manager environment. This section shows how to use the build-in test tools (listed on the Test tab) to do unit tests without leaving the API Manager Environment.
The Test tab is available to the API developer (Shavon’s role), and also to the API consumer (Andre’s role).
7.2.1 Testing in the API developer role (Shavon)
Section 3.4.4, “Starting and testing the API definitions” on page 69 briefly shows how to use the Test tab to test resources of an API. This section provides more detail of how to use the Test tab.
Figure 7-1 indicates the various configurations that one can use to test the API resource in the API developer role.
Figure 7-1 Test tab in the API developer role
Understanding the Test tab
You can use the Test tab to quickly select the resource you want to test. You select the resource from the Select an API drop-down list (shown in the upper-left corner of Figure 7-1 on page 129).
The Test tab also allows you to test the resource with a predefined application called Test application. Select Test application from the Select an Application drop-down list (shown in the upper-right corner of Figure 7-1 on page 129) if you have not created any registered application.
If the API resource has been configured to require the App Secret, type the App Secret in the App Secret field.
Any parameters or request headers that the resource is configured to receive as input are listed in the Parameters and Request Headers section (shown at the bottom-left of Figure 7-1 on page 129).
If your API resource requires basic authentication, enter the Basic Auth user name and password in the appropriate fields.
|
Note: For testing API resources secured by OAuth, you must use external tools. The Test tab cannot be used to test OAuth use cases. Chapter 5, “API security” on page 81 describes how to configure OAuth testing.
|
Using the Test tab in the API developer role (Shavon)
To use the Test tab in the API developer role, log in to the API Manager with the user name and password provided by your administration team. After you are authenticated, the Getting Started page opens. Follow the instructions:
1. Navigate to the APIs menu on the upper-left corner (shown in Figure 7-2 on page 131). Then, click JKHLE API in the bottom-left corner. A new window opens.
Figure 7-2 Navigating to the APIs Menu
3. From the Select API/Resource drop-down list, select GET my/bookedFlights.
4. From the Select an Application drop-down list, select Test application.
5. Keep the App Secret text box empty.
6. Type any user name and password in the Basic Auth Username and Basic Auth Password fields.
|
Note: We type any user name and password because there is no specific authentication that is being done. In reality, this user name and password will be used by either an LDAP endpoint or an Authentication Service that requires the user name and password to be in the Basic Auth HTTP header (Authorization).
|
7. Type [email protected] in the ibm-app-user field of Request Headers section.
8. Click Test.
Figure 7-3 Configuring the Test tab
9. Review the returned response of the test (shown in Figure 7-4).
Figure 7-4 Review the returned response
Ensure that you have the correct business response as indicated. In our case, the back end must respond with all the flights booked by the requestor. Along with the business response, another important step is to review the Response Headers and the Response code, as indicated in the bottom and top-left corners of Figure 7-4.
7.2.2 Testing in the application developer role (Andre)
Application developers must use the Test tab in the Developer Portal to verify whether the API is functioning as expected, and ensure that applications that use the API are not broken.
To use the Test tab in the application developer role, you must log in to the Application Developer Portal with the user name and password provided after the signup task is completed.
Using the JKHLE case scenario in this book as an example, complete the following steps to test as an application developer:
1. Sign in to the application Developer Portal for JKHLE API:
Figure 7-5 Signed-in Application Developer Portal
.gif)
Figure 7-6 Configuring the Test tab (Andre role)
4. Review the Returned Response. The details of the response must be the same as the response received as shown in the previous section.
|
Note: When testing in the App Developer role from the Developer Portal, you must have at least one entitlement selected during the sign-up process. If this is not done, you will not be able get any response from the back end.
|
7.3 Debugging APIs
Debugging is a common task throughout the development process of the API, including during the development phase while unit testing, during the end-to-end integration testing phase, and sometimes during the production phase if it is necessary and possible.
Debugging APIs involves these tasks:
•Tracking the data flow through an API resource
•Reviewing the audit trail
•Checking for any alerts that have been reported by the API Manager
Advanced debugging involves more detailed and thorough inspection of the logs and analysis provided by the various moving parts of the IBM API Management:
•System and multiprotocol gateway logs of DataPower and the related objects
•System logs of Cast Iron
Debugging, problem determination, and analysis can be carried out in the roles of Shavon and Bob.
7.3.1 Debugging in IBM API Management 2.0 (Shavon's role)
IBM API Management provides Activity Feed as a tool to track the API resource data flow, view the API deployment related activity, and check for any alerts from the API Manager.
API developers must use the Activity Feed as the first option to determine any issues with a problematic of failing API and resource.
Understanding the Activity Feed
To view the Activity feed from the API Manager, navigate to Activity Feed from the top-level menu section as indicated in Figure 7-7.
Figure 7-7 Navigating to Activity Feed
Three categories of information are available to the API developer in the Activity Feed menu:
•Audit Activity
•API Activity
•Alerts
Audit Activity
The Audit Activity page (Figure 7-8 on page 136) lists all the changes that are carried out by the authors of all the APIs and their resources. Use this page to verify whether your APIs and resources were created successfully. Starting and stopping APIs and resources are also recorded in the Audit Activity section. They are useful to confirm if APIs have started or stopped successfully.
Figure 7-8 The Audit Activity page
Figure 7-8 indicates the following activities:
•The API Author dinu.shetty@redbooktenant deactivated or stopped the API definition named "JKHLE API v2" at 3:32 PM on November 4.
•This resulted in the deactivation of all the resources of that API.
•At 3:33 PM on November 4, the same API author activated or started the API again.
•There are no entries with a warning or error icon, thus both stopping and starting the API were successful. All entries are of type “information.”
API Activity
All API resource transactions are captured in detail in the API Activity page. It is used to determine the data flow of the API resources. Use this page to do these tasks:
•Determine if the transaction was successful.
•Review request and response headers of the transaction.
•Review the actual request and response data.
•Determine the time-stamp information of each transaction.
•Request, response and data of each individual task in assembly resources.
API developers must click API Activity on the Activity Feed page to navigate to the API activity information. Figure 7-9 indicates the information displayed by the activities page.
Figure 7-9 Reviewing the API Activity page
Figure 7-9 indicates the following information:
•All transactions to the GET my/profile, PUT my/profile, and GET my/bookedFlights resources were successful.
•Clicking Response Body link of the GET my/bookedFlights resource in the passengersv1 API reveals the actual response received from the back-end.
•The Details drill-down button is available for all the transactions to display the details of each task of the assembly resource.
Using the Details drill-down
For each transaction, API developers can drill down into the details of each task of the assembly resources by clicking the Details drill-down link, which is to the right side of each transaction. This is a simple way of debugging a complex assembly resource that might have several tasks. See Figure 7-10 on page 138.
Figure 7-10 Reviewing Details of an assembly resource
To review the details of each task in assembly resource, locate and click the Details drill-down on the right of transaction of interest. Then click the particular task to be reviewed as shown in the top-left corner.
Figure 7-10 shows the following information:
•The details of the HTTP GET Operation task that is configured to invoke the back-end resource:
http://jkhleapi.castironcloud.com:9080/AirlineExample/rest/passenger
•Actual response of that HTTP GET invocation
•Request Parameters if any were used
Assemblies can become complex if several tasks must be configured. Most often, complex assemblies may make several invocations to external or internal services and aggregate data before sending the consolidated response back to the calling application. The Details drill-down is an effective tool when you are dealing with such complex assemblies.
Alerts
The Alerts page captures issues that the API Management system experiences during deployment of APIs. Typical issues of connectivity between the management, analytics and the Gateway nodes will be displayed in the Alerts page.
7.3.2 Debugging in IBM API Management 2.0 (Bob's role)
Bob, as the API administrator, maintains the API Management Environment. To perform this role, Bob also has debugging tools in the Environment Console.
The View Activities page in the Environment Console is where Bob can do the following debugging activities:
•View User Actions
•Review events that have occurred across the environment and
•Check System Alerts
To navigate to the View Activities page, log in to the Environment Console with the administration credentials, drill down through the top-level menu, and click View Activities (Figure 7-11).
Figure 7-11 Navigating to the View Activities page
Understanding the User Actions page
The User Actions page lists all the actions that the administrator, for example, Bob. For example, adding a tenant, gathering postmortem information, adding a node, publishing an environment, deploying DataPower extensions etc. The severity icon also indicates if the entry in the list was an error or a successful action. This is similar to the Audit Activity page in the API developer or Shavon's role. Figure 7-12 on page 140 shows a typical User Actions page.
Figure 7-12 Typical User Actions page
Figure 7-12 reveals this information:
•Bob added a few tenants into the environment on Oct 21, 2013.
•Gathered postmortem information for a particular management node.
•Added a Gateway node.
•Reconfigured the domain name for the Developer Portal in the Environment Settings.
Understanding the Environment Events page
Information displayed in the Environment Events page requires deep technical knowledge of the components of the API Management environment. It is best to review this information in the presence of a DataPower or Cast Iron expert. The Environment Events page shows the following types of information:
•Internal details of the resources that provide leads to identify the failing resource, tenant or any other component
•Specific DataPower and Cast Iron error logs
•Connectivity issues between the four nodes in the environment
Figure 7-13 shows a typical Environment Events page.
Figure 7-13 Typical Environment Events page
Figure 7-13 is from a typical Environment Console from another IBM API Management environment. The figure depicts entries that indicate the following information:
•A couple of errors that indicate that internal DataPower connectivity errors.
•A Cast Iron related error that indicates the inability to connect to a certain back end.
•The icons next to the warning, error, or information indicate the type of node that is reporting the error.
Use the Environments Events page to review the internal issues with the administrator and the DataPower or Cast Iron expert to resolve any issues that have been identified.
Understanding the Alerts page
The Alerts page in the View Activities section is similar to the Environment Events page in that it records and notifies the issues of the environment. The Environment Events tab is a superset of the Alerts tab in that it includes all the Alerts, but can also include other things too. The Alerts page can display identical events as shown on the Environment Events page.
Use the Alerts page to review any potential environment related issues that have may be notified.
7.3.3 Advanced debugging techniques
Advanced debugging techniques can be complex and sometimes also time consuming. These techniques must be used only when the cause of issues in the API Management solution cannot be determined by the available diagnostic information explained in the previous sections.
Significance of the Gateway node and debugging
As you are aware, the Gateway node (DataPower node) receives and processes the inbound API calls by integrating and interacting with the assembly, analytics and the management nodes.
After the IBM API Management Environment is defined initially, a system multiprotocol gateway (MPG) is deployed into the Gateway node. This service is deployed in a DataPower domain with a name that starts with APIMgmt_NNNN (where NNNN is a unique number). The MPG contains DataPower configurations that process incoming request data, headers, security, analytics along with transformation and routing by integrating with the other nodes of the environment. The Gateway node basically brings together all the functions that are provided by the IBM API Management solution.
Because of the role of the Gateway node, debugging the activities that occur in the node can become crucial in identifying and isolating the API transaction issues.
Debugging the Gateway node
This section does not provide details about the DataPower best practices for debugging services. To learn about best practices of DataPower debugging and troubleshooting, go to the following web locations:
•Troubleshooting IBM WebSphere DataPower SOA Appliances: An introduction:
•DataPower Problem Determination Techniques, REDP-4445:
To carry out problem determination in the Gateway node, you will need to ensure that you have at least read-only access to the APIMgmt_NNNN domain and also the authority to switch the debug probe on.
Two multiprotocol gateways are created in the version 2 of the Gateway node:
•webapi
This is the main client-facing gateway that is the entry point into the API Management run time. The preference is to start debugging this MPG for any problem determination and analysis.
•webapi-internal
This is an internal gateway that interfaces with the webapi gateway to carry out internal operations.
Also consider the following items during debugging the Gateway node:
•Step through the debug probe for inspecting headers, variables, and other parameters of interest.
•Inspect the log for the webapi MPG for errors or warnings.
•Check for any error or warning in the Status of System Logs.
•Check the Object Status for down objects and testing the network connectivity to all the nodes from the debug page also.
•Check system, CPU, memory and file system usage to make sure you have enough resources.
•Check the Audit Log for recent changes, and as a last resort run XML File Capture or IP Packet Capture.
Debugging the Assembly node
Assembly resources translate into underlying Cast Iron projects that contain one orchestration. If your APIs have resources that implement assembly type resources and if there are errors and issues that are insufficient to isolate the problem, then you must use advanced debugging techniques for Cast Iron.
This book does not cover Cast Iron debugging as a separate topic. However, for help with debugging Cast Iron problems, see the “MustGather - Collecting data: WebSphere Cast Iron problem determination” technote:
Debugging example
This example reviews a typical error and the corresponding debugging process. The scenario in this case is a simple assembly resource that contains a task to connect to a DB2 table in a database and execute a simple SQL query. A typical configuration is shown in Figure 7-14.
Figure 7-14 DB2 query example
Review the Configure tab, which includes a DB2 Execute Query task and its corresponding SQL query that contains a query parameter. The value of this parameter eventually appears in the incoming request.
Assuming that this resource was a previously working resource, we review the results of a unit test of the resource, as indicated in Figure 7-15.
Figure 7-15 Failing resource
The test indicates that an internal error with code 500 occurred; this must be investigated.
Reviewing the API Activity tab, and drilling down through the details of the affected transaction reveals the error and it is explained in more detail (Figure 7-16). The message in the activity log indicates that the error occurred during implicit rebind or prepare.
|
An error occurred while processing the message for the parameter-ized query activity at row number 1. The error is: SQLSTATE: 56098 ERRORCODE: -727 Error Message: [CastIron Systems][DB2 JDBC Driver][DB2]An error occurred during implicit rebind or prepare
|
Figure 7-16 Error in the activity log
Further, a closer look the Assembly node logs can be used to confirm the errors and the problem. For this, you must log in to the Assembly node (Cast Iron Node). Figure 7-17 shows the findings in the API Activity page.
Figure 7-17 Cast Iron Job Log
Upon reviewing the message in the Cast Iron job log, it shows the exact same error that appears in the API Activity page. Some investigation about the error reveals that the DB2 Execute Query task cannot find the table. Upon checking, the table does exist and it is also in the correct schema.
More investigation reveals the possible existence of a privilege issue of the user (CASTIRON) that connects to the DB2 table. Adding the privileges to the user can fix the issue in such a situation.
Figure 7-18 shows the DB2 Control center depicting the privileges of the table CASTIRON.CITIES.
Figure 7-18 Privileges to the DB2 USER
Running the test again resolves the issue, as shown in Figure 7-19. This exercise shows how you can use advanced debugging techniques to resolve issues.
Figure 7-19 DB2 Problem resolved
7.3.4 Using monitoring, alerting, and postmortems
You can use monitoring, alerting, and postmortems to help manage your API.
Monitoring
IBM API Management offers monitoring as a ready-to-use capability for the API administrator role. The full details of this capability are at the following location:
Several graphs are available in the Environment Console to monitor the health of all nodes in the system.
The following figures are from the Environment Console that depict the health of the nodes in the environment:
Figure 7-20 depicts the various vital statistics of the Gateway node, such as memory, CPU, system load, disk usage, and also the DataPower domain memory against a set threshold.
Figure 7-20 Monitoring the health of the environment
Thresholds that have been exceeded in the environment are indicated as warning on the graphs. Administrators must watch the threshold exceeded warning and investigate further into the specific node that reported the warning. One such example of threshold warning is shown in Figure 7-21.
.gif)
Figure 7-21 Threshold warning on the Environment Console (Garbage Collection)
Alerting
Monitoring relies on the administrator to regularly log in to the Environment Console and check the health of the nodes. Moving parts of the API Management system, like the Gateway and the Assembly nodes, offer the SNMP capability individually and ready-to-use and can provide another source of health and diagnostic information. This means that administrators can configure monitoring applications like IBM Tivoli® Composite Application Manager or any other SNMP-based monitoring solutions to directly monitor DataPower and Cast Iron appliances.
Postmortems
Postmortem is a term for an archive file that contains all the relevant logs of a node. Use postmortems to interact with IBM Support for any resolution required for problems that you are facing in IBM API Management.
Administrators can obtain postmortems from the Environment Console. Figure 7-22 shows the tasks that the administrator does to generate and download the postmortem files.
Figure 7-22 Gathering the postmortem of an Assembly node
7.4 Error handling
Error handling for API resources is no different from error handling for HTTP applications. Always remember that the invoking applications, application developers, and support are consumers of your API resources and will base much of the business and technical logic on the error codes and error messages that you return from your API. Providing short and descriptive return error messages, including meaningful HTTP status codes, enable effective problem determination and analysis.
Popular best practices
There are many known best practices for HTTP error or exception handling. Several popular practices are listed here:
•Avoid using HTTP status code 200 regardless of success or failure. Provide a HTTP status code for every response.
•Use codes that start with 2 for successful requests.
•Use codes that start with 4 for failed requests that can be fixed by the client.
•Use codes that start with 5 for any kind of unknown internal error.
•Be as descriptive, but as short as possible to avoid being contacted constantly by users of your API. The following message, for example is short but not descriptive
Incorrect input
The following short messages, however, are more descriptive:
Could not book shipment. Unknown ZIP Code provided. Provide a valid ZIP Code.
•Mask back-end errors. You want to do this to avoid providing unnecessary information to the client. The following message is perhaps too revealing; it reveals that the database in use is Oracle:
ORA-00947: not enough values
A more generic database error should be used instead:
Database Error: More descriptive error description.
•Document each and every error condition handled by the API resource.
Existing solutions
There are several open API providers in the industry today that have used tried and tested ways of handling errors. For instance the Twitter, Facebook and Github APIs are good examples for study and a source of best practices.
In the following sections you will see how you can implement the best practices using the out-of-box error handling capabilities in IBM API Management 2.0.
7.4.1 Implementing Error Handling in IBM API Management 2.0
In IBM API Management, setting up error handling for your API resource is optional, but recommended. In the case where you have not setup any error handling, the default error handling will be carried out. For instance, any HTTP error message returned by the back end will be relayed as-is to the invoking application.
The API developers must set up error handling. For this, the API developer should be aware of all the possible error conditions that may occur or return from the back-end service. From a practitioner perspective, API developers must work with the back-end application teams to understand all the error conditions and validate the corresponding actions that must be taken in the actual API response.
Implementing error handling in the IBM API Management is simple and requires no coding. It is as easy as configuring an if-then-else condition. To set up error handling, select the error condition that you want to handle, configure the appropriate HTTP code, and set the error message or description relevant to the HTTP status code.
In the API definition process, error handling is set up in the Review step of the task. That is, your Connect, Define, and Configure steps are complete. This is much like configuring an if-then or when-then with otherwise construct.
Figure 7-23 shows the Review step of an HTTP PUT operation task. In this figure, the previous tasks of configuring the Connect, Define and Configure steps are complete. As shown, you can set predefined error conditions for connection and operation failures in the Select Error Code drop-down list.
Figure 7-23 Setting up error handling
Consequently, you can set up the corresponding HTTP Code from a list of predefined HTTP codes in the Select the HTTP Code drop-down list as shown in Figure 7-24.
Figure 7-24 Selecting predefined HTTP codes
Next, configure the type of action you want to take when the response is sent back to the client. You can choose to send back either a system message or a user-defined message that conveys a specific error description, by selecting the appropriate action from the Select Action drop-down list.
For the HTTP PUT operation, you can handle operation and connection error conditions. You can add error handling for both of these conditions by clicking Add on the right of the condition.
Finally, you configure the otherwise part of the construct. This is done by selecting HTTP Status Code from the For ALL other cases Return this HTTP code drop-down list and the corresponding action under the Take this action drop-down list. At run time, when the back end returns an error code that does not match any that have been configured, the default OR otherwise HTTP code and action will be executed.
Figure 7-25 shows a typical error-handling configuration for a resource.
Figure 7-25 Typical error handling configuration
Figure 7-25 shows the following information:
•A known connection error has been handled by sending a 408 HTTP status code and system message.
•A known operation error is configured to send a 400 HTTP status code for a bad request with a specific user-defined message that prompts the user to review the ZIP Code field being passed in the request.
•For all other error codes, the system will send out HTTP status code 500 along with a system message.
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.