Introduction
In my previous article, I described some general concepts of the Messages Processing by API Gateways, and the unique approach Sentinet API Management platform takes in their implementations. Workflows and message orchestrations are particular features considered to be “must-haves” by any capable API Gateway. Not only workflows must allow the implementation of simple conditional message transformations, but they also should permit the implementation of the custom orchestrations which can enable API Management with advanced and yet very practical workflows and design patterns.
In this article, I will describe how API Gateways can implement a Circuit Breaker design pattern to enable backend APIs with extra protection.
What is the Circuit Breaker design pattern and why do you need it for your APIs?
Circuit Pattern is a well-known software design pattern, which is used for detection of system failures followed by the implementation of logic, which prevents cascading failures particularly in the distributed system. In modern IT architectures, APIs are often seen as the gateways into such systems. Suppose your backend APIs connect to other distributed applications such as databases, network devices, or cloud services. These connections or their remote services may experience transient failures because of the slow network, timeouts, or the resources being overcommitted or temporarily unavailable. Circuit Breaker pattern detects such failures and temporary disables traffic to API giving the remote resources a chance to recover, so that “likely to happen” volume failures do not create a cascading effect on the entire system and ultimately stall it.
API Gateways with Circuit Breaker implementations will give you a “free” extra protection for your backend APIs and potentially your entire backend systems.
A quick recap on how the Circuit Breaker design pattern works
Circuit Breaker pattern implements a State Machine which can be in one of three states:
1. Closed state. In this state, request messages fly through an API Gateway normally.
2. Open state. If an API Gateway detects X (configurable) number of errors during T1 (configurable) time interval while calling a backend API, then it puts its State Machine in an Open state and immediately returns an error message to an API Consumer on its every request message received during T2 (configurable) time interval. Effectively, an API Gateway prevents connections to a backend API for the period of T2 wait time, giving the backend system a chance to recover without being overloaded with “likely to happen” volume failures.
3. Half-Open state. After staying T2 time in an Open state, an API Gateway places itself in a Half-Open state when it allows Y (configurable) number of request messages to fly through normally. If they all succeed, an API Gateway changes its state back to Closed (see above). If any fail, it changes its state back to Open and starts a new T2 wait time interval.
Options for Circuit Breaker implementation by an API Gateway
By looking at the description of the Machine States above and the nature of its configuration parameters X, Y, T1 and T2, we can see that Circuit Breaker implementation will require from an API Gateway at least the following capabilities:
- Managing timers (T1 and T2)
- Managing counters (X and Y), which are better be distributed and persistent to accurately handle cases with multiple load-balanced API Gateway instances.
- Caching
- Conditional Request-Response coordination
The need for the four capabilities above means that implementation:
- Is not trivial, and better be delegated to an API Gateway rather than embedded into each backend API’s implementation/deployment.
- Requires an API Gateway, which has built-in capabilities listed above, and can implement workflow specific to the Circuit Breaker pattern.
An API Gateway can take two possible approaches for the Circuit Breaker pattern implementation:
- An API Gateway can offer a built-in Circuit Breaker implementation as a stand-alone feature.
Pros: it’s a built-in feature, which means it requires minimum effort from an API Management user to enable and configure Circuit Breaker.
Cons: actual implementation and configuration options are hardcoded, which means they are limited to what API Gateway vendor decided to put into Circuit Breaker (for example, with hardcoded implementation you may not be able to control different error messages returned by the Circuit Breaker, different status codes, extra conditions for Machine States, etc.) - An API Gateway can offer Circuit Breaker implementation in the form of a custom workflow created from the basic Message Processing building blocks (such as If condition, Caching, Context Property, Custom Code, Timer, etc.)
Pros:- you can build your workflow using any configuration parameters, any custom logic, and any custom code.
- If you can build Circuit Breaker just as a Message Processing workflow – it means you get a high level of confidence that you can build virtually any other pattern and workflow regardless of what is built-in in an API Gateway.
Cons: you may have a bit more configuration work to do because you are building a custom workflow (even though configuration complexity will highly depend on what it takes to build and deploy a custom workflow, and the ease of workflow development can compensate the Cons).
Circuit Breaker implementation by the Sentinet API Management
Nevatech Sentinet API Management takes the second approach by offering flexible workflow implementation with the high level of confidence you can build similar workflows for any other patterns, scenarios, or use cases. It also provides a simple drag-and-drop User Interface for workflow creations with their automatic deployment.
The screenshot above shows example of a Circuit Breaker workflow designed for the Sentinet using drag-and-drop of the basic Message Processing building blocks. A Sentinet workflow automatically generates its XML script (found behind the Source tab highlighted inside the green rectangle on the screenshot above). XML script can also be used as a copy-paste text to add this or modified Circuit Breaker to any number of APIs managed by Sentinet. This example uses the following Circuit Breaker configuration parameters in its workflow: X=5, Y=1, T1=10 seconds, T2=10 seconds, where X, Y, T1 and T2 are explained above in this article. Complete XML source of this pipeline for Sentinet 6.6 can be downloaded from here, and for Sentinet prior to version 6.6 from here. Helpful Visio diagrams can be downloaded from here.
But there is more. Instead of building workflow using drag-and-drop Designer, Sentinet also allows complete implementation of any message processing (including Circuit Breaker implementation) in a custom .NET code, where the entire .NET Framework and even third-party libraries are available for your custom message processing. Your code compiled in .NET assembly(ies) can be registered as a custom message processing Pipeline component, and then used with any API right in the Sentinet UI.
The screenshot above shows a simple reusable UI with the custom CircuitBreaker pipeline component, which is compiled from a custom .NET code. The source code of this sample component can be found here.
In the next article, I will give another very practical example of using API Gateway workflows when an API Gateway is required to make an outbound call to a partner API which requires custom OAuth authentication.