Messages Pipeline Processing
Messages Processing Pipeline is where virtual service runtime can be enhanced with custom messages processing. Sentinet provides several built-in message processing components and offers extensibility with custom-defined message processing components.
A virtual service runtime pipeline consists of four pipelines that follow the flow of messages from the client application to the physical service and optionally back (for Request-Response operations). These are Request-Inbound, Request-Outbound, Response-Outbound and Response-Inbound pipelines.
Inbound pipelines are executed in all virtual service's inbound endpoints, while Outbound pipelines are executed in all virtual service's outbound endpoints. There might be some implicit "unconditional" message transformations between inbound and outbound pipelines (for example, automatic conversion from SOAP 1.1 to SOAP 1.2, conversion from REST to SOAP message, or conversion of a REST request messages driven by the mapping between virtual and virtualized Uri templates).
The result of some specific message processing functions may differ depending on where the processing occurs, inbound or outbound pipelines, because the content of the message-in-transit may be different in these pipelines. Because of these possible differences, the whole virtual service's pipeline is split into Inbound and Outbound pipelines, and, because there is Request and possibly Response processing, each inbound and outbound pipeline is further split into Request and Response pipeline. Thus, the total number of distinct pipelines is four that comprise a single virtual service pipeline (see figure above).
Virtual service's Request-Inbound and Response-Outbound pipelines are always executed after all WCF "standard" channels, thus they are executed on "clear-text" messages (for example, messages that arrive to the virtual service in encrypted format will be processed by the Pipeline once they are decrypted by the WCF security channel, or net.tcp binary encoded messages will be executed in the Pipeline in XML decoded format). Similarly, Request-Outbound and Response-Inbound pipelines are always executed before WCF "standard" channels, resulting in the same "clear-text" processing.
Note
Technically speaking, a single Pipeline is executed as a special built-in client or service WCF endpoint message inspector, where a single inspector can be graphically configured with multiple message processing components, including custom components
Sentinet console provides a graphical designer for the Pipeline configuration. The designer consists of the graphical representation of the Pipeline configuration and the Designer Toolbox with predefined (built-in) and custom message processing components (Designer Toolbox is visible when Pipeline is being edited).
Internally, a Pipeline configuration is stored in XML format and can be viewed and edited from the Source view.
There are four groups of the pipeline component that can be placed in a single pipeline: Flow, Modification, Generic and Custom components:
Once pipelines are configured with the pipeline components, they will be executed in the order they were graphically placed in. The order of execution follows the flow of the connecting arrows.
Components Drag-and-Drop
Drag-and-drop component on the empty pipeline surface to place component in the pipeline:
To drop another component on the same pipeline use drop "before" or drop "after" areas to define position of the dragged component relative to the existing component. "Before" and "after" areas are equally split between the right and left of an existing component. Note: "before" and "after" areas have reverted locations in the Request and Response pipelines. To easily determine the correct location, just follow the flow of the Request and Response arrows on the diagram.
The Conditional Flow If component, explained in more detail later in this guide, has four drop areas, "before" and "after" in addition to "true" and "false" areas. To drop on a "true" or "false" area, drop component next to the word true or false on the diagram.
When you drop a component on the pipeline surface, its configuration dialog box shows component specific configuration properties. Some components have default configuration, others require explicit configuration. If a component is not fully configured, it will be shown with a red background and the pipeline cannot be saved until component configuration is fully completed.
Pipeline components can be dragged-and-dropped within the same pipeline to re-arrange their position relative to other pipeline components in the pipeline.
Double-click existing pipeline component or use right-click mouse context menu to modify the configuration of an existing pipeline component.
Modification Components
Modification components provide built-in message transformation functions.
HTTP Header component
HTTP Header component adds, removes or modifies one or more HTTP headers of the Request or Response message. Each HTTP header added to the component's configuration must be defined with the Action that instructs what to do with the header value if the HTTP header already exists. If HTTP header is not found in the message, it will always be added as a new header.
The Action attributes are:
Remove – header will be removed (only header name can be entered in this case)
Overwrite – header value will be overwritten with the new value
Skip – header value will remain unchanged (header is skipped)
Append – new value will be appended to the existing header value
To add an HTTP header to the list of headers managed by the instance of this component, click + Add button.
Important
HTTP Header component can be placed in both Request-Outbound and Response-Inbound pipelines.
Header Name and Value properties may include Context Property values (see Context Properties component for more details). For example, MyHeader value may be defined as 123<{MyContextProperty}>, where <{MyContextProperty}> is the token placeholder for the MyContextProperty value. If at runtime MyContextProperty happens to have value 456, then the HTTP header MyHeader will get the value 123456.
Set HTTP Method component
Set HTTP Method component sets the HTTP Method (HTTP Verb) of the Request message. It is used when HTTP Method of an originally received Request message must be changed.
Set HTTP Status Code component
Set HTTP Status Code component sets the HTTP status code of the Response message.
You configure component with returned response’s HTTP Status code from the dropdown list of available HTTP statuses, or use a Context Property to assign HTTP status code. An optional Status Description can be added to the returned status code.
Important
HTTP Status component can be placed in both Response-Outbound and Response-Inbound pipelines.
The component can be used to overwrite HTTP status codes returned by the business service API. For example, a business service may return some error text in the body alone with the 401 (Not Authorized) status code. The API client (for example the browser application) that receives this response might be interested in the error message text, but can only process it if the return status code is 200 OK. The Set HTTP Status component can be used to ensure that HTTP status code is 200, even though the business API responds with 401.
Query Parameters component
Query Parameters component adds, removes or modifies one or more Query Parameters of the Request message. Each Query Parameter added to the component's configuration must be defined with the Action that instructs what to do with the Query Parameter if it is already found in the message. If the Query Parameter is not found in the message, it will always be added as a new parameter regardless of the Action value.
The Actions are:
Remove – Query Parameter will be removed (only Query Parameter name can be entered in this case)
Overwrite – Query Parameter will be overwritten with the new value
Skip – Query Parameter remains unchanged (Query Parameter is skipped)
Append – new value will be appended to the existing Query Parameter value
To add a Query Parameter to the list of Query Parameters managed by the instance of this component, click [+ Add] button.
Important
Query Parameter component can be placed in Request-Outbound pipeline only.
Query Parameter Name and Value properties may include Context Property values (see Context Property component for more details). For example, customerId value may be defined as abc<{MyContextProperty}>, where <{MyContextProperty}> is the token placeholder for the MyContextProperty value. If at runtime MyContextProperty happens to have value 123 then the Query Parameter customerId will get the value of abc123.
Set Relative URL component
Set Relative URL component modifies the request's URL path segments according to the Source and Binding templates that transform the actual request's URL path. The Relative request URL Path is a part of a full request URL, that follows an endpoint address. For example, if the endpoint address of a virtual service is http://myserver/myapi/v1/10/2017, and the actual full URL of a request was http://myserver/myapi/v1/10/2017/a/1/2?x=3, then only the "/a/1/2?x=3" part can be modified by this pipeline component.
Set Relative URL component modifies only the path segments not the Query Parameters, which remain unchanged when Set Relative URL component executes. Use Query Parameters pipeline component when you need to add, remove or modify Query Parameters.
Source template describes how the current request must be matched and parsed against this template. For example, if the URL path is a/1/2?x=3 and Source template is defined as /a/{b}/{c=c_defaultvalue}?x={q}, then there will be a match and the original URL path will be parsed to produce variables and their values. In this example: {b}=1, {c}=2, {q}=3. If the URL path does not match the Source template, then this pipeline component will not change anything during its execution.
Binding template describes how the new URL path will be constructed from the parsed variables. For example, if the Binding template is defined as y/z/{q}/{c}/{b}, then the original request URL path described above will be transformed to y/z/3/2/1.
Note, that all Query Parameters will remain unchanged in the original request URL path. That means, in the above example, ?x=3 will remain unchanged in the actual request despite the fact that it is also transformed in the URL path segment by this component. Do not use Query Parameters in the Binding Template field.
Set Relative URL component provides a Test Template section to test component and configuration against a sample URI path.
Both Source and Binding templates may contain Context Properties. Note, that you cannot test templates in this case, because the values of Context Properties are only set at runtime.
Set Absolute URL component
Set Absolute URL component allows to rewrite the entire URL of a forwarded Request (note that Query Parameter and Set Relative URL components can modify only specific parts of the Request’ URL). This is the only component that among other changes can also modify endpoint’s host name.
SOAP Header component
SOAP Header component adds, removes or modifies a SOAP header of the Request or Response message. SOAP header configured with the component must be defined with the Action that instructs what to do with the header value if SOAP header already exists in the message. If the SOAP header is not found in the message, it will always be added as a new header.
The Actions are:
Remove – Header will be removed. Header Name is required field in the configuration of the components in this case. Header Namespace is optional.
Overwrite – Header's value will be overwritten with the new value.
Skip – Header's value will remain unchanged (header is skipped).
Append – New value will be appended to the existing Header value.
Important
SOAP Header component can be placed in both Request-Outbound and Response-Inbound pipelines.
Header Name, Namespace and Header Content properties may include Context Property values (see Context Property component for more details).
Set Message Body component
Set Message Body component is used to replace the entire content of the Request or Response message body. For REST services the content of the Message Body field must contain the entire content of an HTTP request or response body. When this component is used for SOAP services, its Message Body field must contain the desired content of the SOAP <Body> XML element. For example, if you expect virtual service’s complete SOAP response as:
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<SearchResponse xmlns="http://www.nevatech.com/samples/04/2011/CustomerSearchContract">
<SearchResult>Found: 123</SearchResult>
</SearchResponse>
</s:Body>
</s:Envelope>
Then the Set Message Body component’s Message Body field must be configured with:
<SearchResponse xmlns="http://www.nevatech.com/samples/04/2011/CustomerSearchContract">
<SearchResult>Found: 123</SearchResult>
</SearchResponse>
because the <Body> and the <Envelope> elements will be later automatically added according to the required policy of an endpoint the message will be sent through.
Message Body property may include Context Property values (see Context Property component for more details). For example, Message Body component placed in the Response pipeline may be configured with a test response message that prints out the value of the <{CustomerIdContextProperty}> Context Property:
My Test Response message. Customer Id is : '<{CustomerIdContextProperty}>'
The component will overwrite any message body that is currently flowing through the Response pipeline (regardless of response body received from the business service).
JSON Transformation component
JSON Transformation component is used to:
Implement message transformations between XML and JSON message formats.
Add JSON with padding (JSONP) support to allow cross-domain calls from JavaScript browser-based clients.
Important
JSON Transformation component can be placed in both Request-Inbound and Response-Inbound pipelines.
When this component is placed in the Request-Inbound pipeline, it can be configured with explicit XML to JSON or JSON to XML transformation. Each conversion type can be further configured with additional configuration properties explained in the User Interface of the component's configuration dialog box.
Figure. Conversion from XML to JSON configuration. 
Figure. Conversion from JSON to XML configuration. When component is placed in the Response-Inbound pipeline, it can be configured with explicit XML to JSON or JSON to XML transformation and Auto conversion. Each conversion type can be further configured with additional configuration properties explained in the User Interface of the component's configuration dialog box.
The Response-Inbound pipeline component can also be configured with support for JSONP to allow cross-domain calls from JavaScript browser-based clients.
Figure. Example of the component configuration with Auto conversion of XML to JSON and JSON to XML.
Note
XML to JSON transformation automatically removes all namespaces from the XML message during the conversion process, so that namespaces do not show up in the resulting JSON message (unless Preserve XML Namespaces configuration property is checked).
Regular Expression component
Regular Expression is used to replace content based on regular expression searches:
Source value specifies where the Regular Expression will be applied to: Message Body, HTTP Header value or the value of a Context Property.
Explanation for the Regular Expression configuration properties can be found in the Regular Expression Options documentation in the Microsoft library.
Replace Value property may include Context Property values (see Context Property component in this User Guide for more details).
String Replacement component
String Replacement component modifies content based on the string replacement function. More than one replacement string can be configured with the component.
Source specifies where the string replacement will be applied: Message Body, HTTP Header value or the value of a Context Property.
Old Value and New Value properties may include Context Property values (see Context Property component for more details).
Namespace Replacement component
Namespace Replacement component replaces all XML namespaces in an XML message with a new value. If the new value is an empty string, then all namespaces are removed from the XML message. The component is particularly useful for REST clients expecting XML messages with all namespaces removed from the messages.
Source specifies where the namespace replacement will be applied: Message Body, HTTP Header value or the value of a Context Property.
Namespace property may include Context Property values (see Context Property component for more details).
XDT Transformation component
XDT Transformation component converts message body content according to specified XDT transformation document provided for component configuration.
Source specifies where the XDT transformation will be applied: Message Body, HTTP Header value or the value of a Context Property.
XDT transformation syntax can be found in this document in the Microsoft library.
XSL Transformation component
XSL Transformation component converts message body content according to the specified XSL transformation document provided for component configuration.
Source specifies where the XSL transformation will be applied: Message Body, HTTP Header value or the value of a Context Property.
XSL transformation syntax can be found in this document.
Important
For SOAP messages (for virtual SOAP services) XDT and XSLT components execute transformation of the SOAP Body XML element’s content, but not the entire SOAP XML message.
Liquid Transformation component
This component allows transformation of any textual content extracted from component's Source according to an open-source template language Liquid, created by Shopify and implemented in the Sentinet by means of using its open source .NET version, DotLiquid. Review Liquid Template documentation for complete details and examples of its usage.
Source specifies what Liquid template will be transforming: Message Body, HTTP Header value or the value of a Context Property. Extracted content from the Source is called the original content for the Liquid Transformation.
The original content will be parsed out as a graph of objects according to the Liquid template syntax, where the root of the object graph is always called Content by the Sentinet parsing engine. Graph objects and their properties will be denoted in a template by double curly brackets, for example {{ Content.age }}.
Liquid Content Type specifies how Sentinet Liquid transformation engine will parse original content in order to build its objects graph. Options for this component’s property are:
Auto - component will try to parse the original content as XML, if it fails then as JSON, and if it fails again, Content graph object will be initialized with the original content string. If parsing succeeds then Content will be provided as a graph of objects.
Xml - the original content will be parsed out from XML and then provided as a graph of objects (see examples below). The properties corresponding to XML attributes will be prefixed with attr- characters. For example, {{ Content.name.attr-first }}.
Json - the original content will be parsed out from JSON and then provided as a graph of objects (see examples below).
Raw - Content graph object will be initialized with the original content string, which means that {{ Content }} token in a template will be replaced with the original string from the Source.
None - Content graph object will not be initialized and will not be used in a Liquid template.
XML Source example
Let’s say that the Source, which is extracted from the Message Body has the following XML content:
<user xmlns="http://tempuri.org">
<name first="John" >Smith</name>22
</user>
After parsing that content (as XML Source) by the Sentinet Liquid transformation engine, the following objects graph will be created:
- Content.user
- Content.user.attr-xmlns
- Content.name
- Content.name.attr-first
- Content.name.text
- Content.user.text
If we apply Liquid template like this:
{
"FullName": "{{ Content.user.name.attr-first | upcase }} {{ Content.user.name.text | upcase }}",
"Age (years)": {{ Content.user.text }},
}
the result of transformation will be:
{
"FullName": "JOHN SMITH",
"Age (years)": 22,
}
JSON Source example
Let’s say that the Source, which is extracted from the Message Body has the following JSON content:
{
"name":
{ "First": "John",
"Last" : "Smith"
},
"age": 22
}
After parsing that content (as JSON Source) by the Sentinet Liquid transformation engine, the following objects graph will be created:
- Content.name
- Content.name.First
- Content.name.Last
- Content.age
If we apply Liquid template like this:
{
"FullName": "{{ Content.name.First | upcase }} {{ Content.name.Last | upcase }}",
"Age (years)": {{ Content.age }},
}
the result of transformation will be:
{
"FullName": "JOHN SMITH",
"Age (years)": 22,
}
In addition to the Content graph object, Liquid template can also use Context object that has properties initialized with the context of a Request or Response message. The properties of the Context object are:
RequestUri – initialized with the Uri of the original Request message.
Example of usage: {{ Context.RequestUri }}
IsEmptyMessage – initialized with true if Request or Response (depending where Liquid transformation is used) body is not empty.
Example of usage: {{ Context.IsEmptyMessage }}
IsFaultOrHttpError – initialized with true if Response message is a SOAP Fault or an HTTP error.
Example of usage: {{ Context.IsFaultOrHttpError }}
HttpStatusCode – initialized with the status code of the Response message.
Example of usage: {{ Context.HttpStatusCode }}
HttpHeaders – collection of HTTP Headers Properties initialized from HTTP Headers of the Request or Response message.
Example of usage: {{ Context.HttpHeaders ["MyHttpHeader"] }}
Properties – collection of Context Properties initialized from the previously known pipeline’s Context Properties.
Example of usage: {{ Context.Properties["MyContextProperty"] }}, or using Sentinet built-in Context Properties:
{{ Context.Properties["Sentinet.TransactionId"] }}
Variables – collection of variables that is initialized from the variables defined for the virtual service. Variables’ names can be seen in the Parameters table of a Request object.
Example of usage: {{ Context.Variables["subscriptionId"] }} (see screenshot below).
Figure. Virtual service’s variables.
Process Empty Content flag controls how to process empty original content. If flag is set to true (check mark is checked) and the original content is empty, the component should still render the Liquid template and assign the result (this will be the same result as if Liquid Content Type is set to None). If flag is set to false (check mark is unchecked, this is the default option) then the component does not transform anything and the source remains empty. When Liquid Content Type is set to None, this flag is ignored and component always uses Liquid template to generate result while Content graph object remains uninitialized.
Template field must be initialized with the Liquid template content.
The component's template can be tested before its configuration is saved with the virtual service pipeline. Click Test button to test your Liquid template:
Enter Sample Content and click Test button to see generated result of transformation:
Custom Code component
Custom Code component is very similar to a generic Custom pipeline component, except that its code is entered inline in component’s configuration, and Custom Code component does not require any up-front compilation and deployment of a custom assembly. Your custom C# code will be compiled “on-a-fly” and executed in an automatically compiled and cached assembly. Custom Code component is best suited for custom extensibility that requires reasonably little of custom C# code, which can fit in a single C# method. Internally the component implements ProcessMessage method of the IMessageProcessor interface described here, and as part of its configuration it requires just the body of the method.
Let’s consider an example. Suppose you need to create a custom HTTP header, XDate (which you could have created using another built-in HTTP Header component), but in this case the value for the custom HTTP header must come from the current date-time, which can be assigned only programmatically (and that’s why we need Custom Code component here). We can certainly use generic Custom pipeline component, but in this case, we would have to write complete .NET class code, compile it into our own assembly and deploy custom assembly with Sentinet Node(s). By using Custom Code component, we can just provide C# code for the body of the ProcessMessage method in the Sentinet Administrative Console:
The component has the following fields to be populated:
References – list of optional assemblies, which will be used (referenced) by the code of your Custom Code component. The list is optional because by default Sentinet will automatically use the following referenced assemblies:
- mscorlib
- System
- System.ServiceModel
- Nevatech.Vsb.Repository
Note: example above does not need any assemblies added to the default list.
If not empty, the list must be entered separated by spaces or new line. If additionally referenced assemblies are coming from the GAC, they must be specified with fully qualified names, for example:
System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 System.IdentityModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 System.IdentityModel.Tokens.Jwt, Version=4.0.2.28, Culture=neutral, PublicKeyToken=31bf3856ad364e35Using – semicolon separated optional list of C# using namespaces that may be needed for the code of your Custom Code component. The list is optional because by default Sentinet will automatically use the following namespaces:
- System
- Nevatech.Vsb.Repository
- Nevatech.Vsb.Repository.Processing
Note: example above does not need any using statements added to the default list.
Method – body of the ProcessMessage method, where context variable is assumed to available by your custom code.
Before Custom Code component is saved, it is recommended to test that its code compiles without any compilation errors. When you click Validate button, Sentinet will validate that component can be compiled with no errors, otherwise compilation errors will be shown and component’s code can be fixed before component is saved in the Pipeline.
Important
Validation of compiled custom code happens on the Repository Server machine, while at runtime Custom Code component will be used on the Sentinet Node(s) machine(s). That is why it is important to make sure any additionally referenced assemblies and using namespace statements are available on Repository Server and Sentinet Node(s) machines.
Generic components
Generic pipeline components do not modify in-flight messages. They are typically used to extract part of the message (or its associated context) in the pipeline variable (called Context Property), or to create a custom recording out of the messages data (Monitoring Property).
Context Properties component
Context Properties component can be placed anywhere in any pipeline. It is a special pipeline component used to extract defined data from the Request or Response message and assign it to a pipeline "variable" called Context Property. The Context Property can then be used further in the pipeline(s) to configure other pipelines' components. For example, a specific Query Parameter value can be extracted from the Request message in the Request-Inbound pipeline and assigned to a Context Property. Later, in the Response pipeline, the value of the Context Property can be used in a Conditional If component (see Conditional If component chapter in this guide for more details) to modify the content of the Response message based on the value of the Context Property (effectively creating conditional modification of the Response message based on the value of the Request's Query Parameter). Another example: Response body can be modified with the String Replacement component, where the new string value is built using the value of the Context Property.
A single Context Properties pipeline component can store one or more actual Context Properties.
Context Property's Property Name field specifies the name of the context property. Later in the pipeline, that Context Property component can be referenced by its name in the format of <{PropertyName}> token, where PropertyName is the value entered in the Property Name field.
For example:
Context Property is named CustomerId
At runtime, it is initialized with the value 123456 (for example from the value of the request's Query Parameter)
<{CustomerId}> token is later used in the pipeline as a new string value of the String Replacement component, for example:
CustomerId: is replaced with Customer Id: <{CustomerId}>
String Replace component now replaces CustomerId with Customer Id: 123456
Property Name field may also be initialized from the values of some other Context Properties. For example: <{MyProperty1}>_and_<{MyProperty2}>. If <{MyProperty1}> was initialized with value One and <{MyProperty2}> was initialized with value Two, then this Context Property's name becomes One_and_Two.
Context Property can be set or removed from the pipeline. If it is configured to be set, then If Already Set option specifies what to do with the Context Property if it is already found in the pipeline. If Context Property is not found in the pipeline, then it will always be added with the Set Property option.
If Already Set options are:
Overwrite – Context Property value will be overwritten with the new value
Skip – Context Property value will remain unchanged
A Context Property component must be configured with the Value Source and optional properties associated with the selected Value Source.
Value Source property specifies where the value for the Context Property should be extracted from, while Value Extractor specifies how to extract the value from the Value Source.
Available Context Property Value Sources are:
Constant – Context Property is not extracted from the messages; it is assigned a static constant string value (that's why Constant Context Property does not need Value Extractor).
Example: Context Property component is named IsValidRequest and it is dropped in the Request pipeline. This component may be configured with the constant string value false based on some previous Conditional If component flow. Later in the Response pipeline there might be another Conditional If component that modifies the content of the Response message based on the value of the IsValidRequest Context Property (effectively IsValidRequest Context Property in this case is used as a static flag that is flowing through the pipelines and drives pipelines execution logic).
HTTP Method – Context Property value is initialized with the value of the Request message HTTP method name, for example: GET, POST, PUT, DELETE, etc. This Value Source does not require any additional configuration.
HTTP Header – Context Property value is extracted from the HTTP Header value. Specify HTTP Header Name property to identify specific HTTP header that will be used as the source for the Context Property value.
HTTP Status - Context Property value is extracted from the HTTP Status Code of the response message.
SOAP Header – Context Property value is extracted from the SOAP Header value. Specify SOAP Header Name property to identify specific SOAP header that will be used as the source for the Context Property value. Name property is specified in the format [namespace]:[name] , where [name] is a SOAP header name and optional [namespace]: is SOAP header namespace. Note that only the last ":" character separates [namespace] from [name] if [namespace] is needed, for example: http://tempuri:MyHeader
Query Parameter – Context Property value is extracted from the Query Parameter. Specify Query Parameter Name property to identify specific Query Parameter that will be used as the source for the Context Property value.
Relative URI – Context Property value is extracted from the relative URI of the Request message. Relative URI is the part that follows the address of the invoked service endpoint that the service is registered with. For example: if service is registered with http://myserver/myapi/v1_0 endpoint address and full request is http://myserver/myapi/v1_0/create/12345/789, then the Relative URI will be: create/12345/789. Relative URI string will be used as the source for the Context Property value.
Message Property – Context Property value is extracted from the WCF Message Property of the Message object. Specify Message Property Name to identify specific Message Property that will be used as the source for the Context Property value.
Message Body – Context Property value is extracted from the Request or Response message body.
Inbound Identity Claim – Context Property value is extracted from the claim that describes the identity of the virtual service caller. User must specify which claim to extract the value from by specifying the claim type. Claim type can be a standard or a custom claim type depending on the nature of the original identity token (SAML, JWT/OAuth, Username/Password, X.509 certificate, Kerberos, etc.). For example, user name identity in the JWT token issued by the Azure Active Directory OAuth server will be represented with the name claim type, while the same user name identity received over Basic Authentication security will be represented by the http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name claim type.
If you want to configure Context Property with Inbound Identity Claim Value Source and you do not know which claim types to expect, use Sentinet monitoring to record all claims with inbound identity and then use that knowledge to configure Context Property pipeline component accordingly (see Monitoring and Recording Control chapter for more details on how to record identity claims).
Context Property – Context Property value is extracted from some other Context Property's value. Specify Name property to identify the name of the other Context Property from which the value will be extracted for this Context Property.
Cache Property – Context Property value is extracted from a Cache Property’s value. Specify Name property to identify the name of a Cache Property from which the value will be extracted for this Context Property.
GUID – Context Property value is initialized with a new Globally Unique Identifier value (GUID).
Some Context Property Value Sources can be further configured with the Value Extractor. Value Extractor specifies how to extract the value for the Context Property from the Value Source.
Available Value Extractor options are:
String – Context Property is initialized with the full string content of the Value Source. For example: if Value Source is a SOAP Header, then String Value Extractor assigns Context Property a string value that is a complete XML representation of the whole SOAP Header.
Encrypted String – Context Property is initialized with the full string content of the Value Source, where string content is expected to be encrypted using Sentinet’s ProtectString.exe utility. Typically, this Value Extractor is used when Context Property is initialized with Constant Value Source with the intent to “hide” some secret from the Pipeline’s UI.
Figure. Example of MySecret Context Property initialized with encrypted string. ProtectString.exe utility is installed with Sentinet in its installation root folder (by default in C:\Program Files\Nevatech\Sentinet). It can be freely distributed and used on any computer that has .NET 4.8 framework installed with no dependency on the Sentinet installation on the same computer.
Note
Launch this utility from command prompt with no parameters to print its usage syntax and samples.
Important
Sentinet runtime and its ProtectString.exe utility can use either symmetrical encryption (default behavior), or certificates-based encryption. Symmetrical encryption does not represent the strongest security. It provides Encrypted String value extractor with some simple mechanism to “hide” secrets from being directly exposed by the Sentinet Administrative Console’s User Interface. Certificate-based encryption requires the use of ProtectString.exe.config configuration file (which usage is optional by default). Sentinet Node which decrypts protected strings at runtime must be configured with certificate containing its private key. Certificate must be installed on the machine where Sentinet Node is running and node’s IIS Application Pool must be given at least read access to certificate’s private key. Node’s web config must be configured with non-empty findValue in the protectionCertificate element:
… <protectionCertificate findValue="" storeLocation="LocalMachine" storeName="My" x509FindType="FindByThumbprint" /> …Important
On the machine where ProtectString.exe utility is running, only public part of this certificate must be installed, and configured with non-empty findValue in the protectionCertificate element in the ProtectString.exe.config configuration file:
… <nevatech.vsb.tools.protect> <protectionCertificate storeLocation="LocalMachine" storeName="My" x509FindType="FindByThumbprint" findValue="" /> </nevatech.vsb.tools.protect> …Regular Expression – Context Property value is initialized from the content of the Value Source using Regular Expression applied to the whole content of the Value Source.
XPath – Context Property value is initialized from the content of the Value Source using XPath. For example: if Value Source is a SOAP Header, then XPath Value Extractor may assign a Context Property a string value that is extracted from the SOAP header's XML representation using a specified XPath expression. Note, that for SOAP messages when Value Source is Message Body, XPath is applied to the SOAP Body XML element's content only.
URI Template – Context Property value is initialized from content of the Value Source using URI Template binding. This Value Extractor is most typically used as part of the Relative URI Value Source configuration. For example, consider that Request's message Relative URI at runtime is:
create/12345/789. If Source Template is defined as {segment1}/{segment2}/{segment3}, while Binding Template is defined as {segment2}, then the Context Property value will be extracted from the second segment of the Relative URI and initialized with the value 12345. Source Template and Binding Template must use curly brackets {} for parameterized URI elements as shown in the example above.
Source and Binding templates can be pre-tested against a sample URI to ensure correct definition. Expand the Test Templates section to provide the sample URI and to test the result of the extraction (see Set Relative Path component in this chapter for similar examples on how Source and Binding templates can be tested).
JSON Pointer – Context Property value is initialized from the content of the Value Source using JSON Pointer syntax. This Value Extractor is similar to XPath extractor by its purpose, except that it is applied to JSON strings (rather than XML) and uses JSON Pointer syntax described by this specification https://tools.ietf.org/html/rfc6901. For example: if Value Source is a Message Body, then JSON Pointer Value Extractor may assign a Context Property a string value that is extracted from the request's or response's message body using JSON Pointer syntax.
Trim Quotes check mark (if checked) instructs component to remove leading and trailing double-quote character from the resulting value if value needs to be trimmed off double-quotes. For example, if JSON fragment was { .... "x" : "y", ...}, then extracting value for "x" without trimming will be "y", but with trimming will be just y.
JSON Path – Context Property value is initialized from the content of the Value Source using JSON Path syntax. This Value Extractor is similar to XPath extractor by its purpose, except that it is applied to JSON strings (rather than XML) and uses JSON Path syntax described by this specification https://tools.ietf.org/id/draft-goessner-dispatch-jsonpath-00.html. For example: if Value Source is a Message Body, then JSON Path Value Extractor may assign a Context Property a string value that is extracted from the request’s or response’s message body using JSON Path syntax.
Trim Quotes check mark (if checked) instructs component to remove leading and trailing double-quote character from the resulting value if value needs to be trimmed off double-quotes. For example, if JSON fragment was { .... "x" : "y", ...}, then extracting value for "x" without trimming will be "y", but with trimming will be just y.
Built-in Context Properties
Sentinet offers a number of built-in Context Properties. They can be used anywhere in the pipeline by referencing them by their unique names:
Sentinet.ApiKey Context Property contains API Key associated with a Subscription to an API Product. The value of this Context Property is initialized only when request message contains an API Key.
Sentinet.InboundUserName Context Property contains inbound username identity if it is available.
Sentinet.InboundPassword Context Property contains inbound password for username identity if it is available.
Sentinet.TransactionId Context Property contains Id of the current message exchange transaction that is used by Monitoring and Logging.
Sentinet.ProductKey Context Property contains the Key value of an API Product that a particular message transaction is associated with. The value of this Context Property is initialized only when request message contains an API Key associated with a Subscription to an API Product.
Sentinet.AuthorizationRuleName Context Property contains the name of a Sentinet Access Rule that permitted access to a virtual service via its Access Control assignment.
Sentinet.ApplicationName Context Property contains the name of a registered in the Developer Portal application associated with Sentinet.ApiKey Context property described above. The value of this Context Property is initialized only when request message contains an API Key.
Sentinet.ProductName Context Property contains the name of an API Product. The value of this Context Property is initialized only when request message contains an API Key.
Built-in Context Properties can be used in other pipeline components using regular Context Property syntax, for example <{Sentinet.InboundUserName}> , <{Sentinet.InboundPassword}> , <{Sentinet.TransactionId}>.
Monitoring Properties component
This component can be placed anywhere in any pipeline and can be configured with one of more Monitoring Properties. A Monitoring Property is a name/value pair, where both name and value can include Context Property(ies) referenced by Context Property name. Sentinet provides automatic recording of these name/value pairs against the message exchange transaction allowing recording of any part or any context of the message transactions in a flexible scenario, including conditional recording. For example, only a specific part of the message will be recorded, but only if the caller identity or some other condition is met.
Figure above shows an example of 4 different monitoring properties configured within a single Monitoring Properties component. My Property 1 will be recorded with the constant value 123 against the message transaction. My Property 2 will be recorded with the value of My Context Property1 context property initialized somewhere else in the pipeline. My Property 3 will be recorded with the value of a custom string that includes My Context Property1 context property value. The last Monitoring Property has dynamically generated name initialized from the current value of My Property context property's value and the value of this monitoring property is initialized from the value of the My Context Property2 context property.
Monitoring Properties are recorded against message transactions independently of the current Monitoring Profile set for the virtual service with one exception: when the Monitoring Profile is set to None, no monitoring data is recorded against this virtual service including all Monitoring Properties configured in the pipeline.
Recorded Monitoring Properties can be viewed from the Properties tab of the recorded transaction (see Monitoring->Monitoring Properties chapter for more details).
Message exchanges can be searched for by the Monitoring Properties and their values (see Transactions Search chapter).
Cache Properties component
Cache Properties component can be placed anywhere in any pipeline. It is a special pipeline component, which is used to cache any custom data in Sentinet Nodes’ memory until cache expires. Most often custom data (Value of a Cache Property component) is initialized from a named Context Property. A single Cache Properties pipeline component can be configured with one or more individual Cache Property components. Cache Property is maintained for a specific Virtual Service Version; it does not span across different service versions.
A Cache Property component is configured with the following properties:
Property Name – specifies the name of a Cache Property (required). Property Name is the key in a Key-Value cached collection. It may include one or more Context Properties as <{Property Name}> tokens. For example <{Username}>:<{Password}> Property Name will create an element in a cache collection with the element’s key created from the combination of <{Username}> and <{Password}> Context Properties separated by colon symbol.
Property Value – can be a constant value, or a value of a Context Property.
Cache Expiration – expiration in seconds for this Cache Property. Cache Expiration can be specified as a constant value in seconds, and may also be initialized from the value of some other Context Property. If Cache Expiration is not specified, then this Cache Property does not have expiration.
Keep Existing Expiration – if checked and cached value with the same Key (Property Name) already exists, then cached value will change without changing its current expiration.
To remove Cache Property at run-time from the cache (if property with this name was set earlier), select Remove Property option.
Cache Property’s value can be accessed in a Pipeline via a Context Property. In this case, a Context Property is configured with the Value Source Cache Property where this Cache Property’s Name used as a cache Key. Effectively, this allows to use Cache Property anywhere in the Pipeline’s message processing and its workflow logic.
Another typical example of a Cache Property component is when a Pipeline makes a custom external call from the Pipeline using HTTP Invoke component, and stores something from its response in the Cache Property with the goal to use this response from the cache and to avoid making new external calls until Cache Property’s value expires. This is often used when external calls are made to a custom OAuth/OpenID Connect server to get and cache returned JWT token and when custom OAuth/OpenID Connect server is not described with exact industry standard covered out-of-the-box with Sentinet policies for OAuth/OpenID Connect.
Note
By default, Sentinet Node(s) use built-in, in-memory caching. For details on how to configure Sentinet Node(s) with distributed and/or persistent cache of a third-party product or a custom caching solution, see Appendix C of this User Guide.
HTTP Invoke component
HTTP Invoke component allows to make any external call from the pipeline to use its response for further message processing and/or modification. This component allows to create workflows that leverage external calls inside virtual service’s Pipeline.
Request section must specify HTTP Method and URL. Context Properties may be included as <{PropertyName}> tokens in these section’s fields, for example <{myMethod}>, or http://oauthserver/<{myproperty1}>?q=<{myproperty2}>.
Response section can be assigned the names (only the name) of the Context Property without open and close <{ }> tags, where Response’s properties (such as Status Code, HTTP Headers and Response Body) will be stored, for example myStatusCode, myResponseBody. Hence a user can control (assign) request’s HTTP Method, URL, HTTP headers and Body using either static values or dynamic values that leverage some Context Properties defined earlier in the Pipeline. Information captured from the Response can be stored in some other Context Properties that can be used further through the Pipeline.
Timeout in seconds specifies timeout for making external HTTP(s) call.
Exception Message Property Name specifies optional name of a Context Property that will hold the text of an exception that may happen during HTTP Invoke component’s execution.
If Continue On Exception is checked then execution of a pipeline will continue even if an exception happens during HTTP Invoke component’s execution.
Cache Request component
This component works together with the Cache Response component to provide response caching when a physical service is not expected to be called and the response is returned to the API client application from the Node’s cache. Cache Request component is responsible for creation of the Key for the request message. Later in the Response pipeline, that key will be linked and cached to the response object received from the physical service. If the Cache Request component identifies that both Key and cached by that key response message already exists, it will immediately return the cached response message either to the beginning of the Inbound-Response pipeline (where is can be further processed by this pipeline), or directly to the client application bypassing Inbound-Response pipeline processing (default behavior).
The cached response message is always keyed by the request message body if there is any (includes only BODY element for SOAP messages), URL and HTTP method (for RESTful messages), and Action header for SOAP messages. In addition, calculation of the cache key may include the Vary By properties such as SOAP header names in the form of namespace:localname, HTTP header names and user identity Claim type names.
Duration property specifies how long a keyed cache is maintained in memory before it is invalidated, forcing Sentinet to continue pipeline(s) processing and potentially call the physical service to update the cached response. Cache Request component can only be placed in the Inbound-Request pipeline.
Bypass Response Pipeline property specifies if cached response is immediately returned to the client application bypassing Inbound-Response pipeline processing (default behavior), or cached response message returned to the beginning of the Inbound-Response pipeline (where is can be further processed by this pipeline).
Note
By default, Sentinet Node(s) uses built-in, in-memory caching. For details on how to configure Sentinet Node(s) with distributed and/or persistent cache of a third-party product or a custom caching solution, see Appendix C of this User Guide.
Cache Response component
This component does not have any configuration. It is responsible for placing response message in a keyed request prepared by the Cache Request component. Cache Response component does nothing if no keyed request was created for it by the Cache Request component.
Important
Cache Response component’s position in the pipeline determines when Response from the physical service will be placed in the cache. If pipeline’s execution flow does not reach this component, the Response will not be cached. For example, if you place Response Cache component after conditional If component which checks response’s HTTP status code for value 200, then you may have a pipeline that caches responses only if they come from the physical services with HTTP status code 200.
Flow components
Flow components control the execution flow of the request message through the Sentinet pipelines. Two flow components are available; Conditional If and Stop as described below.
Conditional If component
Conditional If component provides capabilities to branch pipeline execution depending on certain conditions. For example, a specific string replacement component may be executed only for a specific operation, while for a different operation a different pipeline component will be executed. Conditional If components can be embedded within each other.
Component If configuration consists of two independent configurations:
Condition configuration
True and/or False branches with sub-pipeline configuration
Condition configuration is implemented in the same way as the Access Rule Graphical Designer configures Access Rule Expressions. The difference here is that the result of the condition evaluation affects the flow of the pipeline execution rather than the results of the Access Control.
Among other pipeline components, Conditional If component can also be configured with the pipeline's Context Property value that provides flexible and powerful capabilities to drive pipelines execution flow by the value(s) of the Context Property(s).
Either true or false branch of a condition must be provided with at least one pipeline component.
Stop component
Stop component instructs Sentinet to stop request execution flow and return execution to the beginning of the Response-Inbound pipeline (where is can be further processed by this pipeline – default behavior), or directly to the client application bypassing Inbound-Response pipeline processing. Effectively, this component provides Sentinet with the powerful capability to generate custom response messages by the virtual service without accessing the physical backend service. Stop component can be placed only in the Request-Inbound pipeline. The Stop component is often used in conjunction with a conditional If component to abort execution under specific conditions and then return a custom message (for example, a custom error message) to the client application.
Configuration properties of the Stop component are:
HTTP Status Code – specifies optional HTTP status code of the response message that will be returned to the Response-Inbound pipeline. You can select status code from the drop-down list, or enter a Context Property’s name, which value will be assigned to HTTP status code at runtime.
Status Description – specifies optional description of the HTTP status code of the response message that will be returned to the Response-Inbound pipeline.
Bypass Response Pipeline - specifies if pipeline’s execution flow returns to the beginning of the Inbound-Response pipeline (where response can be created and/or further processed by this pipeline - default behavior), or response message is immediately returned to the client application bypassing Inbound-Response pipeline processing.
Set Response Body - specifies if this Stop component provides response message’s body content.
Content Type - specifies Content-Type HTTP header for the response message’s body content.
Response Body - editable text box can be populated with the content of the response message body content.
If Stop component is configured with no Bypass Response Pipeline, then response message can be created or modified in the Inbound-Response pipeline.
Custom Pipeline components
Custom pipeline components represent the powerful Sentinet feature that augments Sentinet runtime with custom messages processing (see Sentinet Extensibility chapter for generic description of the Sentinet extensibility features).
Unlike WCF endpoints and service behaviors, custom pipeline components are much easier to develop and deploy. A custom .NET component is a .NET class that implements an IMessageProcessor interface defined in the Nevatech.Vsb.Repository.dll assembly:
namespace Nevatech.Vsb.Repository.Processing
{
public interface IMessageProcessor
{
void ImportConfiguration(string configuration);
MessagePipelineResult ProcessMessage(MessagePipelineContext context);
}
}
MessagePipelineContext received by the custom component allows full access to the message content and properties, and to modify the message content if needed:
namespace Nevatech.Vsb.Repository.Processing
{
public sealed class MessagePipelineContext
{
public MessagePipelineContext(Message message, MessageFlow flow);
public MessageFlow Flow { get; }
public bool IsOutbound { get; }
public RoutingEndpointDefinition Endpoint { get; private set; }
public bool IsRequest { get; }
public Message Message { get; set; }
public RoutingContextMessageProperty RoutingContext { get; }
public string GetMessageBodyContent(bool preserveOriginalMessage);
public void SetMessageBodyContent(string content);
public void SetMessageBodyContent(XmlReader content);
}
}
Complete Sentinet .NET API can be found in online API Reference documentation.
Developers can access the raw WCF Message (which is a Message property of the MessagePipelineContext class) to inspect and/or manipulate its content using .NET and WCF class libraries. Alternatively, Sentinet offers static MessageHelper class (see Appendix B for more details) to simplify access to most basic request and response message's properties (in addition to extended access via .NET WCF class library). Custom pipeline component must be compiled in .NET assembly, which may consist of many .NET classes, many custom pipeline components and can use other third-party .NET libraries (assemblies).
Note
Sentinet is shipped with Visual Studio .NET sample projects that demonstrate different Sentinet extensibility components. The projects are located by default in C:\Program Files\Nevatech\Sentinet\Samples folder.
A custom pipeline component placed in the Request-Inbound pipeline can stop pipeline execution, and for request-response operations, can return a custom Response message to the beginning of the Response-Inbound pipeline (short-cutting the call to the physical service) or it can immediately return the Response message to the client bypassing the Response-Inbound pipeline.
MessagePipelineResult enumeration value, which is returned from ProcessMessage(MessagePipelineContext context)method instructs pipeline to continue execution or to stop execution. Returned value has no effect on execution flow, when a custom component is not in the Request-Inbound pipeline.
namespace Nevatech.Vsb.Repository.Processing
{
public enum MessagePipelineResult
{
Continue = 0,
Return = 1,
Reply = 2
}
}
Before a Custom Pipeline Component can be used with the Pipeline Designer, it must be registered with the Sentinet Administrative Console. The registration process is similar to all other Sentinet extensibility components that use the Sentinet .NET API.
To add a Custom Pipeline Component, navigate to the Repository root tree item, select CONFIGURATION tab, then EXTENSIBILITY tab and click the [+ Add] above the Custom Pipeline Components table.
In the Add Pipeline Component dialog box, specify:
A friendly name for your Custom Pipeline Component.
Assembly that contains Custom Pipeline Component.
.NET class (Type) that implements Custom Pipeline by implementing Nevatech.Vsb.Repository.Processing. IMessageProcessor interface.
Mark a component as reusable if its code is thread-safe. The Sentinet runtime will create and use a single instance of this component every time it is required by a pipeline instance that needs to use that component.
Optionally specify valid positions for the custom pipeline component in any of the four service pipelines. For example, your custom component can be designed to execute only in the Request-Inbound pipeline, in which case registration may enforce that limitation in the component configuration screen.
Optional default configuration of the Custom Pipeline Component. Default Configuration field can be populated with optional configuration text (for example in XML format) that will be configured by default with the pipeline component once it is dropped in the Pipeline Designer.
Figure. Specifying valid positions for custom pipeline component.
Sentinet is shipped with the Visual Studio sample project, Nevatech.Vsb.Samples.CustomMessagePipelineComponent that demonstrates the development of a simple custom pipeline component. Readme.txt file of this project provides additional details describing the sample project.
Custom Pipeline components can access pipeline Context Properties by the Context Property name. For example:
public override MessagePipelineResult ProcessMessage(MessagePipelineContext context)
{
…
object value = context.RoutingContext.Properties["MyContextProperty"];
Pipeline Tracing
Pipeline can be enabled with the tracing logs. Tracing can be enabled for the whole Virtual Service and helps to monitor and troubleshoot execution of the pipeline for each message sent through the virtual service (see Tracing chapter on how to enable Pipeline tracing). Monitoring of the recorded Pipeline trace logs is described in the Pipeline Tracing chapter.