Services and APIs discovery and description has always been the focus of APIs and integration architectures. Cloud, mobile applications and exponentially growing number of REST APIs, drive the need for services discovery and description as never before. Organizations that build services and APIs are looking for ways to make their services easily available with complete and up-to-date metadata and documentation by offering developer portals and service Repositories. Enterprises that build EAI integration solutions want to know where their services are deployed, how they are deployed, and how they are transitioning through their life-cycle and versioning.
This article briefly describes how services and APIs discovery concept evolved over the last decade, and what this concept is really all about in the modern IT landscape.
Years ago, service discovery term was typically used in the context of automated service discovery when SOAP services deployed in a particular environment could be automatically discovered by the software agents and auto-registered in UDDI Registries or SOA Repositories. The ultimate outcome of the discovery was the expectation that services metadata including service address and associated security policies will be auto-registered and become available in the Repository. And of course, the expectation was that any service (regardless of the technology used to implement and deploy service) must be discoverable as soon as it is deployed. Those days many enterprises considered services auto-discovery to be a “must” feature for their SOA infrastructures.
Fundamentally there are two approaches for auto-discovery implementation:
1. Services are specifically deployed to be discoverable in the first place (for example, services send “announcement” messages during startup to a discovery agent, or they are deployed with the special “discovery” endpoints - see examples here). Even though this approach might have some value under certain conditions, it would not be really an auto-discovery because those services that are not deployed in a special “discoverable” way, will not be discovered at all. That’s why this approach is not discussed in this article from most of the practical use case scenario perspective.
2. Agents embedded in application servers search for deployed services using some search criteria.
Over the years the concept of automated service discovery faded away simply because in today’s state of technologies the auto-discovery mechanism (even if it existed) would highly depend on the specific technology used to develop and deploy services. For example, Java SOAP services can be discovered only by knowing specific deployment mechanism(s) offered by specific Java Web Services vendor, while Microsoft SOAP services can be discovered only by knowing how Microsoft services are deployed.
Moreover, service implementation vendors (like Microsoft) may offer multiple ways of developing and deploying services. For example, Microsoft ASMX services are deployed in IIS server with .asmx files, while WCF services are typically deployed with .svc files, and WF (workflow) services might be deployed with .xamlx files. In addition, WCF and WF services can be deployed outside of IIS (self-hosted services) and with no particular extensions or even files themselves associated with the service. Modern security best practices often recommend not to emit WSDL metadata at all (unless service metadata is meant to be publicly and openly available to anybody). How would you even define what service auto-discovery is, if service metadata is not even provided by the service? And what is expected to be discovered about the service when it does not “tell” anything about itself and behaves as an invisible ghost? A more practical scenario of course, is when service metadata is meant to be available, but only to select authenticated and authorized business partners and registered users. This scenario will still be a showstopper for third-party “discovery agents” because they would all have to be registered with all deployed services as authorized metadata discovery “users” (and that will again defeat the purpose of auto-discovery of any service in the first place).
Lastly, consider modern RESTful services as opposed to SOAP services. How would the software agent “discover” them, and what does it actually mean to discover a RESTful service? Does it mean that RESTful service’s “documentation” is supposed to be discovered by an agent? Well, some vendors offer some form of “documentation” in WADL or their own proprietary formats, but this is always a “custom” and vendor specific solution, it cannot possibly be covered by the agent that is meant to discover any REST API.
Ultimately, discovery term transitioned into a generic capability to access some central Repository where services and all other associated artifacts (like documentation for REST services) can be registered in and discovered from. There is no magic here, and this approach requires Repository owners to manually maintain the content of the Repository consistent with the actual service deployments (for example by re-uploading service WSDL when WSDL changes, or updating RESP API documentation).
Services and API portals has very specific benefits:
1. Assurance that discovery process is very generic. Any service (SOAP or REST) regardless of its implementation details and deployment model can be discovered.
2. Developer portals and Repositories can easily control security access to discovered services - can a service be discovered by anybody (public service) or only by specific authorized Repository users?
3. Using Repository allows to discover virtually everything about the service, not just the WSDL (that does not even exist for non-SOAP services), but many other artifacts. For example documentation, pricing, service owner, etc.
4. Repository can easily maintain and manage discoverable service versions and their life-cycle states.
Pure UDDI Registries also faded away over the years because UDDI spec does not provide easy means to accommodate all the capabilities above - the UDDI spec was originally designed for public SOAP services. That’s why today vendors either provide hybrid UDDI Registries (that are highly extended with private interfaces and capabilities beyond UDDI spec), or non-UDDI generic SOA/API Repositories that can be accessed via vendor specific but well defined SOAP and REST interfaces.
Here are few other critical capabilities of modern SOA and API Repositories that are closely related to the discovery concept:
1. Can Repository 100% accurately describe service from its WSDL representation, or in other words, can service be 100% accurately discovered from the Repository? For example, most of the Java-based Repositories would not even understand Microsoft-specific WSDL extensions, which would leave Microsoft users of this Repository powerless against their needs to maintain services in the Repository with all their Microsoft-specific attributes and security model descriptions.
2. Does Repository support the concept of service and APIs discoverable versions?
3. Does Repository provide support for and capability to manage services life-cycle?
So, turning back to the title of this article, what are (in short statements) the Myths and Reality of Services and APIs Discovery and Description?
Myth: Services and APIs auto-discovery mechanisms exist in a modern, universal and practical form.
Reality: Ever growing need for Services and APIs discovery can be (is) provided by means of using Services and APIs Repositories and developer portals.
Sentinet SOA Repository and Sentinet Administrative console (browser-based portal) https://www.nevatech.com/sentinet/overview from Nevatech provide a good example of such generic and capable SOA and APIs Repository.