Custom connectors and Cosmos graph databases

As we have seen throughout this book, we can build integration workflows powered by Logic Apps, Microsoft Flow, or Power Apps without having any hand in the code. In this section, we will discuss how we can leverage existing APIs and make standard custom connectors to connect integration platforms and services, whether they are cloud resources or on-premises applications.

To get started with the custom connector, we have created a Gremlin API to connect with a Cosmos graph database using the Gremlin.Net library. To find out more about graph databases and the Gremlin query language, you can read through the Cosmos DB documentation available at https://docs.microsoft.com/en-us/azure/cosmos-db/graph-introduction.

The Gremlin custom API performs create, retrieve, update, and delete (CRUD) operations on graph nodes (documents) with the Gremlin query language. The following code blocks show the operation definitions for create, get, and delete graph database vertexes (nodes):

  • Method to create graph database vertexes (nodes): The PostComosGraphVertex operation creates a node document in a Cosmos graph database using the Gremlin.net driver. In this method, the client application (for example, Logic Apps) sends vertex content, which includes the type of vertex and a JSON string. Once the request is received through the post operation, Logic Apps iterates through the JSON string and creates a Gremlin query using the request data:
// POST api/GraphVertex Create Vertex with JSON string 
[HttpPost]
public async Task<ActionResult> PostCosmosGraphVertex(GraphVertex graphcontent)
{
try
{
var result = (dynamic)null;
dynamic content = JsonConvert.DeserializeObject<dynamic>(graphcontent.Content);
string type = graphcontent.VertexType;

StringBuilder builder = new StringBuilder();
foreach (var item in content)
{
string query = $".property('{item.Name}','{item.Value}')";
builder.Append(query);
}
var queryString = string.Concat($"g.addV('{type}')",builder);
result = await _client.SubmitAsync<dynamic>(queryString);

var responsemessage = JsonConvert.SerializeObject(result);
return this.Content(responsemessage, "application/json");
}

catch (Exception ex)
{
return Content(ex.Message);
}

}
  • The operation then calls the Gremlin client with the SubmiteAsync method and sends the Gremlin query to the Gremlin driver to create a record in the graph database. If there are any exceptions while processing the request message, the controller method will return the exception details to the calling application. 
  • Method to get graph database vertexes with properties (nodes): The GetVertexByProperty operation queries the graph database to find the node (documents) with a specific set of properties and values using the Gremlin.net driver. In this method, the client application sends vertex details in the form of a key and a value. The operation will send the key-value details of the node to the Cosmos graph database using the Gremlin query language and get the required node information from the graph database:
// Get Graph Node with key value Property

[HttpGet]
public async Task<ActionResult> GetVertexByProperty(string
Property, string Value )
{
try
{
var queryString = $"g.V().has('{Property}', '{Value}')";
dynamic result = await _client.SubmitAsync<dynamic>(queryString);
var responsemessage = JsonConvert.SerializeObject(result);
return this.Content(responsemessage,"application/json");
}
catch (Exception ex)
{
return Content(ex.Message);
}
}
  • Similar to the preceding method, to create a node in the graph database, the GetVertexByProperty operation uses the Gremlin client with the SubmiteAsync method and sends the Gremlin query to get the record document from the graph database. If there are any exceptions while processing the request message, the controller method will return the exception details to the calling application.
  • Method to delete graph database vertexes with properties (nodes): DeleteGraphVertex performs the delete graph node operation on the graph database. This operation requires a property set in the key-value pair to identify the record in the graph database. Again, this method uses the Gremlin.net driver to post a Gremlin query to the Cosmos graph database:
[HttpDelete]
public async Task<object> DeleteGraphVertex(string Property, string Value)
{
try
{
var queryString = $"g.V().has('{Property}', '{Value}').Drop()";
dynamic result = await _client.SubmitAsync<dynamic>(queryString);
var responsemessage = JsonConvert.SerializeObject(result);
return this.Content(responsemessage, "application/json");
}
catch (Exception ex)
{
return Content(ex.Message);
}
}
  • Like other methods in this graph web API, DeleteGraphVertex also uses the Gremlin client to send a Gremlin query to the SubmiteAsync method. If any exceptions occur when deleting the node from the graph database, the controller method will return the exception details to the caller. 

The PATCH operation is the same as the POST operation. It will overwrite the graph vertex property or create a new if none of the vertices are found with request data.

In the web API, we have defined the dependency injection to pass the Gremlin connection properties that are defined in the startup.cs file of the web API. The definition for Startup.cs contains properties and a method to retrieve the Cosmos graph database connection from Key Vault using application.configuration of the web API:

  1. The code to retrieve a Cosmos database secret from Azure Key Vault through the web API application.configuration file is shown in the following code snippet:
public class Startup
{
private static string hostname = System.Environment.GetEnvironmentVariable("cosmosDbHostaddreess");
private static string port = "443";
private static string collection = System.Environment.GetEnvironmentVariable("cosmosDbdatabasecollectionName");
private static string authKey = System.Environment.GetEnvironmentVariable("cosmosDbdatabaseAuthKey");
private static string database = System.Environment.GetEnvironmentVariable("cosmosDbdatabase");

private GremlinClient client;
}
  1. Once the Cosmos graph database secret has been retrieved from the application settings, the next step is to create a connection to the Cosmos graph database through the gremlin.net driver. To do this in the Startup.cs file, update the startup method to create a Cosmos DB connection using Cosmos graph database properties: 
public Startup(IConfiguration configuration)
{
Configuration = configuration;
var server = new GremlinServer(hostname, 443, enableSsl: true, username: "/dbs/" + database + "/colls/" + collection, password: authKey);

client = new GremlinClient(server, new GraphSON2Reader(), new GraphSON2Writer(), GremlinClient.GraphSON2MimeType);
}
  1. Finally, we need to enable the OpenAPI definition for the custom connector through a swagger definition. To do this, open the Startup.cs file and update ConfigureServices and the Configure method as shown: 
void ConfigureServices(IServiceCollection services)
{
services.AddMvc();
services.AddSingleton(client);
services.AddSwaggerGen(c =>
{
c.SwaggerDoc("v1", new Info { Title = "Cosmos Graph Database", Version = "v1" });
});
}

// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app)
{
app.UseSwagger();

app.UseSwaggerUI(c =>
{
c.SwaggerEndpoint("/swagger/v1/swagger.json", "My API V1");
});

app.UseMvc();
}
}
}

The build setup for the Gremlin web API is complete now. You can host your web API in multiple environments, including on-premise, on virtual machines, in a container, or an App Service in an Azure resource group.

  1. In this example, we have deployed our web API in an Azure resource group through the DevOps pipeline. When you browse to the OpenAPI definition of the Gremlin API, you can see the list of operations in the swagger definition file:

We will now create a custom connector for this web API:

  1. Search for logic apps custom connector and click on Logic Apps Custom Connector:

  1. Populate the custom connector's Name field, choose your Subscription, Resource group, and Location (choose a location near to your Logic Apps instance), and click on Create. This will open up the definition section for the Logic Apps custom connector:

  1. Choose your API endpoint. As we have our web API running over the REST protocol, we have chosen REST and imported the swagger definition. You can also use the OpenAPI URL or collection details from Postman. Based on the method of import definition, you will be prompted to use a collection, URI, or swagger definition:

  1. In the next section, you can list the Gremlin API image along with the description. As we've hosted our API in Azure, we have not ticked Connect via on-premises data gateway. If you are hosting within your own infrastructure, then you can connect your API through the API gateway. We have also chosen HTTPS as the Scheme for our web API invocation. It's important to list the web API host to appropriately identify the HTTP endpoint:

  1. In the security section, you can choose a security model. In this custom connector, we have chosen OAuth 2.0 with Azure Active Directory as Identity Provider. To do this, you need to register an application with Azure Active Directory and provide management access to the Azure resource:

  1. You need to update the Security section with the client ID, also known as the Application ID, and the security key that is generated during app registration. Once saved, the custom connector will generate a redirect URI, which needs to be updated:

In this case, the custom connector definition section will use information from the OpenAPI definition of the swagger file, or you can also manually enter the properties of the custom connector:

  • As we have already used swagger, we can see the list of available operations in the custom connector. We will only update the basic information, such as the summary and description.
  • Now you need to close the custom connector configuration. Within a couple of seconds, you'll see your custom connector listed within Logic Apps for CRUD operations, along with other Microsoft built-in connectors:

  • To test the connector's behavior, create a recurrent Logic Apps instance to find the vertex details with a property name and value:

When working with real use cases, it is advisable to lock your web API with a Logic Apps IP range, or you can implement mutual OAuth authentication both on the web API and the custom connector to provide an extra layer of protection.

In this section, we have discussed how to build and leverage custom connectors for Azure Logic Apps and how to leverage the custom connector definition with a Logic Apps workflow. As we discussed in the previous chapter, logic apps and Azure Functions communicate through HTTP endpoints and webhooks. You can also enable the OpenAPI definition in Azure Functions to work efficiently within a function app. To find out more about how you can enable an OpenAPI definition in a function app, you can read through the following resource: https://docs.microsoft.com/en-us/azure/azure-functions/functions-openapi-definition.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.142.171.253