Goal

To be able to understand the Rest API and learn how to configure them to support creation, retrieval and search of Super Heroes.

Starting code

Start tag: lesson/13

Background

Verne employs Swagger OpenAPI Specification to provide interfaces to perform business services (registration, view, maintenance and search) via a Rest API channel. You can read the full documentation about the Open API from the Swagger website, giving you a good overview of what the Rest API is and how to configure using the Open API in general. 

Note that an understanding of Open API is a prerequisite for this lesson.

Verne extends Open API to expose the Rest API schema. Verne processes incoming JSON requests and returns JSON responses after Verne processes the request. 

This lesson mainly focuses on the Verne extension to describe how to implement the Rest API requirements via the standard configuration. The rest of the lesson describes how to configure Rest API interfaces for the registration, search and view in the Super Hero.

Preparation

To expose business services via the Rest API channel, you need to create one YAML file, list all Rest entry points for the Super Hero and create one XML file to register the new YAML file. Both of them need to be placed under a folder called "openApis". Here is an example of what it looks like.

The name of the yaml file and xml file can be anything, but it is recommended to give it a meaningful name. Let’s call them heroesApi.xml and heroesApi.yaml for the lesson.

heroesApi.xml

This is the main file where you can register a new YAML file as an open API service. You can specify the location of the yaml file in the attribute specificationResource. Here is the configuration for registering the new yaml file you have created.


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<catcfg:Configuration xmlns="http://www.fostermoore.com/schema/cat-ng"
                      xmlns:catcfg="http://www.fostermoore.com/schema/catcfg-ng">
  <openApiServices>
    <openApiService name="heroes-api" specificationResource="/openApis/heroesApi.yaml"/>
  </openApiServices>
</catcfg:Configuration>

You can define many open service APIs to support different versions over time.

heroesApi.yaml

This is the main file that defines the Rest API interface with entry points and schema. It follows the OpenAPI specification. If you are not familiar with the syntax, you can refer to the Swagger OpenAPI Specification documentation. 

Open API has many root level objects you can specify. This lesson only covers those core root objects required to implement a fully functional Rest API for the lesson. It includes info, servers, paths and components.

info

This section describes the general information about the Rest API, including the title, version and description. Before the info section, there is one property called openapi with version 3.0.1. This is the version of the OpenAPI specification, not the version exposed to the Rest API clients. The version exposed to the client is defined inside the info section as below:

openapi: 3.0.1
info:
  title: Heroes Open API
  version: 1.0.0 # This version is used to support the versioning in the service form.
  description: Restful APIS for the Heroes Verne Register for Demo.

Note the version number inside info can be used to support multiple versions of Rest APIs, and the version number will be available to the application layer during the marshalling process so that different rules can be applied or different data can be captured depending on a version. An example would be to capture newly introduced mandatory data due to some legislation change after the initial rollout.

servers

This section describes the list of server URLs that will serve the Rest API requests. Here is the configuration of a single server for the lesson.

servers:
  - url: http://localhost:8080/heroes/rest/heroes-api/v1
    x-verne:
      pathPrefix: /v1

The url value above is a default value for the local environment and cannot be used in a production environment. To address the issue, Verne has introduced an extension property pathPrefix under x-verne. Once the value is specified, the system will derive the server URL dynamically based on the host value stored in the incoming HTTP request header and append the path prefix defined here to build a full URL.

components

This section defines various schemas for the specification. Note that all objects defined within the components section will not affect the API unless they are explicitly referenced from properties outside. There are two main sections:

  • securitySchemes: To specify the authentication method of a request
  • schemas: To specify the data structure of registry data

The lesson will focus on discussing the schemas section in details.

Schemas can be manually defined or can be auto-generated, or a combination of both. The schema auto-generation is not part of the Open API implementation, and Verne supports it via an x-verne extension. 

Here is an example of a schema definition that contains manual schema objects (HeroSearch, HeroSearchResults and HeroResult) used for the search Rest API and the x-verne configuration to enable the schema auto-generation from the Super Hero data structure.

components:
  x-verne: # triggers generation of schemas and replacing all of them below
    generate: true
    serviceCode: heroSchemaWs
    rootSchemaName: Hero
  securitySchemes:
    basicAuth:
      type: http
      scheme: basic
  schemas:
    HeroSearch:
      type: object
      x-verne:
        remove: false
      properties:
        SearchResults:
          $ref: '#/components/schemas/HeroSearchResults'
    HeroSearchResults:
      type: array
      x-verne:
        remove: false
      items:
        $ref : '#/components/schemas/HeroResult'
    HeroResult:
      type: object
      x-verne:
        remove: false
      properties: 
        identifier:
          type: string
        name:
          type: string

To enable the schema auto-generation, you need to set the generate to true and specify the business service for the schema generation via serviceCode and the root domain in the business service form via rootSchemaName under the x-verne section as above. You might also notice that those schema objects manually defined have the property remove set to false via an x-verne extension. This is to ensure they are intact when the final schema is generated.

Below is the business service definition used for schema generation. It is basically the same as other business services, except it doesn’t have permission settings. This will allow the system to access the business service to generate a schema-based form configuration. Make sure to create an XML file heroSchemaWs.xml under the heroes/services folder.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<catcfg:Configuration xmlns="http://www.fostermoore.com/schema/cat-ng"
                      xmlns:catcfg="http://www.fostermoore.com/schema/catcfg-ng"
                      xmlns:platform="http://www.fostermoore.com/schema/catalyst-platform/v1"
                      xmlns:component="http://www.fostermoore.com/schema/catcfg-ng"
                      xmlns:slot="http://www.fostermoore.com/schema/catcfg-ng">
  <businessServices>
    <businessService code="heroSchemaWs" mode="Create" formItem="heroApi">      
    </businessService>
  </businessServices>
</catcfg:Configuration>

Here is the form definition used for schema generation. It uses three existing components component:HeroDetails, component:Confidants and component:Sidekicks. These are the same components used to build UI forms. Make sure to create an XML file heroApi.xml under the heroes/forms folder.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<catcfg:Configuration xmlns="http://www.fostermoore.com/schema/cat-ng"
                      xmlns:catcfg="http://www.fostermoore.com/schema/catcfg-ng"
                      xmlns:platform="http://www.fostermoore.com/schema/catalyst-platform/v1"
                      xmlns:text="http://www.fostermoore.com/schema/catalyst-text/v1"
                      xmlns:component="http://www.fostermoore.com/schema/catcfg-ng"
                      xmlns:slot="http://www.fostermoore.com/schema/catcfg-ng">
  <serviceFormItems>
    <record shortCode="heroApi" domain="Hero">
      <component:HeroDetails/>
      <component:Confidants/>
      <component:Sidekicks/>
    </record>
  </serviceFormItems>
</catcfg:Configuration>

Once you define the business service and the form, you can check what the generated schema looks like in two different ways, either via Swagger OpenAPI console or via Magellan.

To access the Swagger OpenAPI console, you need to log on to the Super Hero with the username ‘External01@fostermoore.com‘ and password ‘External01’; you will see the menu item OpenAPI’s under the System Operations menu. Here is the Swagger Open API console generated from our Super Hero specification you have defined above.

The console has four main interfaces.

  • A dropdown list at the top right corner shows all specifications defined under the system with the application code prefixed.
  • A hyperlink at the top left corner under the screen title Heroes Open API. If you click on the link, you will see the final version of the schema generated in YAML format. This is what will be exposed to Rest API clients.
  • A list of entry points in the middle section. At the moment, it just displays No operations defined in spec!. This is normal as you haven’t defined any operation entry points. Once you add them later on, you will see those appearing in the section.
  • A list of schema objects at the bottom half of the screen showing schema objects either manually defined or auto-generated. You can drill down to its details to see all fields available for each object, including children.

The screen below shows the full list of schema objects either manually defined or generated from your specification. The first three objects HeroSearch, HeroSearchResults and HeroResult are the ones you have specified in your yaml file and the rests are the ones generated through the schema generation via x-version extension under components section.

Another way to access the generated schema is via Magellan. If you right mouse click on a YAML file, it will show you a menu item called Show Open API Spec as shown below:

If you click on the menu item, You will see the schema’s final version, including auto-generated schemas as below. This is basically the same schema you saw via the Swagger interface.

If you want to see insight into how this schema is generated from the schema business service you specified in your YAML, you can use the menu item Show Open API Schema Generation. It will show you how each attribute, domains and repeaters are mapped to the generated schema. Here is a sample screen:

In the screenshot above, it says 25 schema mapping(s) found. This includes attributes highlighted in green and records and repeaters in blue. You might also notice that some of the attributes are excluded from the generation. This is because those are either hidden or inactive in the tree.

The schema object name can be customized using a property webServiceName in your service form item configuration. Otherwise, it will be derived based on the value of shortCode on a service form item. Here is an example of a webServiceName name specified in a repeater

<repeater shortCode="superPowers" min="1" max="5" widget="pillbox" textKeyPrefix="${textKeyPrefix}" webServiceName="superPowerList">
</repeater>

The generated schema name in the example above will be superPowerList instead of superPowers.

paths

This section covers the available paths and operations for the API. You can define multiple paths. Each path is appended to the URL defined at the servers section to construct the full URL for each operation.

Create a Super Hero

Let’s start with the Super Hero registration. Here is the configuration of an entry point using the post method to create a new Super Hero.

paths:
  /hero:
    post:
      summary: Creates hero
      operationId: heroRegister
      x-verne:
        serviceCode: heroRegister
        formCode: heroApi
        action: submit
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Hero'
      responses:
        '201':
          description: created new hero
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Hero'

Note that there are four operation methods available for each path as described in the Swagger Open API.

  • post: To perform an insert operation
  • put: To perform an update operation on an existing record
  • delete: To perform a delete operation
  • get: To retrieve data

Since you are creating a new Super Hero, you need to use the post method. The operation entry point with the post method has a request body and responses. On top of them, it has an x-verne extension to support the mapping from an incoming JSON request to the registration business service and back to the JSON response. Here is the full description of properties under the x-verne section:

  • serviceCode: To specify a business service code to perform the registration. You can use a separate business service from UI, or you can use the same business defined for the UI. The lesson here uses the same business service created for the UI. This is a mandatory field.
  • formCode: To specify a form code to handle the mapping of an incoming JSON request and generate a JSON response. It is an optional field, in which case the default form specified in the business service configuration will be used. In this lesson, you will use a separate form created for the Rest API channel. It makes sense to use the same form as the one used for the schema generation earlier in the components section.
  • action: To specify what action to perform in the form. Currently, submit is the only action available. This is basically the same as when you press the Submit button on the UI form.

In the requestBody section above, there is a schema reference field $ref. The reference value should be the form of #/components/schemas/<SCHEMA_OBJECT_NAME> where SCHEMA_OBJECT_NAME is the name of the schema object. You need to use a valid schema object name shown in the Swagger Open API console. This might be the one you manually defined in your YAML file or the one generated automatically. The reference value used for the registration here is ‘#/components/schemas/Hero‘ and the schema object name ‘Hero‘ is one of the schema objects generated automatically through the schema generation.

The registration entry point makes sense to set it to the root schema object in the exposed schema.

Similarly, the response section has the reference field with the same value used in the request body. This reference will generate a JSON response mapped from the service transaction created to process the incoming JSON request.

If you open the Swagger OpenAPI console, you will see a new entry point appearing in the screen as below:

In the request body section, it shows a sample JSON request created from the schema. You can modify the sample JSON and use the Try It Out button to submit your first request. Here is an example of JSON data used for the lesson.

{
  "name" : "Difyzptsvs",
  "secretIdentities" : [
    {
      "firstName" : "Secret",
      "middleNames" : "I",
      "lastName" : "Dentity"
    }
  ],
  "yearOfArrival" : 2001,
  "costumeYn" : true,
  "capeYn" : false,
  "superPowers" : [
    {
      "name" : "agility"
    }
  ],
  "confidants" : [ ],
  "sidekicks" : [ ]
}

If you submit it, you will see a response with the code 201 (successful) and a JSON response as below:

Note that if you are getting a ws.rest.api.unauthorized in the JSON response when submitting the request, this means your browser session has expired, and you need to log in and resubmit the request.

If you want to see the incoming request and how it has been processed, you can access the request via the Recent Requests menu item under the System Administration functions (System Operations > System Administration). Note that the Recent Requests menu is only available in the developer environment. Here is a screenshot of recent requests when a Rest API request has been received.

The Recent Request screen monitors all incoming requests coming into the application. This includes UI and Rest. You can check it via the Channel column. In the example above, you see one Rest API request received and processed successfully. In the Service column, it shows the Rest request Uri and the method name. It also shows you the business service used to serve the request and a link to the service transaction created. If you click on the link, you will see the service transaction with incoming JSON mapping details.

If you click on the REST tab on the right-hand side, you will see the JSON request and response.

There is also a button called Show Form to show the service transaction in the UI form. Here is the form generated from the service transaction.

Note the form above is not a valid UI form exposed to the online client. It is a form rendered from a service transaction created from the incoming Rest API request to help developers and testers check the data capture easily.

This lesson has covered the happy path scenario. How about invalid cases? Say you remove some of the mandatory fields from your JSON request. Here is a JSON request with three mandatory fields missing (name, firstName and costumeYn).

{
  "secretIdentities" : [
    {
      "middleNames" : "I",
      "lastName" : "Dentity"
    }
  ],
  "yearOfArrival" : 2001,
  "capeYn" : true,
  "superPowers" : [
    {
      "name" : "agility"
    }
  ],
  "confidants" : [ ],
  "sidekicks" : [ ]
}

If you submit it, you will see a response with the code 400 (fail) and a JSON response with three errors listed in the response as below:

Note that in the errors sections above, code and message above are the same ones used by UI. The path is the JSON path in the request.
If you look at the recent Requests screen request, you will see a failed request highlighted in red with several errors.

If you open the failed service transaction, you will see all errors highlighted in red.

If you click on the Show Form button, you will see the same UI form errors.

The Super Hero registration (happy case and invalid case) has been covered. Let’s move on to retrieving a Super Hero.

Retrieve Super Hero

Retrieving a Super Hero is different from the registration in that it doesn’t need any request body. Instead, it needs to specify the Super Hero identifier that was created after the registration. Here is the configuration of an entry point for retrieval.

  /hero/{businessIdentifier}:
    get:
      summary: Retrieves hero
      operationId: heroView
      x-verne:
        serviceCode: heroView
        formCode: heroApi
        identifierParameterName: businessIdentifier
      parameters:
        - name: businessIdentifier
          in: path
          required: true
          description: Identifier of the hero (Business or Logical)
          schema:
            type: string
      responses:
        '200':
          description: view of a single hero
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Hero'

The URI for retrieval consists of /hero/ followed by a place holder for an identifier ({businessIdentifier} in this case). The get method is used instead of the post. The get method has a parameters section where you can define a list of parameters. In this case, there is one path parameter, ‘businessIdentifier’, defined as a type string. 

The name of the parameter must match the parameter used in the URI. There is a responses section that is the same as the one used by the creation. Let’s discuss those parameters under the x-verne extension.

  • serviceCode: To specify a business service code to perform the retrieval. You can use a separate business service from UI, or you can use the same business defined for the UI. The lesson here uses the same business service created for the UI. This is a mandatory field.
  • formCode: To specify a form code to generate a JSON response. It is an optional field, in which case the default form specified in the business service configuration will be used. Note the lesson uses the same form, ‘heroApi’, for the registration and the view. The same form will be used to perform maintenance business services later on.
  • identifierParameterName: To specify the name of an identifier parameter in the URI.

If you add the entry above to your YAML file, you will see a new entry appearing in the Swapper Open API console as below.

You can test it out using the Try It Out button. You can use the business identifier returned from the registration. Enter the business identifier in the parameter section and process Execute button. You will get a response with status 200 with the Super Hero details in JSON format.

You have learned how to configure entry points for the registration and the retrieval of a Super Hero. Let’s move on to the Search Super Heroes.

Search Super Heroes

The search operation is quite similar to retrieve operations in that they retrieve records. The search operation can retrieve many records based on the search criteria, whereas the retrieve operation can retrieve a single record with a business identifier. You can use the search business service created for UI, or you can define a separate business service purely for Rest API. The general recommendation is to reuse the online business service to share the same business rules and less code maintenance. The lesson here uses the same business service. Here is the configuration of an entry point for the search.

  /heroes:
    get:
      summary: Search heroes
      description: This is the search API to allow one to search the heroes register
      operationId: heroSearch
      x-verne:
        serviceCode: heroSearch
        actionExecution:
          ruleReference: hero.ws.search.execute
      parameters:
      - name: query
        in: query
        required: false
        description: Search query
        schema:
          type: string
      responses:
        '200':
          description: search results
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HeroSearch'

The entry point above has one optional query parameter query to capture search criteria. You can define many parameters to support advanced search. This lesson focuses on one parameter to give you a basic concept of how the parameter is mapped to a business service search field. 

If you look at the schema reference under the responses section, it points to HeroSearch that you have manually defined in the schemas section under components. On top of the standard Open API spec, Verne provides additional properties via x-verne to allow you to implement a custom rule to perform a search. Here are descriptions of additional properties.

  • serviceCode: To specify a business service code to perform the search. This is a mandatory field.
  • actionExecution.ruleReference: To specify the name of a custom rule to perform the search. This is a mandatory field.

The rule reference ‘hero.ws.search.execute’ for the action execution needs to be configured in ruleReference.xml as below. If you already have the file, you can add the following configuration into the file.

     <ruleReference name="hero.ws.search.execute">
      <rule type="groovy">
        log.warn("Hero Rule hook: WS search execute")
        serviceTransaction.domainTree.attributes['Name'] = appCtx.request.getParameter('query')
      </rule>
      <rule resourceName="/components/search/ElasticSvcTxnSearchHandler.groovy" methodName="search"/>
    </ruleReference>

If you don’t have the ruleReference.xml, create the file under heroes folder and replace the content with the following configuration.

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<catcfg:Configuration xmlns:catcfg="http://www.fostermoore.com/schema/catcfg-ng"
                      xmlns="http://www.fostermoore.com/schema/cat-ng">
  <ruleReferences>
     <ruleReference name="hero.ws.search.execute">
      <rule type="groovy">
        log.warn("Hero Rule hook: WS search execute")
        serviceTransaction.domainTree.attributes['Name'] = appCtx.request.getParameter('query')
      </rule>
      <rule resourceName="/components/search/ElasticSvcTxnSearchHandler.groovy" methodName="search"/>
    </ruleReference>
  </ruleReferences>
</catcfg:Configuration>

The groovy rule above reads the Rest API request’s query parameter and assigns it to the Name attribute in the search service transaction. It then calls another groovy rule to perform an actual search.

You can add more custom logic here to implement the requirements.
If you open the Swagger API console, you will see a new entry point created for the search

If you click on the Try It Out button, enter the Super Hero name ‘Difyzptsvs’ in the query field and execute the search, you will get a search result with the status code 200 as below.

If you look at the search result section SearchResults above, you might notice that it contains one matching record, but it has no details in it. This is because the lesson uses the existing search business service configured for UI users only. It needs to be changed to accommodate the Rest API search requirement. Let’s open the heroSearch.xml under the forms folder and see the search result section as below:

If you look at line 31 (slot:searchResult), it includes the search header. By default, it doesn’t expose any attributes. You need to extend it to expose attributes in the search result for the Rest API call.

Replace the following code

        <slot:searchResult>
          <component:SearchHeader targetServiceMode="View"/>
        </slot:searchResult>

with

    <slot:searchResult>
      <component:SearchHeader targetServiceMode="View">
        <box shortCode="restResult" visible="false">
          <platform:set-property visible="true" when-has-channel="rest"/>
          <attribute attribute="identifier"/>
          <attribute attribute="Name"/>
        </box>
      </component:SearchHeader>
    </slot:searchResult>

The configuration above has an additional box restResult hidden by default and uses a platform rule to make it visible when the channel is rest. The box contains two attributes identifier and name. These will be included in the JSON response for the Rest API search. If you perform the search again in the Swagger Open API console, you will see the same search result with identifier and name populated as below.

Wrap up

These lessons covered:

  • Rest API with the Open API specification with four main sections: info, servers, components and paths.
  • How to set up the schema generation
  • How to create an entry point for creating a Super Hero
  • How to create an entry point for retrieving a Super Hero
  • How to create an entry point for searching Super Heroes
+1
+1
Previous
Lesson 12: Relationships & privacy
Next
Configuration Best Practises

Jump to Section