FlexService Runtime

The FlexService Runtime (FSR) is a Node.js serverless runtime environment for Kinvey Flex services. It allows you to deploy your FlexData, FlexFunctions, or FlexAuth services in the cloud, on Kinvey-managed infrastructure. Such Flex services are called internal Flex services.

If you want to deploy your Flex service elsewhere, you need to deploy it as an external Flex service.

Preparing Your Environment

Before you can write and deploy Flex services, you need to create yourself a Node.js environment.

• Flex allows you to develop your project against several Node.js versions. Check the Runtime Selection section for the Node.js versions that you can choose and then install the one you want in your development environment.
• Install NPM (Node Package Manager).
• Install Kinvey CLI which you'll use to deploy and manage the service. Go to the Kinvey CLI GitHub page for installation instructions.

We recommend installing Node.js and NPM using NVM as it allows installation and usage of multiple versions of Node.js on the same machine.

Creating an Internal Flex Service

Creating a Flex service includes several steps, from configuring Kinvey Console to setting up a Node.js project. After you complete all the steps, you will have a Flex service ready to run your server-side code.

Setting Up Kinvey Console

Before you can deploy your Node.js project, you need to provision a Flex service using the Kinvey Console. The Flex service makes the connection between your Node.js project and the Kinvey backend.

Take the following steps to provision a Flex service:

1. Log in to Kinvey Console.
2. Select the Service Catalog tab and click Add a service.
3. Select Flex and then Flex Services Runtime.

4. Complete the form and then click Save.

   • Name—Arbitrary name for the service.
   • Description—Optional details about the service.

   • Scope/App—App or organization that will use the service.

     At this point, you will have a service created. The Console will start a new service environment for you, which is required for the service to be usable.

5. In the Connection Options tab of the service environment, enter a value for the following option or accept the default:

   • Secret—A passphrase that you can optionally specify in your Node.js project as an additional server-to-server authentication step. The recommended minimum length is 265 bits.

6. Click Save to create the service environment and complete the service.

After completing these steps, the new Flex service appears in your Service Catalog.

Initializing a Node.js Project

Underneath, each Flex service is a Node.js project. Choose your preferred method to create the project. The following steps use NPM for that purpose.

Take these steps to initialize a Node.js project:

1. Create a directory to contain your project files and enter it.

2. Initialize a Node.js project using NPM:

    npm init

   This utility guides you through setting your Node.js project settings and finally writes a package.json file. Flex does not pose any particular requirements on the settings' values; accept the default values if you are in any doubt.

3. Add the Flex SDK as a dependency in package.json:

    npm install kinvey-flex-sdk --save

4. In the project directory, create an empty text file and name it after the entry point that you specified during the npm init execution.

   For example, if you accepted the default value, create a file called index.js.

   The Flex service will load this file first when executing your code.

Initializing the Flex SDK

After you create the file structure needed for the project, you can start adding code. The Flex SDK is the only mandatory piece. Aside from it, you can require any npm module you need for your project with the exception of modules with external dependencies because they will not be available on the FSR.

To use the Flex SDK, require it in your project:

const sdk = require('kinvey-flex-sdk');

Then initialize the SDK to retrieve a reference to the backend service:

sdk.service((err, flex) => {
  // code goes here
});

When running locally, you can specify a host and port to listen on by passing an options object with an optional host and port. If no host and port are specified, localhost:10001 is used:

sdk.service({ host: 'flex-machine', port: 7777 }, (err, flex) => {
  // code goes here
});

Securing your Flex Service

Communication to Flex services running on FSR are encrypted by default. In addition, Flex services can be secured by use of a shared secret which serves as server-to-server authentication.

All requests to a Flex service from Kinvey include the shared secret to prevent unauthorized access to the service. When configuring your service in the Kinvey Console, it suggests a secure, randomly generated string. You can accept it or enter your own (256-bit minimum recommended).

To enable the shared secret within your project code, pass the shared secret into the Flex SDK initialization method:

sdk.service({ sharedSecret: <my shared secret> }, (err, flex) => {
    // service code
})

If the shared secret is specified within your project code, the service rejects any request that does not include the shared secret key.

For local testing, you can pass the shared secret in the X-Auth-Key HTTP header.

Deploying Flex Services

Kinvey CLI is the utility that lets you manage Flex services on the FSR and is the only way to deploy them. Before you can deploy a Flex service, you need to configure Kinvey CLI.

For the complete documentation of Kinvey CLI, see its GitHub page.

Configuring Kinvey CLI

The fastest way to get started is to run the kinvey init command. It prompts for your credentials (the same credentials you use when logging in to the Kinvey Console) and instance ID and creates a working profile for you, which stores the provided settings for future executions.

When prompted for Profile name, enter a name for your new working profile that Kinvey CLI will create for you. Kinvey CLI will use this profile automatically for future executions as long as it is the only profile on the system. You can create new profiles and select an active profile if you need to.

$ kinvey init
? E-mail john.doe@kinvey.com
? Password ***********
? Instance ID (optional) kvy-us1
? Profile name dev

You can run kinvey init from any directory as it always writes your new profile in your home directory.

Next, you need to configure Kinvey CLI to connect to a Flex Service that you've already created using the Kinvey Console.

For the following commands, you need to switch to the Node.js project directory that you will be deploying as a Flex service. The configuration they create and read is project-specific.

cd <node.js project dir>
kinvey flex init

Through a series of prompts, this command asks you for a domain in which to operate (app or organization) and a Flex service to deploy to. If your service has multiple environments, you will have to select an environment to work with; otherwise the command will silently select the default service environment.

Running the Deploy Command

The Kinvey CLI kinvey flex deploy command deploys a complete Node.js project to an existing Flex service on the FlexService Runtime and starts it. Always run this command at the root of your Flex service:

$ cd my-kinvey-node-service
$ kinvey flex deploy

The deploy operation sends your service code to Kinvey, processes it, and deploys it to the FlexService Runtime. This process can take up to several minutes depending on network conditions, the number of dependencies, and other factors. The way to track the deploy progress is to run the kinvey flex status command.

For the complete documentation of Kinvey CLI, see its GitHub page.

Deployment Specifics

The Deployment Process

After a Kinvey service deploys, the FSR runs npm install to install all dependencies listed in package.json. Its next step is to start the service using node .. You must have a properly configured entry point at the root of your project for your service to successfully start post-deployment.

Updating Deployed Services

When you are ready to update a Flex service, rerun the kinvey flex deploy command at the root of your source directory, but first ensure that you have incremented the project version in package.json. The FSR uses a rolling-restart technique which helps to eliminate downtime during the upgrade process. A few old replicas will remain online until the new service has been completely and successfully deployed.

Mapping Flex Resources to Backend Features

A typical Flex service exposes a set of resources that you can use in your app. For example, a FlexData service would expose a data set, a FlexFunctions service would expose one or more callable handlers, and FlexAuth—a set of authentication-related handlers.

To use the resources exposed by your Flex service, you need to map them to the corresponding part of the Kinvey backend.

FlexData

Kinvey presents data coming from FlexData services as Kinvey collections. This way you can take advantage of the data abstraction provided by Kinvey and utilize the same APIs as you would when working with collections from the Kinvey Data Store.

Take these steps to map FlexData data to a Kinvey collection:

1. Log in to the Kinvey Console.
2. On the Apps tab, open an app environment.
3. Go to DATA > Collections and click Add a Collection.

4. Name the collection and click Save.

   The Data source tab of the collection's Settings appears.

5. Select the Use a Data Service option.

6. Click the FlexData service that you created for your project.
7. From the Service Object drop-down list that appears, select the service object that you want to map to the collection.
8. Click Confirm.

FlexFunctions

Take these steps to map a FlexFunctions handler to a Collection Hook:

1. Log in to the Kinvey Console.
2. On the Apps tab, open an app environment.
3. Click BUSINESS LOGIC > Collection Hooks.
4. Click Add a Hook.
5. Select a Collection to attach the handler to.
6. Select a Hook type.
7. For Business Logic type, select Microservice and then click Create collection hook.
8. Select the FlexFunctions service that you created for your project.
9. From the Available handler functions drop-down list that appears, select the handler that you want to map to the hook.
10. Click Save changes.

Take these steps to map a FlexFunctions handler to a Custom Endpoint:

1. Log in to the Kinvey Console.
2. On the Apps tab, open an app environment.
3. Click BUSINESS LOGIC > Custom Endpoints
4. Click Add an Endpoint and give the ndpoint a name.
5. For Business Logic type, select Microservice and then click Create an endpoint.

FlexAuth

Take these steps to map a FlexAuth handler to a MIC service:

1. Log in to the Kinvey Console.
2. On the Apps tab, open an app environment.
3. Click IDENTITY > Mobile Identity Connect.
4. Click Add Auth Service.
5. Select Flex Runtime.
6. In the Kinvey Microservice Runtime section, select the FlexAuth service that you created for your project.
7. From the list that appears, select the handler that you want to map and click Save Changes.
8. Set up the rest of the MIC service settings and click Save Service.

Using Environment Variables

Environment variables for Flex services enable you to dynamically pass settings to your Flex code without making changes to it. Environment variables are key-value pairs that you create and modify as part of your Flex project using Kinvey CLI.

Each Flex service environment has an independent set of environment variables. You can write your Flex service code to follow different logic depending on the environment variables' values.

Naming

Environment variable names are case-sensitive and must adhere to the following additional rules:

• Start with a letter or the underscore character
• Continue with letters, numbers, or underscores

The value of an environment variable must be a string and can be empty.

You cannot use the following variable names which are reserved for internal use. Any attempt to set any of these variables will be ignored.

• SDK_RECEIVER

Another set of names is used for Preset Variables, which, however, are user editable.

Managing

Use Kinvey CLI to create, update, and delete environment variables. You can do that on several stages, including when creating a Flex service, when deploying a service version, and after a version is deployed. In the latter two cases, you can choose between setting individual variables and replacing the entire set of variables used on a service.

Setting

You can set environment variables in the following ways:

• As part of the create Flex service operation
• As part of the deploy Flex service operation
• As an independent action any time after the initial creation of the Flex service

In a service environment that already runs a deployed service, actions like creating, updating, or deleting environment variables take effect immediately. Each change triggers a graceful restart of the Flex service. It is recommended to batch the changes to minimize the number of restarts.

The following commands append new variables to the set and replace the value of variables that already exist in the set.

# Before execution: "MY_APP_A=oldValueA", "MY_APP_B=valueB"
# After execution:  "MY_APP_A=newValueA", "MY_APP_B=valueB", "MY_APP_C=valueC"

# When creating a service
kinvey flex create MyService --vars "MY_APP_A=newValueA,MY_APP_C=valueC"

# When deploying a service
kinvey flex deploy --set-vars "MY_APP_A=newValueA,MY_APP_C=valueC"

# On an existing service
kinvey flex update --set-vars "MY_APP_A=newValueA,MY_APP_C=valueC"

The following commands replace the entire existing set of variables with the set specified at the command line.

# Before execution: "EXSTNG_A=exmplA", "EXSTNG_B=exmplB", "EXSTNG_C=exmplC"
# After execution: "NEW_A=valueA", "NEW_B=valueB"


# When deploying a service
kinvey flex deploy --replace-vars "NEW_A=valueA,NEW_B=valueB"

# On an existing service
kinvey flex update --replace-vars "NEW_A=valueA,NEW_B=valueB"

Viewing

At any time after the creation of a Flex Service, you can view the set of environment variables stored on the server. The kinvey flex show command prints the variable information among other things.

$ flex show
key                            value                                   
-----------------------------  -----------------------------------
serviceName                    MyService
secret                         9e6127851b1520c433d50e407f587...11b
environmentVariables.MY_APP_A  newValueA                                   
environmentVariables.MY_APP_C  valueC

Accessing in Code

To access an environment variable in your flex code, use process.env.ENV_VAR_NAME where ENV_VAR_NAME is the name you gave to the variable.

The following example uses an environment variable called NODE_ENV to determine what service environment the Flex code is running on for troubleshooting purposes.

const sdk = require('kinvey-flex-sdk');

function isDevEnvironment() {
 return process.env.NODE_ENV === 'development';
}

function logDebug(loggingModule, message) {
 if (isDevEnvironment()) {
   loggingModule.info(message);
 }
}

function doSomething() {
 // do some work, return a result
}

sdk.service(null, (err, flex) => {
 const { logger } = flex;
 const config = { ... };

 if (err) {
   return logger.fatal(err);
 }

 logDebug(
     logger,
     `Service initialized with config: ${JSON.stringify(config, null, 2)}`
 );

 const funcs = flex.functions;

 funcs.register('my-func', (context, complete, modules) => {
   logDebug(logger, 'About to do something');
   const result = doSomething();
   logDebug(logger, `Successfully did something. Result: ${result}`);
   complete().setBody({ result }).ok().done();
 });
});

Preset Variables

Flex Runtime automatically sets the NODE_ENV environment variable to the current name of the service environment. This happens on project deployment or service environment update. You are free to manipulate the variable as you see fit, but keep the following specifics in mind:

• Flex updates the variable automatically if you change the environment name but will stop to do so as soon as you set the variable manually.
• Services that existed before the Flex preset variables feature was introduced will not have the variable set until you deploy a new service version. If you have already created a variable with the same name, Flex will preserve the value you have set. Use the kinvey flex show Kinvey CLI command at any time to see if NODE_ENV is set.
• If you delete the NODE_ENV variable after setting it manually, its behavior immediately returns back to the normal automatic setting.

Managing, Troubleshooting, and Terminating Services

Service Information

The kinvey flex status command prints server-side information that includes the status of the current Flex service environment, its version, the runtime the service code is running on, deployment status, and so on.

kinvey flex status

Optionally, add the --service and --env command-line options to request the state of another service and environment combination or only --env if you want to check the status of another service environment inside the current service.

The following table details the various service environment statuses you can receive.

Service Environment Status	Meaning
NEW	The configured Flex service environment has not been deployed to.
UPDATING	The service environment is in the process of deploying, updating, or scaling.
ERROR	Error starting one (but likely every) instance of your service environment. Check kinvey flex logs for more information.
ONLINE	Service environment instances have been started on the FlexService Runtime and are responding to pings.

The following table details the various deployment statuses you can receive.

Deployment Status	Meaning
QUEUED	The deploy has not yet started.
BUILDING	The backend is in the process of turning the Node.js project into a runnable image.
DEPLOYING	The backend is replacing the previous image with the new image.
ERROR	The deploy has completed unsuccessfully.
COMPLETED	The deploy has completed successfully.

Recycling Failed Services

When your service fails for any reason and cannot self-recover, terminate by calling process.exit() in your Node.js code with a zero or non-zero result. Terminated instances are replaced automatically.

In cases of catastrophic failure, Flex services can be recycled by running kinvey flex recycle. This allows you to get your service running once again until you are able to deploy an updated version.

The recycle command operates on service environment level. You need to specify the environment to recycle, otherwise the command recycles the current service environment.

After the recycle operation completes, you can observe the state of your cluster as it restarts by running kinvey flex status.

Logging

The output of any console.log(), console.error(), or logger statements in your service can be accessed with Kinvey CLI kinvey flex logs command.

The following console output shows the logs for a single "read all" requests to a "Books" collection:

Count: 3

containerId   message                                 threshold  timestamp
------------  --------------------------------------  ---------  ------------------------
a862587614db  getRecordByAll SDK request received     null       2018-02-21T11:47:04.570Z
a862587614db  Retrieving all books                    null       2018-02-21T11:47:04.570Z
a862587614db  Preparing to return books...            null       2018-02-21T11:47:04.571Z

You can request a subset all log entries using paging and time bounds. For example, the following command would return the fifth 200-entry set of all results that fall between Dec 1, 2017 and Feb 1, 2018.

kinvey flex logs --page 5 --number 200 --from "2017-12-01" --to "2018-02-01"

In addition to being a valuable tool for monitoring deployed services, Flex service logs are useful for troubleshooting services. The following is an example of a service index.js file with a startup error (ReferenceError):

console.log('Service initialization started');

abc

console.log('Service initialization complete');

Following a deploy, logs for the above service would contain lines similar to these:

Count: 5

containerId   message                                               threshold  timestamp
------------  ----------------------------------------------------  ---------  ------------------------
9d0134ba442d  ReferenceError: abc is not defined                    null       2018-01-16T18:31:46.697Z
9d0134ba442d  at Object.Module._extensions..js (module.js:550:10)   null       2018-06-16T18:31:46.699Z
9d0134ba442d  at Function.Module._load (module.js:407:3)            null       2018-01-16T18:31:46.700Z
9d0134ba442d  at tryModuleLoad (module.js:415:12)                   null       2018-01-16T18:31:46.699Z
9d0134ba442d  at 0bject.<anonymous> (/opt/kinvey/business-logic-runner/index.js:3:1) null       2018-01-16T18:31:46.698Z

Runtime

Runtime Selection

Internal Flex services run on a preconfigured runtime but you can still choose between three runtime variations powered by different major versions of Node.js: 6.x, 8.x, 10.x, and 12.x.

The runtime selection is limited to the major Node.js version. You can choose to run your project on any of the supported major branches but the minor and patch versions are always determined by the Flex Runtime.

New Flex services deploy on the 10.x branch unless otherwise specified.

Kinvey does its best to provide the latest LTS versions of the supported major Node.js branches. The frequent updates mean that the version that you start developing on may be replaced by the time you are ready to deploy the final version of your project.

After you deploy a Flex project, it remains on the same Node.js version until any of the following happens:

• You upgrade it to a new major version.
• You deploy a new project version and Kinvey has moved the runtime template to a new minor or patch Node.js version in the meantime, in which case the latest runtime will be used, preserving the selected major version.
• Kinvey chooses to upgrade all Flex services' runtimes to a more recent minor and patch version because of security or efficiency reasons.

Go to the Kinvey CLI documentation to learn how to select runtime version for your Flex projects.

Service Idling

The Flex Service Runtime may sometimes idle your Flex service. This means that it will free up resources associated with it, so that it can allocate more resources on other services.

A Flex service will only be idled if it has no requests for a certain period of time. That period varies based on system capacity and resource availability, but is never less than 4 hours.

After a Flex service has been idled, the next request to it will bring it up again. This is transparent for the client, but will have a few(typically 4-5) seconds delay in responding to the request.

If you have a use case where service idling will prevent you from doing what you need, please contact Kinvey Support to request disabling the idling functionality for your service.

Testing Locally

It is recommended to test your code locally before deploying a new version of your project. To do that, start Node.js in the project directory:

node .

The Flex SDK exposes your configured service handlers on local port 10001. You can query and test your handlers by hitting various URLs in the following format: http://localhost:10001/:ServiceObjectName/1. This would emit an onGetById event in your service.

Checking the Node.js Version on the Server

Before you start testing, you might want to make sure that you are running the exact same Node.js version locally as on the server. Follow these steps to check the Node.js version on the server:

1. Deploy an interim version that prints the Node.js version in the logs.

   The code below registers a simple Flex function that you can call as an endpoint at any time to retrieve the Node.js version. It also logs the Node.js version as soon the service runs using logger.info(), which is recommended for any Flex service.

    const sdk = require('kinvey-flex-sdk');
    const service = sdk.service(function(err, flex) {
      const flexFunctions = flex.functions;
      const logger = service.logger;
      const nodeVersion = process.version;
      // print the version when starting the service
      logger.info('Node.js version: ${nodeVersion}'); 
   
      function getNodeVersion(context, complete, modules) {
        complete().setBody({nodeVersion}).ok().done();
       }
      // set the handler
      flexFunctions.register('getNodeVersion', getNodeVersion);
    });

2. Use Kinvey CLI to view the service logs:

$ kinvey flex logs

98c7d39e9adc  2018-11-29T11:53:37.211Z  info  Node.js version for local testing: v6.14.4

Providing Context

In production use, Kinvey injects a variety of context data in your Flex service code. Some Flex SDK modules require the data to work properly. You can simulate it for local testing through the use of HTTP headers.

See the full list of headers in the Flex API reference guide.

Architectural Notes

Several containers are spawned per service during the Flex service deployment phase. Scaling is handled automatically and updates are performed in a rolling fashion in order to avoid downtime. Flex services run under heavily-restricted user accounts within their respective containers for security purposes. Security, updates, and maintenance are handled by Kinvey.

Restrictions and Timeouts

Services deployed on the FlexService Runtime are subject to certain resource and time bounds, as well as other precautionary measures which aim to ensure quick and steady execution.

• Each Flex service can access a limited memory pool, based on the Kinvey plan of its organization. If the memory consumption exceeds the limit, the Flex Service Runtime will restart the service.
• After deployment of a new version of a Flex service, the Flex Service Runtime attempts to connect to the service. If it cannot connect, it will revert your service to the previous working version. You can use the kinvey flex logs to debug startup error(s).
• If your Flex service crashes too often, it will be put in a suspended state. This means that the Flex Service Runtime will not try to start it anymore. The only way to fix a suspended service is to deploy a new version.
• Flex requests are subject to the general Kinvey 60-second request timeout. To ensure that the request can pass through the remainder of the Kinvey pipeline before the timeout elapses, FlexService Runtime automatically returns a timeout error after 50 seconds. Any result that your Flex service tries to return after that is ignored unless you implement your own return mechanism. Note that Flex provides a mechanism for executing long-running code that utilizes the timer functions of Node.js.