FlexService Runtime

The FlexService Runtime (FSR) is a Node.js serverless runtime environment for Kinvey Flex services. It allows you to deploy your FlexData, FlexFunctions, or FlexAuth services in the cloud, on Kinvey-managed infrastructure. Such Flex services are called internal Flex services.

If you want to deploy your Flex service elsewhere, you need to deploy it as an external Flex service.

Preparing Your Environment

Before you can write and deploy Flex services, you need to create yourself a Node.js environment.

  • FlexService Runtime requires you to develop against a specific version of Node.js. See the Environment section for the current version of Node.js run by FlexService Runtime and install it in your development environment to ensure maximum compatibility.
  • Install NPM (Node Package Manager).
  • Install Kinvey CLI which you'll use to deploy and manage the service. Go to the Kinvey CLI GitHub page for installation instructions.

We recommend installing Node.js and NPM using NVM.

Creating a Flex Service

Creating a Flex service includes several steps, from configuring Kinvey Console to setting up a Node.js project. After you complete all the steps, you will have a Flex service ready to be deployed.

Setting Up Kinvey Console

Before you can deploy your Node.js project, you need to provision a Flex service using the Kinvey Console. The Flex service makes the connection between your Node.js project and the Kinvey backend.

Take the following steps to provision a Flex service:

  1. Log in to Kinvey Console.
  2. Select the Service Catalog tab and click Add a service.
  3. Select Flex and then Flex Services Runtime.
  4. Complete the form and finally click Save.
    • Name—Arbitrary name for the service.
    • Description—Optional details about the service.
    • Scope/App—App or organization that will use the service.
    • Secret—A passphrase that you can optionally specify in your Node.js project as an additional server-to-server authentication step. The recommended minimum length is 265 bits.

After completing these steps, the new Flex service should be visible in your Service Catalog.

Initializing a Node.js Project

Underneath, each Flex service is a Node.js project. Choose your preferred method to create the project. The following steps use NPM for that purpose.

Take these steps to initialize a Node.js project:

  1. Create a directory to contain your project files and enter it.
  2. Initialize a Node.js project using NPM:

     npm init

    This utility guides you through setting your Node.js project settings and finally writes a package.json file. Flex does not pose any particular requirements on the settings' values; accept the default values if you are in any doubt.

  3. Add the Flex SDK as a dependency in package.json:

     npm install kinvey-flex-sdk --save
  4. In the project directory, create an empty text file and name it after the entry point that you specified during the npm init execution.

    For example, if you accepted the default value, create a file called index.js.

Initializing the Flex SDK

After you have created the file structure needed for the project, you can start adding code. The Flex SDK is the only mandatory piece. Aside from it, you can require any npm module you need for your project with the exception of modules with external dependencies because they will not be available on the FSR.

To use the Flex SDK, require it in your project:

const sdk = require('kinvey-flex-sdk');

Then initialize the SDK to retrieve a reference to the backend service:

sdk.service((err, flex) => {
  // code goes here
});

When running locally, you can specify a host and port to listen on by passing an options object with an optional host and port. If no host and port are specified, localhost:10001 is used:

sdk.service({ host: 'flex-machine', port: 7777 }, (err, flex) => {
  // code goes here
});

Securing your Flex Service

Communication to Flex services running on FSR are encrypted by default. In addition, Flex services can be secured by use of a shared secret which serves as server-to-server authentication.

All requests to a Flex service from Kinvey include the shared secret to prevent unauthorized access to the service. When configuring your service in the Kinvey Console, enter in a secure, random, strong shared secret (256-bit minimum recommended).

To enable the shared secret within your project code, pass the shared secret into the Flex SDK initialization method:

sdk.service({ sharedSecret: <my shared secret> }, (err, flex) => {
    // service code
})

If the shared secret is enabled in the Flex Service, the service will reject any request that does not include the shared secret key.

Deploying Flex Services

Kinvey CLI is the utility that lets you deploy and manage Flex services on the FSR. Before you can deploy a Flex service, you need to configure Kinvey CLI.

For the complete documentation of Kinvey CLI, see its GitHub page.

Configuring Kinvey CLI

The fastest way to get started is to run the kinvey init command. It prompts for your credentials (the same credentials you use when logging in to the Kinvey Console) and instance ID and creates a working profile for you, which stores the provided settings for future executions.

You only need to specify an instance ID if you are on a dedicated Kinvey instance or using the Progress Health Cloud. Otherwise just press Enter to continue.

When prompted for Profile name, enter a name for your new working profile that Kinvey CLI will create for you. Kinvey CLI will use this profile automatically for future executions as long as it is the only profile on the system. You can create new profiles and select an active profile if you need to.

$ kinvey init
? E-mail john.doe@kinvey.com
? Password ***********
? Instance ID (optional) kvy-us1
? Profile name dev

You can run kinvey init from any directory as it always writes your new profile in your home directory.

Next, you need to configure Kinvey CLI to connect to a Flex Service that you've already created using the Kinvey Console.

For the following commands, you need to switch to the Node.js project directory that you will be deploying as a Flex Service as the configuration they create and read is project-specific.

cd <node.js project dir>
kinvey flex init

Through a series of prompts, this command asks you for a domain in which to operate (app or organization) and a Flex Service to deploy to.

Runnning the Deploy Command

The Kinvey CLI flex deploy command deploys preconfigured Node.js services to the FlexService Runtime and starts them. This command must be run at the root of your Flex service:

$ cd my-kinvey-node-service
$ kinvey flex deploy

Kinvey CLI sends binary data (content type "multipart/form-data") during the deploy process. The deploy job will fail if traffic of this type is blocked within your network.

You cannot deploy the same service to the FlexService Runtime more than once. You must increment the version in package.json before redeploying.

The deploy operation sends your service to Kinvey, processes it, and deploys it to the FlexService Runtime. This process can take up to several minutes depending on network conditions and other factors. Each deploy request returns an ID which can be used with the flex job [id] command to check the status of a pending deploy. Skipping the ID shows status information about the most recent job.

For the complete documentation of Kinvey CLI, see its GitHub page.

Managing, Troubleshooting, and Terminating Services

IP Whitelisting

Flex services running on the FlexService Runtime are assigned dynamic IPs for outbound requests. These IPs can change at any time and therefore cannot be used as part of an IP whitelisting strategy for any external services that the Flex service interacts with. If IP whitelisting is required, Kinvey provides an HTTP(S) Proxy Server that can be used for routing traffic.

To use this proxy, set the HTTP_PROXY and HTTPS_PROXY environment variables at the top of the index.js file inside your Node.js project:

process.env.HTTP_PROXY = 'http://' + '<proxy URL>';
process.env.HTTPS_PROXY = 'http://' + '<proxy URL>';

This is how to obtain the proxy URL for your instance:

  • For customers on multitenant instances, find the proxy URL in the next table.

    InstanceProxy URL
    US Multitenant (kvy-us1)kvy-us1-proxy.kinvey.net:3128
    US HIPAA-compliant instance (kvy-us2)kvy-us2-proxy.kinvey.net:3128
    EU Multitenant (kvy-eu1)kvy-eu1-proxy.kinvey.net:3128
  • For customers on dedicated instances or Progress Health Cloud customers, follow these instructions to assemble the proxy URL for your instance:

    1. Go to the Kinvey Console and find your Instance ID on the dashboard. It is listed next to the App Key and App Secret.
    2. Insert your Instance ID in the following URL template:

       <Your Instance ID>-proxy.kinvey.net:3128

The proxy solution only works for libraries and HTTP modules that honor the HTTP_PROXY and HTTPS_PROXY environment variables (e.g. request). The proxy URLs can also be used with libraries that support specifying the HTTP/HTTPS proxy as part of the library usage.

Service Health

You can check the health of your Flex service at any time after a deploy. Simply run the following command:

kinvey flex status

The command shows the current state of the last deployed service. Optionally, add the --serviceId command-line option to request the state of another service.

The following table details the various statuses you can receive.

StatusMeaning
NEWThe configured Flex service has not been deployed to.
UPDATINGService is in the process of deploying, updating, or scaling.
ERRORError starting one (but likely every) instance of your service. Check kinvey flex logs for more information.
ONLINEService instances have been started on the FlexService Runtime and are responding to pings.

Recycling Failed Services

When your service fails for any reason and cannot self-recover, terminate by calling process.exit() in your Node.js service with a zero or non-zero result. Terminated instances are replaced automatically.

In cases of catastrophic failure, Flex services can be recycled by running kinvey flex recycle. This allows you to get your service running once again until you are able to deploy an updated version.

The kinvey flex recycle command entails downtime. Your service will be unavailable anywhere from a few seconds to a few minutes while the operation completes.

Similarly to the deploy command, kinvey flex recycle returns a job ID. After the recycle operation completes you can observe the state of your cluster as it restarts by running kinvey flex status.

Logging

The output of any console.log() or console.error() statements in your service can be accessed with the Kinvey CLI flex logs command.

The following console output shows the logs for a single "read all" requests to a "Books" collection:

Count: 3

containerId   message                                 threshold  timestamp
------------  --------------------------------------  ---------  ------------------------
a862587614db  getRecordByAll SDK request received     null       2018-02-21T11:47:04.570Z
a862587614db  Retrieving all books                    null       2018-02-21T11:47:04.570Z
a862587614db  Preparing to return books...            null       2018-02-21T11:47:04.571Z

The threshold column shows the log level (error, warn, info, fatal) if you used the Flex SDK logger to log the message or null if you used console.log().

You can control how log entries are returned using paging and time limits. For example, the following command would return the fifth 200-entry set of all results that fall between Dec 1, 2017 and Feb 1, 2018.

kinvey flex logs --page 5 --number 200 --from "2017-12-01" --to "2018-02-01"

In addition to being a valuable tool for monitoring deployed services, Flex service logs are useful for troubleshooting problematic services. The following is an example of a service index.js file with a startup error (ReferenceError):

console.log('Service initialization started');

abc

console.log('Service initialization complete');

Following a deploy, logs for the above service would contain lines similar to these:

Count: 5

containerId   message                                               threshold  timestamp
------------  ----------------------------------------------------  ---------  ------------------------
9d0134ba442d  ReferenceError: abc is not defined                    null       2018-01-16T18:31:46.697Z
9d0134ba442d  at Object.Module._extensions..js (module.js:550:10)   null       2018-06-16T18:31:46.699Z
9d0134ba442d  at Function.Module._load (module.js:407:3)            null       2018-01-16T18:31:46.700Z
9d0134ba442d  at tryModuleLoad (module.js:415:12)                   null       2018-01-16T18:31:46.699Z
9d0134ba442d  at 0bject.<anonymous> (/opt/kinvey/business-logic-runner/index.js:3:1) null       2018-01-16T18:31:46.698Z

There is a limit of 100 MB to the size of the Flex service logs that are kept. When log entries exceed that size, the oldest ones are deleted.

Runtime

Environment

FlexService Runtime containers are based on Ubuntu (LTS) and run Node.js version 6.12.3 with node-gyp support. Your services should be developed against this version of node. For more information on this version of Node.js and its features, refer to the official changelog.

More on Deploys

The Deploy Process

After Kinvey services are deployed, package.json dependencies are installed via npm install. The service is started with node . . A file named index.js must exist at the root of your project in order for your service to successfully start post-deployment.

npm install is executed with the ‘--production’ flag when services are deployed to the FlexService Runtime. Any contents within the devDependencies section of your package.json file will not be installed.

Updating Deployed Services

When you are ready to update a Flex service, re-run the flex deploy command at the root of your source directory. The FlexService Runtime uses a rolling-restart technique which helps to eliminate downtime during the upgrade process. A few old replicas will remain online until the new service has been completely and successfully deployed.

Testing Flex Services

Before attempting to deploy a service you should run it locally (node .) to verify that it is free of startup errors.

The Flex SDK exposes your configured service handlers on port 10001 when you start your service locally (node .). You can query and test your handlers by hitting various URLs in the following format: http://localhost:10001/:ServiceObjectName/1 (this would emit an onGetById event in your service).

Architectural Notes

Several containers are spawned per service during the Flex service deployment phase. Scaling is handled automatically and updates are performed in a rolling fashion in order to curb downtime. Flex services run under heavily-restricted user accounts within their respective containers for security purposes. Security, updates, and maintenance are handled by Kinvey.

Restrictions and Timeouts

Services deployed on the FlexService Runtime are subject to certain resource and time bounds, as well as other precautionary measures which aim to ensure quick and steady execution.

  • Each Flex service can access a guaranteed memory pool of 64 MB, burstable to 256 MB. If memory consumption exceeds the 256 MB threshold, the FlexService Runtime may restart the service.
  • After a successful flex deploy operation, the FlexService Runtime will attempt to connect to your service(s) through the Flex SDK module. Services which fail or crash due to runtime errors are automatically restarted by a mechanism which times out after 30 failed connection attempts. The flex logs command can (and is likely to) yield several copies of the same startup error(s) when this occurs.
  • Flex requests are subject to the general Kinvey 60-second request timeout. To ensure that the request can pass through the remainder of the Kinvey pipeline before the timeout elapses, FlexService Runtime automatically returns a timeout error after 50 seconds. Any result that your Flex service tries to return after that is ignored unless you implement your own return mechanism. Note that Flex provides a mechanism for executing long-running code that utilizes the timer functions of Node.js.
Got a question?