Menu
bl-transactional-support
Tutorials /

Transactional support with Business Logic

Implementing Transactions in the Kinvey Data Store

This tutorial guide shows Kinvey Enterprise customers how to enable transaction support in their Kinvey data stores using Kinvey's Business Logic (BL) feature.

NOTE Kinvey strongly discourages using transactions in your application unless there is no other alternative design pattern. Transactions make your app more complicated and harm the performance of the application.

NOTE Customers using a Kinvey Data Link Connector to connect to an enterprise system should use the transaction support in the underlying system.

NOTE This guide only applies to Kinvey on a dedicated instance and is not guaranteed to work properly on the Kinvey multi-tenant offering.

Overview

There are two methods to enable transactions in your Kinvey data store. The first method uses two-phase commits to enable atomic commits and rollbacks on multiple documents within a single collection. This guide will not cover the two-phase commit method in-depth, but see two-phase commits for an explanation of which functions to use in the collectionAccess module in BL. The second method uses a mutex collection to ensure that records are updated in an atomic fashion across collections.

Implementation

Two-Phase Commits

To follow the guide for Two-Phase Commits, use several different functions from the collectionAccess module:

  • insert -- Use to add new records
  • update -- Use to update the transaction properties and the records of interest
  • find -- Find all transactions in some given state
  • findOne -- Find a single transactions to process
  • findAndModify -- Use for advanced cases where the data is shared

Also, the moment library will help where date operations are listed

Mutex Implementation

One of the best parts of Kinvey's RESTful architecture is the ease of understanding the state of data and the operations on data due to the principles of REST. These principles are incompatible with stateful transactions, so the mutex implementation will rely on Kinvey's custom endpoints instead of the normal Data Store interface.

Collection / Endpoint Setup

There is no limitation to the number of collections that can be modified atomically during this operation, however it will slow down your application if you have too many collections that need to be updated. A general rule of thumb is to only perform the minimum number of operations (and collections) while "locking" the app.

For the examples in this document We'll use CreditAccounts and LoyalityAccounts as our collections.

In addition to the collections that are being modified, you'll need a collection to hold the lock states. This should be called transactionLocks.

You'll also need a Custom Endpoint for your performing transaction.

You may also need "getter" custom endpoints to coordinate reads (see Consistency on Reads for more information).

For our example we're going to be using the following functions:

  • performRewardsTransferTransaction
  • getCreditAccounts
  • getLoyalityAccounts

NOTE Using custom endpoints in this way removes the ability for your app to use many of the Kinvey client library features. You should avoid custom endpoints for accessing collections unless you need to coordinate the reads and writes.

Custom Endpoint Logic

The first step in the process is attempting to obtain the lock to update the collections. This is done with the findAndModify operation from collectionAccess. You want to find the lock matching this transaction, so for the transaction rewardsTransfer you should find the lock with _id rewardsTransfer. 

// Helper functions
var locks = modules.collectionAccess.collection('transactionLocks');

var lock = function(name, locked){
    locks.findAndModify({_id: 'rewardsTransfer'}, {}, {"$inc": {"count": 1}}, {"upsert": true}, function(err, doc){
        if (err){
            locked(false);
        } else {
            if (doc.count === 1){
                locked(true);
            } else {
                unlock(name, function(didUnlock){ locked(false); });
            }
        }
    });
};

var unlock = function(name, unlocked){
    locks.findAndModify({_id: 'rewardsTransfer'}, {}, {"$inc": {"count": -1}}, {"upsert": true}, function(err, doc){
        if (err){
            unlocked(false);
        } else {
            unlocked(true);
        }
    });
};

Using helper functions like those listed above you can get perform the following steps to perform work in a transaction: 

function onPerformRewardsTransferTransaction(){
    lock('rewardsTransfer', function(locked){
        if (locked){
            // Critical section

            // Perform all transaction work here

            // Unlock the lock
            unlock('rewardsTransfer', function(unlocked){
                if (!unlocked){
                    return response.error("Failed to unlock transaction, this shouldn't happen");
                } else {
                    // Set a response body if needed
                    return response.complete();
                }
            });
        } else {
            // This requires upgraded BL and an Enterprise Plan
            // Wait and retry
            setTimeout(onPerformRewardsTransferTransaction, 250);

            // This could also return an error to the client and let the
            // client retry.  There is also a mixed model, where you can
            // try multiple attempts in BL and then fail to the client
        }
    });
}
Roll Back

For complete transaction support you may need to "roll-back" if an error occurred during your transaction. The best way to do this is by using an undo queue.

To use this method, update each collection using the findAndModify method, making sure to return the old entity (Do not set the new option to true).

Add each old entity to the front of the queue (make sure to record the collection that was changed, by adding an object like {collection: "foo", entity: document}).

Should an error occur, you have a complete record of the state of all entities prior to modification. To "roll-back" you just replay these entities using the save method. 

var ca = modules.collectionaccess;
async.forEach(undoQueue, function(change, cb){
    ca.collection(change.collection).save(change.entity, cb);
});

Client Setup

To begin a transaction the client should use the appropriate Kinvey client library for their application to invoke the endpoint.

If the transaction returns an error due to being locked the client should retry after a delay. If repeated errors are signaled from the transaction, the user should be informed.

Consistency on Reads

When reading from collections guarded by transactions, care needs to be taken to ensure the correct data is read.

For many cases it's OK if the data is not 100% consistent on a read (even for applications that deal with money or finite resources), since it will eventually be consistent within the time the user expects the data to be correct. This is the easiest read to implement as you use the normal Data Store methods to access these collections.

If data must be consistent, then you collections need to be locked before doing reads. 

function onGetCreditAccounts(){
    lock('rewardsTransfer', function(locked){
        if (locked){
            // Critical section

            ca.collection('CreditAccounts').find({}, function(err, doc){
                // Unlock the lock
                unlock('rewardsTransfer', function(unlocked){
                    if (!unlocked){
                        return response.error("Failed to unlock transaction, this shouldn't happen");
                    } else {
                        // Set a response body if needed
                        response.body = doc;
                        return response.complete();
                    }
                });
            });
        } else {
            // This requires upgraded BL and an Enterprise Plan
            // Wait and retry
            setTimeout(onGetCreditAccounts, 250);
        }
    });
}

function onGetLoyalityAccounts(){
    lock('rewardsTransfer', function(locked){
        if (locked){
            // Critical section

            ca.collection('LoyalityAccounts').find({}, function(err, doc){
                // Unlock the lock
                unlock('rewardsTransfer', function(unlocked){
                    if (!unlocked){
                        return response.error("Failed to unlock transaction, this shouldn't happen");
                    } else {
                        // Set a response body if needed
                        response.body = doc;
                        return response.complete();
                    }
                });
            });
        } else {
            // This requires upgraded BL and an Enterprise Plan
            // Wait and retry
            setTimeout(onGetLoyalityAccounts, 250);
        }
    });
}

If you don't need full locks during reads, but want more up-to-date data, you can store your updates in a pending collection and have clients fetch and display the pending collection as well. The flow would be:

  1. Obtain lock
  2. Update pending collection with all updates
  3. Perform updates
  4. Clear pending collection
  5. Release lock

Stale lock removal

If you need to protect against stale locks being held for too long, add a lastUpdated parameter to the transaction lock. Update this parameter when you update the count property for the lock.

Run a scheduled code endpoint every 5 minutes that looks at all locks and resets count to 0 if they haven't been updated in the last 5 minutes.

Advanced Concepts

The solution provided in this guide is fully extensible to support more advanced cases.

You could, for example, change the lock function for readers to always get the lock (instead of checking for count === 1). This allows for multiple readers and a single writer.

The lock and unlock functions can also be modified to guard single properties of entities. If this is used with the "getter" custom endpoints then accessing these entities ensures consistency.