Data Store

The simplest use case of Kinvey is storing and retrieving data to and from your cloud backend.

The basic unit of data is an entity and entities of the same kind are organized in collections. An entity is a set of key-value pairs which are stored in the backend in JSON format. Kinvey's libraries automatically translate your native objects to JSON.

Kinvey's data store provides simple CRUD operations on data, as well as powerful filtering and aggregation.

Collections

To start working with a collection, you need to instantiate a DataStore in your application. Each DataStore represents a collection on your backend.

//we use the example of a "Book" datastore
let dataStore = DataStore<Book>.collection()

Through the remainder of this guide, we will refer to the instance we just retrieved as dataStore.

Optionally, you can configure the DataStore type when creating it. To understand the types of data stores and when to use each type, refer to the DataStore Types section.

DataStore Types

When you get an instance of a datastore in your application, you can optionally select a datastore type. The type has to do with how the library handles intermittent or prolonged interruptions in connectivity to the backend.

Choose a type that most closely resembles your data requirements.

  • DataStoreType.Sync: You want a copy of some (or all) the data from your backend to be available locally on the device and you like to sync it periodically with the backend.
  • DataStoreType.Cache: You want data stored on the device to optimize your app’s performance and/or provide offline support for short periods of network loss. This is the default DataStore type if a type is not explicitly set.
  • DataStoreType.Network: You want the data in your backend to never be stored locally on the device, even if it means that the app will not offer offline use.

Head to the Specify DataStore Type section to learn how to request the DataStore type that you selected.

Entities

The Kinvey service has the concept of entities, which represent a single resource.

Model

In the Kinvey iOS library, an entity is represented by a subclass of the Entity class.

import Kinvey

class Book: Entity {

    dynamic var title: String?
    dynamic var authorName: String?
    dynamic var publishDate: Date?

    override class func collectionName() -> String {
        //return the name of the backend collection corresponding to this entity
        return "Book"
    }

    //Map properties in your backend collection to the members of this entity
    override func propertyMapping(_ map: Map) {
        //This maps the "_id", "_kmd" and "_acl" properties
        super.propertyMapping(map)

        //Each property in your entity should be mapped using the following scheme:
        //<member variable> <- ("<backend property>", map["<backend property>"])
        title <- ("title", map["title"])
        authorName <- ("authorName", map["author"])

        //this maps the backend property "date" to the member variable "publishDate" using a ISO 8601 date transform
        publishDate <- ("publishDate", map["date"], KinveyDateTransform())
    }
}
import Kinvey

class Book: Entity {

    @objc dynamic var title: String?
    @objc dynamic var authorName: String?
    @objc dynamic var publishDate: Date?

    override class func collectionName() -> String {
        //return the name of the backend collection corresponding to this entity
        return "Book"
    }

    //Map properties in your backend collection to the members of this entity
    override func propertyMapping(_ map: Map) {
        //This maps the "_id", "_kmd" and "_acl" properties
        super.propertyMapping(map)

        //Each property in your entity should be mapped using the following scheme:
        //<member variable> <- ("<backend property>", map["<backend property>"])
        title <- ("title", map["title"])
        authorName <- ("authorName", map["author"])

        //this maps the backend property "date" to the member variable "publishDate" using a ISO 8601 date transform
        publishDate <- ("publishDate", map["date"], KinveyDateTransform())
    }
}

Supported Types

The following property types are supported by the library:

Swift TypeJSON TypeNotes
Stringstring
Int, Float, DoublenumberVariations of Int, such asInt8, Int16, Int32, Int64, are also supported
Boolboolean
DatestringKinvey uses the ISO8601 date format. Use KinveyDateTransform to map date objects
GeoPointarrayRefer to the location guide to understand how to persist geolocation information

The library also provides a way to represent lists of each supported primitive type, as shown below:

Swift TypeSample code
Stringlet authorNames = List<StringValue>()
Intlet editionsYear = List<IntValue>()
Floatlet editionsRetailPrice = List<FloatValue>()
Doublelet editionsRating = List<DoubleValue>()
Boollet editionsAvailable = List<BoolValue>()
Tlet authors = List<Author>

  • List<T> types must be unmutable using the keyword let instead of the usual declaration @objc dynamic var

  • For List<T> to work with a custom object type such as Author, the type must adhere to the guidelines listed in the custom types section.

Custom Types

This section describes the guidelines to persist a type that is not covered in Kinvey's supported types, including any types that you define in your app.

Enabling your Type to be Cached

The Kinvey iOS SDK uses Realm for persisting data locally in cache. Caching for all the Kinvey-supported types is handled automatically by the SDK. For custom types, you need to extend the Realm Object class to declare that your type should be cached.

When you subclass RealmSwift.Object, all the properties of your custom type are enabled for caching, as long as they are in the Realm supported types. If you need help defining the properties of your type, take a look at this cheatsheet.

Mapping Your Type to JSON

The Kinvey iOS SDK uses the ObjectMapper library to map JSON from the backend to the entities in the app. To persist your custom type to the backend, you need to tell Kinvey how to map your object to and from JSON. ObjectMapper provides you with two ways to do this:

  • Implement Mappable: if your custom type is a class with properties that easily map to Kinvey-supported types, you should extend the Mappable protocol.
import RealmSwift
import ObjectMapper

class Person: Entity {

    @objc dynamic var name: String?
    @objc dynamic var address: Address?

    override class func collectionName() -> String {
        //name of the collection in your Kinvey console
        return "Person"
    }

    override func propertyMapping(_ map: Map) {
        super.propertyMapping(map)

        name <- ("name", map["name"])
        address <- ("address", map["address"])
    }
}

class Address: Object, Mappable {

    @objc dynamic var city: String?
    @objc dynamic var state: String?
    @objc dynamic var country: String?

    convenience required init?(map: Map) {
        self.init()
    }

    func mapping(map: Map) {
        city <- ("city", map["city"])
        state <- ("state", map["state"])
        country <- ("country", map["country"])
    }    
}
  • Implement a custom transform: Custom Transforms provide a flexible way to map complex types. Your transform programmatically defines the mapping of your custom type to JSON.
class Person: Entity {

    @objc dynamic var name: String?

    //Here the Person has an imageURL of type URL, which is not one of the supported Types
    //To allow the imageURL to be persisted in the cache, we use a computed variable of type String
    @objc private dynamic var imageURLValue: String?

    var imageURL: URL? {
        get {
            if let urlValue = imageURLValue {
                return URL(string: urlValue)
            }
            return nil
        }
        set {
            imageURLValue = newValue?.absoluteString
        }
    }

    override class func collectionName() -> String {
        return "Person"
    }

    override func propertyMapping(_ map: Map) {
        super.propertyMapping(map)

        name <- ("name", map["name"])
        imageURL <- ("imageURL", map["imageURL"], URLTransform())
    }

}

//This transform allows the URL type to be mapped to JSON in the Kinvey backend
class URLTransform: TransformType {

    public typealias Object = URL
    public typealias JSON = String

    open func transformFromJSON(_ value: Any?) -> URL? {
        guard let urlString = value as? String else {
            return nil
        }
        return URL(string: urlString)
    }

    open func transformToJSON(_ value: URL?) -> String? {
        return value?.absoluteString
    }
}

Fetching

You can retrieve entities by either looking them up using an id, or by querying a collection.

Fetching by Id

To fetch an (one) entity by id, call dataStore.findById.

let id = "<#entity id#>"
dataStore.find(byId: id) { book, error in
    if let book = book {
        //succeed
        print("Book: \(book)")
    } else {
        //fail
    }
}
let id = "<#entity id#>"
dataStore.find(id, options: nil) { (result: Result<Book, Swift.Error>) in
    switch result {
    case .success(let book):
        print("Book: \(book)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

Fetching by Query

To fetch all entities in a collection, call dataStore.find.

dataStore.find() { books, error in
    if let books = books {
        //succeed
        print("Books: \(books)")
    } else {
        //fail
    }
}
dataStore.find() { (result: Result<AnyRandomAccessCollection<Book>, Swift.Error>) in
    switch result {
    case .success(let books):
        print("Book: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

To fetch multiple entities using a query, call dataStore.find and pass in a query.

let query = Query(format: "title == %@", "The Hobbit")

dataStore.find(query) { books, error in
    if let books = books {
        //succeed
        print("Books: \(books)")
    } else {
        //fail
    }
}
let query = Query(format: "title == %@", "The Hobbit")

dataStore.find(query) { (result: Result<AnyRandomAccessCollection<Book>, Swift.Error>) in
    switch result {
    case .success(let books):
        print("Book: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

It's worth noting that an empty array is returned when the query matches zero entities.

Saving

You can save an entity by calling dataStore.save.

dataStore.save(book) { book, error in
    if let book = book {
        //succeed
        print("Book: \(book)")
    } else {
        //fail
    }
}
dataStore.save(book, options: nil) { (result: Result<Book, Swift.Error>) in
    switch result {
    case .success(let book):
        print("Book: \(book)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

The save method acts as upsert. The library uses the _id property of the entity to distinguish between updates and inserts.

  • If the entity has an _id, the library treats it as an update to an existing entity.

  • If the entity does not have an _id, the library treats it as a new entity. The Kinvey backend assigns an automatic _id for a new entity.

Deleting

To delete an entity, call dataStore.removeById and pass in the entity _id.

let id = "<#entity id#>"
dataStore.removeById(id) { count, error in
    if let count = count {
        //succeed
        print("Count: \(count)")
    } else {
        //fail
    }
}
let id = "<#entity id#>"
dataStore.remove(byId: id, options: nil) { (result: Result<Int, Swift.Error>) in
    switch result {
    case .success(let count):
        print("Count: \(count)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

Deleting Multiple Entities at Once

To delete multiple entities at once, call dataStore.remove. Optionally, you can pass in a query to only delete entities matching the query.

let query = Query(format: "title == %@", "The Hobbit")

dataStore.remove(query) { count, error in
    if let count = count {
        //succeed
        print("Count: \(count)")
    } else {
        //fail
    }
}
let query = Query(format: "title == %@", "The Hobbit")

dataStore.remove(query, options: nil) { (result: Result<Int, Swift.Error>) in
    switch result {
    case .success(let count):
        print("Count: \(count)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

Querying

Queries to the backend are represented as instances of Query. Queries have the ability to gather filter statements using NSPredicates and order statements using NSSortDescriptors. These queries are used with DataStore methods.

Query provides a number of factory methods to query-creating convenience.

The default constructor makes a query that matches all values in a collection.

let query = Query()

Query instances can be created the same way that NSPredicate instances are created. For instance, the code below show how to find all entities that match the exactly supplied value.

let query = Query(format: "location == %@", "Mike's House")

Query(format: "field = nil") creates a query that matches entities where the specified field is empty or unset. This will not match fields with the value null.

For a complete guide of NSPredicate, please check the Predicate Programming Guide

Operators

In addition to exact match and null queries, you can query based upon an several types of expressions: comparison, set match, string and array operations.

Conditional queries are built in Query.

Comparison operators

  • =, == The left-hand expression is equal to the right-hand expression.
  • >=, => The left-hand expression is greater than or equal to the right-hand expression.
  • <=, =< The left-hand expression is less than or equal to the right-hand expression.
  • > The left-hand expression is greater than the right-hand expression.
  • < The left-hand expression is less than the right-hand expression.
  • !=, <> The left-hand expression is not equal to the right-hand expression.
  • BETWEEN The left-hand expression is between, or equal to either of, the values specified in the right-hand side. The right-hand side is a two value array (an array is required to specify order) giving upper and lower bounds. For example, 1 BETWEEN { 0 , 33 }, or $INPUT BETWEEN { $LOWER, $UPPER }.

For a complete guide of NSPredicate Syntax, please check the Predicate Format String Syntax

Modifiers

Queries can also have optional modifiers to control how the results are presented. Query allows for limit & skip modifiers and sorting.

Limit and Skip Modifiers

Limit and Skip allow for paging of results. For example, if there are 100 possible results, you an break them up into pages of 20 items by setting the limit to 20, and then incrementing the skip modifier by 0, 20, 40, 60, and 80 in separate queries to eventually retrieve them all.

let pageOne = Query {
    $0.skip = 0
    $0.limit = 20
}

let pageTwo = Query {
    $0.skip = 20
    $0.limit = 20
}

...

There is an automatic limit of 10,000 entities on any query. If exactly 10,000 items are returned and you think there may be more (you can use count to find the total number), then retry with limit and skip modifier to get all the entities.

Sort

Query results can also be sorted by adding sort modifiers. Queries are sorted by field values in ascending or descending order, by using NSSortDescriptors.

For example,

let predicate = NSPredicate(format: "name == %@", "Ronald")
let sortDescriptor = NSSortDescriptor(key: "date", ascending: true)
let query = Query(predicate: predicate, sortDescriptors: [sortDescriptor])

To sort by a field and then sub-sort those results by another field, just keep adding additional sort modifiers.

Compound Queries

You can make expressive queries by stringing together several queries. For example, you can query for dates in a range or events at a particular day and place.

NSCompoundPredicate can be used to create a query that is a combination of conditionals. For example, the following code uses a NSCompoundPredicate to query for cities with 2 different filter statements combined by an AND operator.

//Search for cities in Massachusetts with a population bigger than 50k.
let query = Query(format: "state == %@ AND population >= %@", "MA", 50000)

The code below does the same thing, but using NSCompoundPredicate.

let predicateState = NSPredicate(format: "state == %@", "MA")
let predicatePopulation = NSPredicate(format: "population >= %@", 50000)
let predicate = NSCompoundPredicate(andPredicateWithSubpredicates: [predicateState, predicatePopulation])
let query = Query(predicate: predicate)

Special Keys

The following pre-defined constant keys map to Kinvey-specific fields. These can be used in querying and sorting.

  • Entity.Key.entityId the object id
  • Entity.Key.acl the ACL object which containts the creator of the object
  • Entity.Key.metadata the metadata object which contains the entity create time and last modification time

Reduce Function

There are five pre-defined reduce functions:

  • @count - counts all elements in the group
  • @sum: - sums together the numeric values for the input field
  • @min: - finds the minimum of the numeric values of the input field
  • @max: - finds the maximum of the numeric values of the input field
  • @avg: - finds the average of the numeric values of the input field

Counting

To count the number of objects in a collection, use the @count keyword.

let query = Query(format: "children.@count == 2")

To count the number of entities in a collection you should use DataStore.count()

dataStore.count { (count, error) in
    if let count = count {
        //succeed
        print("Count: \(count)")
    } else {
        //fail
    }
}
dataStore.count(options: nil) { (result: Result<Int, Swift.Error>) in
    switch result {
    case .success(let count):
        print("Count: \(count)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

You can also pass an optional query parameter to DataStore.count() in order to performe a count operation in a specific query

let query: Query = <#Query(format: "myProperty == %@", myValue)#>
dataStore.count(query) { (count, error) in
    if let count = count {
        //succeed
        print("Count: \(count)")
    } else {
        //fail
    }
}
let query: Query = <#Query(format: "myProperty == %@", myValue)#>
dataStore.count(query, options: nil) { (result: Result<Int, Swift.Error>) in
    switch result {
    case .success(let count):
        print("Count: \(count)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

Aggregation / Grouping

DataStore.group() is a overloaded method that allows you do groupig and aggregation against your data in the backend. You can write your own reduce function using JavaScript or use pre-defined functions like:

  • Count
  • Sum
  • Average (avg)
  • Min (minimum)
  • Max (maximum)

For example:

dataStore.group(keys: ["country"], avg: "age", options: nil) { (result: Result<[AggregationAvgResult<Person, Int>], Swift.Error>) in
    switch result {
    case .success(let array):
        print(array)
    case .failure(let error):
        print(error)
    }
}

If you wold like to see additional functions, email support@kinvey.com.

Setting Custom Options

Whenever you need to set a different option value other then the default value for a single request, a collection instance, or a Kinvey instance, you can set it in the Options structure. Although the SDK allows you to set some of the options separately, setting them through the structure is more convenient and tidy in most cases.

To see the complete list of values defined by the structure, check the API Reference.

All properties in the Options structure are optional. The value of nil is assumed for a property that you don't set explicitly, which is the default value.

To set options, first instantiate the Options structure and then pass the instance to the request or collection or Kinvey instance. You can create as many Options instances as you need and set their options differently.

let dataStore = DataStore<Book>.collection()
let query = Query(format: "title == %@", "The Hobbit")
let options = Options(readPolicy: .forceNetwork, timeout: 120)
dataStore.find(query, options: options) { (result: Result<[Book], Swift.Error>) in
    switch result {
    case .success(let books):
        print("change this line to handle success")
    case .failure(let error):
        print("change this line to handle failure")
    }
}

Caching and Offline

A key aspect of good mobile apps is their ability to render responsive UIs, even under conditions of poor or missing network connectivity. The Kinvey library provides caching and offline capabilities for you to easily manage data access and synchronization between the device and the backend.

Kinvey’s DataStore provides configuration options to solve common caching and offline requirements. If you need better control, you can utilize the options described in Granular Control.

Specify DataStore Type

When initializing a new DataStore to work with a Kinvey collection, you can optionally specify the DataStore type. The type determines how the SDK handles data reads and writes in full or intermittent absence of connection to the app backend.

In this section:

Sync

Configuring your datastore as a Sync datastore allows you to pull a copy of your data to the device and work with it completely offline. The library provides APIs to synchronize local data with the backend.

This type of datastore is ideal for apps that need to work for long periods without a network connection.

Here is how you should use a Sync store -

// Get an instance
let dataStore = DataStore<Book>.collection(.sync)

// Pull data from your backend and save it locally on the device.
dataStore.pull() { (books, error) -> Void in
    if let books = books {
        //succeed
        print("Books: \(books)")
    } else {
        //fail
    }
}

// Find data locally on the device.
dataStore.find() { (books, error) -> Void in
    if let books = books {
        //succeed
        print("Books: \(books)")
    } else {
        //fail
    }
}

// Save an entity locally on the device. This will add the item to the
// sync table to be pushed to your backend at a later time.

let book = Book()
dataStore.save(book) { (book, error) -> Void in
    if let book = book {
        //succeed
        print("Book: \(book)")
    } else {
        //fail
    }
}

// Sync local data with the backend
// This will push data that is saved locally on the device to your backend;
// and then pull any new data on the backend and save it locally.
dataStore.sync() { (count, books, error) -> Void in
    if let books = books, let count = count {
        //succeed
        print("\(count) Books: \(books)")
    } else {
        //fail
    }
}
// Get an instance
let dataStore = DataStore<Book>.collection(.sync)

// Pull data from your backend and save it locally on the device.
dataStore.pull() { (result: Result<AnyRandomAccessCollection<Book>, Swift.Error>) in
    switch result {
    case .success(let books):
        print("Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

// Find data locally on the device.
dataStore.find() { (result: Result<AnyRandomAccessCollection<Book>, Swift.Error>) in
    switch result {
    case .success(let books):
        print("Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

// Save an entity locally on the device. This will add the item to the
// sync table to be pushed to your backend at a later time.

let book = Book()
dataStore.save(book, options: nil) { (result: Result<Book, Swift.Error>) in
    switch result {
    case .success(let book):
        print("Book: \(book)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

// Sync local data with the backend
// This will push data that is saved locally on the device to your backend;
// and then pull any new data on the backend and save it locally.
dataStore.sync() { (result: Result<(UInt, AnyRandomAccessCollection<Book>), [Swift.Error]>) in
    switch result {
    case .success(let count, let books):
        print("\(count) Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

The pull, push, sync, and syncCount APIs allow you to synchronize data between the application and the backend. This section describes how these APIs work with examples.

Pull Operation

Calling pull() retrieves data from the backend and stores it locally in the Sync Store.

By default, pulling will retrieve the entire collection to the device. Optionally, you can provide a query parameter to pull to restrict what entities are retrieved. If you prefer to only retrieve the changes since your last pull operation, you should enable delta set caching.

The pull API needs a network connection in order to succeed.

let dataStore = DataStore<Book>.collection(.sync)

//In this example, we pull all the data in the collection from the backend
//to the Sync Store
dataStore.pull { books, error in
    if let books = books {
        //succeed
        print("Books: \(books)")
    } else {
        //fail
    }
}
let dataStore = DataStore<Book>.collection(.sync)

//In this example, we pull all the data in the collection from the backend
//to the Sync Store
dataStore.pull { (result: Result<AnyRandomAccessCollection<Book>, Swift.Error>) in
    switch result {
    case .success(let books):
        print("Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

If your Sync Store has pending local changes, they must be pushed to the backend before pulling data to the store.

Push Operation

Calling push() kicks off a uni-directional push of data from the library to the backend.

The library goes through the following steps to push entities modified in local storage to the backend -

  • Reads from the "pending writes queue" to determine what entities have been changed locally. The "pending writes queue" maintains a reference for each entity in local storage that has been modified by the app. For an entity that gets modified multiple times in local storage, the queue only references the last modification on the entity.

  • Creates a REST API request for each pending change in the queue. The type of request depends on the type of modification that was performed locally on the entity.

    • If an entity is newly created, the library builds a POST request.

    • If an entity is modified, the library builds a PUT request.

    • If an entity is deleted, the library builds a DELETE request.

  • Makes the REST API requests against the backend concurrently. Requests are batched to avoid hitting platform limits on the number of open network requests.

    • For each successful request, the corresponding reference in the queue is removed.

    • For each failed request, the corresponding reference remains persisted in the queue. The library adds information in the push/sync response to indicate that a failure occurred. Push failures are discussed in Handling Failures.

  • Returns a response to the application indicating the count of entities that were successfully synced, and a list of errors for entities that failed to sync.

let dataStore = DataStore<Book>.collection(.sync)

//In this example, we push all local data in this datastore to the backend
//No data is retrieved from the backend.
dataStore.push { count, error in
    if let count = count {
        //succeed
        print("Count: \(count)")
    } else {
        //fail
    }
}
let dataStore = DataStore<Book>.collection(.sync)

//In this example, we push all local data in this datastore to the backend
//No data is retrieved from the backend.
dataStore.push(options: nil) { (result: Result<UInt, [Swift.Error]>) in
    switch result {
    case .success(let count):
        print("Count: \(count)")
    case .failure(let error):
        print("Error: \(error)")
    }
}
Handling Push Failures

The push response contains information about the entities that failed to push to the backend. For each failed entity, the corresponding reference in the pending writes queue is retained. This is to prevent any data loss during the push operation. The developer can decide how to handle failures. Here are some options -

  • Retry pushing your changes at a later time. You can simply call push again on the datastore to attempt again.
  • Ignore the failed changes. You can call purge on the datastore, which will remove all pending writes from the queue. The failed entity remains in your local cache, but the library will not attempt to push it again to the backend.
  • Destroy the local cache. You call clear on the datastore, which destroys the local cache for the store. You will need to pull fresh data from the backend to start using the cache again.

Each of the APIs mentioned in this section are described in more detail in the datastore API reference.

Sync Operation

Calling sync() on a Sync Store kicks off a bi-directional synchronization of data between the library and the backend. First, the library calls push to send local changes to the backend. Subsequently, the library calls pull to fetch data in the collection from the backend and stores it on the device.

You can provide a query as a parameter to the sync API, to restrict the data that is pulled from the backend. The query does not affect what data gets pushed to the backend. If you prefer to only retrieve the changes since your last sync operation, you should enable delta set caching.

let dataStore = DataStore<Book>.collection(.sync)

//In this sample, we push all local data for this datastore to the backend, and then 
//pull the entire collection from the backend to the local storage
dataStore.sync() { count, books, error in
    if let books = books, let count = count {
        //succeed
        print("\(count) Books: \(books)")
    } else {
        //fail
    }
}

//In this example, we restrict the data that is pulled from the backend to the local storage,
//by specifying a query in the sync API
let query = Query()

dataStore.sync(query) { count, books, error in
    if let books = books, let count = count {
        //succeed
        print("\(count) Books: \(books)")
    } else {
        //fail
    }
}
let dataStore = DataStore<Book>.collection(.sync)

//In this sample, we push all local data for this datastore to the backend, and then
//pull the entire collection from the backend to the local storage
dataStore.sync() { (result: Result<(UInt, AnyRandomAccessCollection<Book>), [Swift.Error]>) in
    switch result {
    case .success(let count, let books):
        print("\(count) Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

//In this example, we restrict the data that is pulled from the backend to the local storage,
//by specifying a query in the sync API
let query = Query()

dataStore.sync(query) { (result: Result<(UInt, AnyRandomAccessCollection<Book>), [Swift.Error]>) in
    switch result {
    case .success(let count, let books):
        print("\(count) Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}
Sync Count Operation

You can retrieve a count of entities modified locally and pending a push to the backend.

// Number of entities modified offline.
let count = dataStore.syncCount()

Cache

Configuring your datastore as a Cache datastore allows you to use the performance optimizations provided by the library. In addition, the cache allows you to work with data when the device goes offline.

The Cache mode is the default mode for the data store. Most of the time you don't need to set it explicitly.

This type of datastore is ideal for apps that are generally used with an active network, but may experience short periods of network loss.

Here is how you should use a Cache store -

// Get an instance
let dataStore = DataStore<Book>.collection(.cache)

// The completion handler block will be called twice,
// #1 call: will return the cached data stored locally
// #2 call: will return the data retrieved from the network
dataStore.find() { books, error in
    if let books = books {
        //succeed
        print("Books: \(books)")
    } else {
        //fail
    }
}

// Save an entity. The entity will be saved to the device and your backend. 
// If you do not have a network connection, the entity will be stored locally. You will need to push it to the server when network reconnects, using dataStore.push().
let book = Book()
dataStore.save(book) { book, error in
    if let book = book {
        //succeed
        print("Book: \(book)")
    } else {
        //fail
    }
}
// Get an instance
let dataStore = DataStore<Book>.collection(.cache)

// The completion handler block will be called twice,
// #1 call: will return the cached data stored locally
// #2 call: will return the data retrieved from the network
dataStore.find() { (result: Result<AnyRandomAccessCollection<Book>, Swift.Error>) in
    switch result {
    case .success(let books):
        print("Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

// Save an entity. The entity will be saved to the device and your backend.
// If you do not have a network connection, the entity will be stored locally. You will need to push it to the server when network reconnects, using dataStore.push().
let book = Book()
dataStore.save(book, options: nil) { (result: Result<Book, Swift.Error>) in
    switch result {
    case .success(let book):
        print("Book: \(book)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

The Cache Store executes all CRUD requests against local storage as well as the backend. Any data retrieved from backend is stored in the cache. This allows the app to work offline by fetching data that has been cached from past usage.

The Cache Store also stores pending write operations when the app is offline. However, the developer is required to push these pending operations to the backend when the network resumes. The push API should be used to accomplish this.

Network

Configuring your datastore as Network turns off all caching in the library. All requests to fetch and save data are sent to the backend.

We don’t recommend this type of datastore for apps in production, since the app will not work without network connectivity. However, it may be useful in a development scenario to validate backend data without a device cache.

Here is how you should use a Network store -

// Get an instance
let dataStore = DataStore<Book>.collection(.network)

// Retrieve data from your backend
dataStore.find() { (books, error) -> Void in
    if let books = books {
        //succeed
        print("Books: \(books)")
    } else {
        //fail
    }
}

// Save an entity to your backend.
let book = Book()
dataStore.save(book) { (book, error) -> Void in
    if let book = book {
        //succeed
        print("Book: \(book)")
    } else {
        //fail
    }
}
// Get an instance
let dataStore = DataStore<Book>.collection(.network)

// Retrieve data from your backend
dataStore.find() { (result: Result<AnyRandomAccessCollection<Book>, Swift.Error>) in
    switch result {
    case .success(let books):
        print("Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

// Save an entity to your backend.
let book = Book()
dataStore.save(book, options: nil) { (result: Result<Book, Swift.Error>) in
    switch result {
    case .success(let book):
        print("Book: \(book)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

Granular Control

Selecting a DataStoreType is usually sufficient to solve the caching and offline needs of most apps. However, should you desire more control over how data is managed in your app, you can use the granular configuration options provided by the library. The following sections discuss the advanced options available on the DataStore.

In this section:

Data Reads

The way to control Data Reads is to set a ReadPolicy.

Read Policy

ReadPolicy controls how the library fetches data. When you select a store configuration, the library sets the appropriate ReadPolicy on the store. However, individual reads can override the default read policy of the store by specifying a ReadPolicy.

The following read policies are available:

  • ForceLocal - forces the datastore to read data from local storage. If no valid data is found in local storage, the request fails.
  • ForceNetwork - forces the datastore to read data from the backend. If network is unavailable, the request fails.
  • Both - reads first from the local cache and then attempts to get data from the backend.

Examples

Assume that you are using a datastore with caching enabled (the default), but want to force a certain find request to fetch data from the backend. This can be achieved by specifying a ReadPolicy when you call the find API on your store.

//create a Sync store
let dataStore = DataStore<Book>.collection(.sync)

//force the datastore to request data from the backend
dataStore.find { books, error in
    if let books = books {
        //succeed
        print("Books: \(books)")
    } else {
        //fail
    }
}
//create a Sync store
let dataStore = DataStore<Book>.collection(.sync)

//force the datastore to request data from the backend
dataStore.find { (result: Result<AnyRandomAccessCollection<Book>, Swift.Error>) in
    switch result {
    case .success(let books):
        print("Books: \(books)")
    case .failure(let error):
        print("Error: \(error)")
    }
}

A tidier approach is to set readPolicy through the Options structure.

let dataStore = DataStore<Book>.collection(.cache)
let options = Options(readPolicy: .forceNetwork)
dataStore.find(query, options: options) { (result: Result<[Book], Swift.Error>) in
    switch result {
    case .success(let books):
        print("change this line to handle succeed")
    case .failure(let error):
        print("change this line to handle failure")
    }
}
Delta Set Caching

The library implements a mechanism to optimize the amount of data retrieved from the backend. When you use a Sync or Cache datastore, data requests to the backend only fetch data that changed since the previous update. We call this mechanism "Delta Set Caching".

Delta set caching applies to Sync and Cache data stores. To use delta set caching, you need to enable it on the store.

//create a Sync data store with delta set caching enabled
let dataStore = DataStore<Book>.collection(.sync, deltaSet: true)
//create a Sync data store with delta set caching enabled
let dataStore = DataStore<Book>.collection(.sync, deltaSet: true)

How it works

Each DataStore maintains metadata per entity such as the last time the entity was updated. When the library requests data from the backend, it makes multiple calls to the backend - first to retrieve the metadata of each entity, and subsequently (if required) to fetch only those entities that changed since the previous update.

Discussion

  • To request a delta update from the backend, the library will make the following two API calls -

    • To retrieve the metadata for entities, the original request for data is appended with the fields modifier, as follows:

      # only retrieve the _id and lmt fields for entities that match the query
      ?query={...}&fields=_id,_kmd.lmt
    • To retrieve the specific entities that have changed, the library makes a request with the in query filter. If necessary, the library may batch entities into multiple requests, to avoid hitting platform limits on URL length.

      # retrieve specific entities by id
      ?query={"_id":{"$in": ["<entity1.id>", "<entity2.id>", ...]}}

Delta set caching requires that your backend support the query modifiers described above - fields and $in. All Kinvey out-of-the-box connectors are built to support fields and $in. To use delta set caching, custom connectors are required to support these query modifiers.

  • The performance benefits of delta set caching are most noticeable on large collections that have low rate of change. That said, our tests show that in most cases, performance is improved with delta set caching enabled.

Data Writes

Write Policy

When you save data, the type of datastore you use determines how the data gets saved.

For a store of type Sync, data is written to your local copy. In addition, the library maintains additional information in a "pending writes queue" to recognize that this data needs to be sent to the backend when you decide to sync or push.

For a store of type Cache, data is written first to your local copy and sent immediately to be written to the backend. If the write to the backend fails (e.g. because of network connectivity), the library maintains information to recognize that this data needs to be sent to the backend when connectivity becomes available again. Due to platform limitations, this does not happen automatically, but needs to be initiated from the user by calling the push() or sync() methods.

For a store of type Network, data is sent directly to the backend. If the write to the backend fails, the library does not persist the data for a future write.

The following example sets Network as a write policy. Set this way, this write policy will be used only for the save operation being requested, overriding the default Cache mode for the collection.

let book = Book()
let dataStore = DataStore<Book>.collection()
let options = Options(writePolicy: .forceNetwork)
dataStore.save(book, options: options) { (result: Result<Book, Swift.Error>) in
    switch result {
    case .success(let book):
        print("change this line to handle success")
    case .failure(let error):
        print("change this line to handle failure")
    }
}
Conflicts

When using sync and cache stores, you need to be aware of situations where multiple users could be working on the same entity simultaneously offline. Consider the following scenario:

  1. User X edits entity A offline.
  2. User Y edits entity A offline.
  3. Network connectivity is restored for X, and A is synchronized with Kinvey.
  4. Network connectivity is restored for Y, and A is synchronized with Kinvey.

In the above scenario, the changes made by user X are overwritten by Y.

The libraries and backend implement a default mechanism of "client wins", which implies that the data in the backend reflects the last client that performed a write. Custom conflict management policies can be implemented with Business Logic.

Timeout

When performing any datastore operations, you can pass a timeout value as an option to stop the datastore operation after some amount of time if it hasn't already completed.

Kinvey.sharedClient.timeoutInterval = 120 //an NSTimeInterval value in seconds, which means 2 minutes in this case

The default timeout for the iOS library is 60 seconds.

Related Samples

Got a question?