# Datasets

A Dataset is used to store your data in the cloud. You can interact with your data via a REST API and our SDKs. All data is protected with policy authorization. You can establish a relation between different datasets or other parts of the platform. You do not need to worry about database performance and security, such as: indexes, foreign keys, scaling, backups, SQL injections and other things related to database management, as we do all this for you to help speedup development. Each project has it's own instance of Dataset and other projects can't impact the stability and security of your project.

Datasets have built-in support for validation, schema & schemaless data and default values for fields.

To create a dataset, click on the Go to project button and then on the next screen Create dataset:

Create Dataset

The name of the dataset is used as an endpoint ( For example: *.app.jexia.com/ds/dataset_name) to allow you to communicate with the automatically generated REST API for your project. There are a few rules regarding dataset names, these are:

  • The name of your dataset can contain only Latin characters and digits.
  • The name of the dataset has to start with a character.

Strict mode - informs the platform to apply strict rules about input data validation. If an input object has extra fields that do not exist in the dataset schema's insert or update operation, it will be rejected.

# Configuration

# Schema

The next step is to add fields to your datasets. To create a field, click the Add field button. In the same window, you can input the different values for name, type, and validation of your field. You can also provide a default value for the field. Field name and validation parameters can be changed in the future via the edit field. However, the field type cannot be changed. If you want to change the type then you can only delete and create the field again. However, by deleting the field you will also lose the data stored in that field. With the Schema approach, you can set specific types for each field: String, Integer, Float, Date, DateTime, Boolean, JSON or UUID. Before a create or update action, the data will be validated against the validators.

Create Field

TIP

If you send a JSON object which has additional fields that are not set within the schema, those fields will be saved as schemaless fields. For those fields, validation rules and default values are not applicable.

# Schemaless

To apply the schemaless approach, just insert your JSON object into a dataset without creating any schema fields for the dataset. The data will be stored automatically with the type provided inside the JSON. Please, note that validations and default values do not apply to schemaless data. You can convert from Schemaless to Schema when the design for your project is stabilized. Jexia supports the following types: String, Integer, Float, Date, DateTime, Boolean, JSON and UUID as field types.

WARNING

Please, keep in mind when you convert the field from schemaless to schema, the data will not be migrated to the appropriate schema field. You need to do it on your own and control the quality of data being inputted.

If you delete fields from a schema, the data related to that field will be deleted as well. It will not be converted back to schemaless.

When fetching data, if a field matches one on the dataset, the schema data will be returned. However, if a field does not match one on the dataset, it will be searched for via a schemaless method. The declared schema field takes priority over that of a schemaless field.

In case you have established relation between datasets, if a field matches one on parent dataset, the schema data will be returned, then the field will be search for in all child schemes. If a field does not match one on any of the related datasets, it will be searched for via a schemaless method. The declared schema field within the parent takes priority over that of both a child schema and schemaless field. However, the declared child schema field takes priority over that of any schemaless field.

# Validation

This depends on the field type. The most validators are available for a string type. Such as: Required, UpperCase, LowerCase, Alphanumeric, Numeric, Alpha, Min/ Max length and RegEx pattern matching.

For Float and Integer, there are only: Required and Min/ Max value validators.

In the future, we plan to add Date range and other validators.

You might see that when you select some validators, others may become unavailable. This is due to logical exclusion. For example, it is not logical to have Upper and Lower case validators at the same time. To reduce the possibility of human mistakes we decided to disable selection for some combinations.

TIP

Please keep in mind that validation is applicable for schema fields only. Jexia applies the same validation rules for every Create and Update action.

# Default values

You can set up default values for each field. This value will be validated against type and validation constraints.

WARNING

Please keep in mind that for a string type it is not possible to set a default value as an empty string '', you can get either set a value or input null which will be passed as a null type.

# Create

To create a record in Jexia's dataset you need to create an action within either a policy for a User or API key. Below you can see the User approach as it has a wider use case for record creation.

TIP

Please keep in mind that the API always returns an array of records, even if you only insert one record. Because of this, you can apply the same approach for data manipulation.

  • JavaScript
  • Python
  • cURL
import { jexiaClient, dataOperations, field } from "jexia-sdk-js/node"; 
const ds = dataOperations();

jexiaClient().init({
  projectID: "PROJECT_ID",
  key: "API_KEY",
  secret: "API_SECRET",
  zone: "PROJECT_ZONE",
}, ds);
  
let orders_data = [{
      "title":"Order1",
      "total":10,
      "verified":false
  }, {
      "title":"Order2",
      "total":100,
      "verified":false
  }]

const orders = ds.dataset("orders");
const insertQuery = orders.insert(orders_data);  
insertQuery.subscribe(records => { 
      // You will always get an array of created records, including their 
      // generated IDs (even when inserting a single record) 
    }, 
    error => { 
      // If something goes wrong, the error information is accessible here 
});
  

After execution, you will receive an array similar to the following array of objects:

[{
    "id": "e0e17683-f494-4f33-8343-ffed792b324e",
    "created_at": "2020-02-15T19:43:39.784342Z",
    "updated_at": "2020-02-15T19:43:39.784342Z",
    "title":"Order1",
    "total":10,
    "verified":false
}, {
    "id": "e0e17683-f494-4f33-9563-dded795e3121",
    "created_at": "2020-02-15T19:43:39.784342Z",
    "updated_at": "2020-02-15T19:43:39.784342Z",
    "title":"Order2",
    "total":100,
    "verified":false
}]

# Read

To fetch your data you need to have the Read action selected on a policy which also contains the particular resource you are trying to access. You can apply different filters to get specific data. In the following example you can see an API key usage as the most common approach.

  • JavaScript
  • Python
  • cURL

Due to the JS SDK being built on top of RxJS. Once installed, you can use the power of the RxJS library and use all available methods provided by this library.

// Jexia client
import { jexiaClient, dataOperations, field } from "jexia-sdk-js/node"; 

const ds = dataOperations();

jexiaClient().init({
  projectID: "PROJECT_ID",
  key: "API_KEY",
  secret: "API_SECRET",
  zone: "PROJECT_ZONE",
}, ds);

const orders = ds.dataset("orders");
const selectQuery = orders
  .select()
  .where(field => field("verified").isEqualTo(true))
  // .where(field("title").isDifferentFrom("test")) 
  // .where(field("total").isBetween(1,30))
  // .where(field("total").isEqualOrGreaterThan(15))
  // .where(field("total").isEqualOrLessThan(7))
  // .where(field("total").isEqualTo(100))
  // .where(field("total").isGreaterThan(57))
  // .where(field("total").isLessThan(100))
  // .where(field("id").isInArray(my_val))   // my_val=[uuid1,uuid2];
  // .where(field("id").isNotInArray(my_val)) // my_val=[uuid1,uuid1];
  // .where(field("title").isLike("Charlotte's Web"))
  // .where(field("title").isNotNull())
  // .where(field("title").isNull())
  // .where(field("title").satisfiesRegex('a-z0-9'))   
selectQuery.subscribe(records => { 
    // You will always get an array of created records, including their 
    // generated IDs (even when inserting a single record) 
  }, 
  error => { 
    // If something goes wrong, the error information is accessible here 
});

After execution, you will receive an array similar to the following array of objects:

[{
    "id": "e0e17683-f494-4f33-9563-dded795e3121",
    "created_at": "2020-02-15T19:43:39.784342Z",
    "updated_at": "2020-02-15T19:43:39.784342Z",
    "title":"Order2",
    "total":100,
    "verified":true
}]

# Delete

To delete a record you need to have selected the Delete action in the policy for the resource you intend to edit. Some examples can be seen below using the Project User method.

When you perform a Delete action, you will get back an array of affected records so you can sync changes with your front-end application.

  • JavaScript
  • Python
  • cURL
import { jexiaClient, dataOperations, field } from "jexia-sdk-js/node"; 

const ds = dataOperations();

jexiaClient().init({
  projectID: "PROJECT_ID",
  key: "API_KEY",
  secret: "API_SECRET",
  zone: "PROJECT_ZONE",
}, ds);

const orders = ds.dataset("orders");
const deleteQuery = orders
.delete()
.where(field => field("id").isEqualTo("2a51593d-e99f-4025-b20b-159e226fc47d"));  

deleteQuery.subscribe(
  records => { 
    // You will always get an array of created records, including their 
    // generated IDs (even when inserting a single record) 
  }, 
  error => { 
    // If something goes wrong, the error information is accessible here 
});
 

After execution, you will receive an array similar to the following array of objects:

[{
    "id": "e0e17683-f494-4f33-9563-dded795e3121",
    "created_at": "2020-02-15T19:43:39.784342Z",
    "updated_at": "2020-02-15T19:43:39.784342Z",
    "title":"Order2",
    "total":100,
    "verified":true
}]

# Update

To update a record you need to have selected the Update action in the policy for the resource you intend to edit. Some examples can be seen below using the Project User method.

When you perform an Update action, you will get back an array of affected records so you can sync changes with your front-end application.

TIP

You can add an id field into the update object, Jexia will find and update it automatically.

  • JavaScript
  • Python
  • cURL
import { jexiaClient, dataOperations, field } from "jexia-sdk-js/node"; 
const ds = dataOperations();

jexiaClient().init({
  projectID: "PROJECT_ID",
  key: "API_KEY",
  secret: "API_SECRET",
  zone: "PROJECT_ZONE",
}, ds);

const orders = ds.dataset("orders");
const updateQuery = orders
  .update([{id:"3005a8f8-b849-4525-b535-a0c765e1ef8e", verified: true }]) // To update 1 record with specific ID
  //.where(field => field("total").isBetween(0,50).and(field("name").isLike('%avg'))); // To update update batch of records 

updateQuery.subscribe(records => { 
    // You will always get an array of created records, including their 
    // generated IDs (even when inserting a single record) 
  }, 
  error => { 
    // If something goes wrong, the error information is accessible here 
});
 

  

After execution, you will receive an array similar to the following array of objects:

[{
    "id": "3005a8f8-b849-4525-b535-a0c765e1ef8e",
    "created_at": "2020-02-15T19:43:39.784342Z",
    "updated_at": "2020-02-16T13:15:00.784342Z",
    "title":"Order1",
    "total":10,
    "verified":true
}]

If you created multiple datasets you can establish relations between them. You can do it under the Relations section. Currently, Jexia supports relation types of:

  • One to One
  • One to Many
  • Many to One
  • Many to Many

Relations

When do you need this? For example, you can keep users and their TODOs in separate datasets. With Jexia you do not need to worry about external keys and index optimizations to organize all of this, it will be automatically managed by Jexia.

Another cool thing, as soon as you set up a relation, you can insert an object which has a parent / child relation inside and Jexia will automatically put data in the proper places. For example, I have dataset: orders and dataset: items with a one to many relation. So I can insert the following object into orders and the data will automatically be organized in its proper place:

{
    "title":"Order1",
    "total":10,
    "verified":false,
    "items": [
        {
            "name":"Item 1",
            "qty":2
        },
        {
            "name":"Item 3",
            "qty":20
        }
    ]
}

During fetching data you can specify if you want to get only the parent or parent and child data.

  • JavaScript
  • Python
  • cURL
sdk.dataset("orders")
  .select()
  .related("items", items => items.fields("qty"))
  .subscribe(
    res => {
      console.log(res);
    },
    error=>{
      console.log(error)
    });

After execution, you will receive an array similar to the following array of objects:

[{
  "id": "...",
  "created_at": "...",
  "updated_at": "...",
  "title":"Order1",
  "total":10,
  "verified":false,
  "items": [
    {
      "id": "...",
      "qty": 2
    },
    {
      "id": "...",
      "qty":20
    }
  ]
}]

If you want to get other related data, you just need to add them to request, Jexia will do matching automatically and send it back in result JSON. Easier than GaphQL, yeh? 😄

TIP

Please keep in mind that currently it is not possible to make a relation with a dataset itself (for example for multi-level menu). It is still possible to create a child and have a one to many relation between them and the dataset however.

# Multi-level Relations

You can have multiple levels of relation. For example: Article / Comments / Author / Salary. With Jexia you can build such a case and you will be able to fetch all data in one response.

  • JavaScript
  • Python
  • cURL
dom.dataset("article")
  .select()
  .related("comments", comments => comments
    .fields("name", "likes")
    .related("authors"), author => author
      .fields("name", "id")
      .related("employee"), data => data
          .fields("salary", "id")
      )
    )
  )
  .subscribe(...)

# Attach and Detach records

If you need create a relation between already existing data, you can use the .attach() and .detach() methods. For this you need to specify which parent for which you want to attach the child record. In the example that follows, it is creating a relation between order (with id = my_uuid ) and two items objects with the IDs b4961b6a-85a2-4ee8-b946-9001c978c801 and e199d460-c88f-4ab9-8373-1d6ad0bd0acb.

  • JavaScript
  • Python
  • cURL
dom.dataset("order")
    .where(field("id").isEqualTo('my_uuid'));
    .attach("items", [
      "b4961b6a-85a2-4ee8-b946-9001c978c801",
      "e199d460-c88f-4ab9-8373-1d6ad0bd0acb",
    ])
    .subscribe();

# This feature is only available for projects with a professional subscription. PRO Real-Time Notifications

If you want to have real-time updates regarding changes on datasets, you can use a real-time notification which is built into the dataset, fileset and project user modules.

After an action has taken place, you will get a notification containing the record ID which was modified. You can then re-fetch this data from Jexia. We only send the ID for security reasons, as there might be a situation where many users will be subscribed to notifications but they should not have access to data itself. With the current approach, you can decide whom to show the data.

You can use the .watch() method to subscribe to the notifications. Allowed actions can be provided either as arguments or as an array:

  1. created
  2. updated
  3. deleted
  4. all (used by default)

You can unsubscribe from notifications any time by calling the .unsubscribe() method. Keep in mind that you would need to import the realTime module from our SDKs.

import { jexiaClient, dataOperations, realTime } from "jexia-sdk-js/node";
const ds = dataOperations();
const rtc = realTime();

// Initialize Jexia client
jexiaClient().init(credentials, ds, rtc);

const subscription = ds.dataset("orders")
  .watch("created", "deleted")
  .subscribe(messageObject => {
    console.log("Realtime message received:", messageObject.data);
  }, error => {
    console.log(error);
  });

// here put Insert or Delete operations

subscription.unsubscribe();

# Filtering

You can use filtering to specify which data to return. You will always receive an array of objects, independent on the number of objects. There are different approaches applied to different languages. Please check your preferable method.

  • JavaScript
  • Python
  • cURL
import { field } from "jexia-sdk-js/node"; 
...

// Below are some examples using filters:
.where(field("title").isDifferentFrom("test")) 
.where(field("total").isBetween(1,30))
.where(field("total").isEqualOrGreaterThan(15))
.where(field("total").isEqualOrLessThan(7))
.where(field("total").isEqualTo(100))
.where(field("total").isGreaterThan(57))
.where(field("total").isLessThan(100))
.where(field("id").isInArray(["uuid1","uuid2"]))
.where(field("id").isNotInArray(["uuid1","uuid2"]))
.where(field("title").isLike("%oby"))
.where(field("title").isNotNull())
.where(field("title").isNull())
.where(field("title").satisfiesRegex('[A-Z][0-9]*')) 

const isAuthorTom = field("user_name").isEqualTo("Tom");  
const isAuthorDick = field("user_name").isEqualTo("Dick");  

// Example for applying AND / OR grouping.
const isAuthorTomOrDick = isAuthorTom.or(isAuthorDick);  
const isAuthorTomOrDick = isAuthorTom.and(isAuthorDick);  

// In order to use these conditions,
// they need to be added to a query through the `.where` method.
ds.dataset("posts")  
 .select()
 .where(isAuthorTomOrDick)
 .subscribe(records => {}, error=>{}) // posts of Tom and Dick); 

# Response Fields

Sometimes you will want to show specific fields from record instead of the whole record. Jexia enables you to do this by allowing you to specifying what fields you want have returned. This is applicable to related data.

  • JavaScript
  • Python
  • cURL
const orders = ds.dataset("orders");

orders.select()
  .fields("title", "items.qty") // You can also pass an array of field names 
  .subscribe(records => {}, err=>{}); // You will get array of {id, title, author} please keep in mind "id" is always returned

// You can also omit fields() method and pass requred fields directly to the select()
// this code is identical to the prevous one:
orders.select("title", "items.qty")
  .subscribe(records => {}, err=>{});
[
  {
    "title": "Order1",
    "id": "bc3f13fd-5e0c-4319-98d4-f4373222488f",
    "items":[
      {
        "id":"...",
        "qty":2
      }
    ]
  }
]

# Limits & Offsets

You can use limit and offset on a query to paginate your records. They can be used separately or together. Only setting the limit (to a value of X) will make the query operate on the first X records. Only setting the offset (to a value of Y) will make the query operate on the following Y records, starting from the offset value.

  • JavaScript
  • Python
  • cURL
const orders = ds.dataset("orders");

orders.select()
  .limit(2)
  .offset(5)
  .subscribe(records =>{}, err=>{}) // Will return an array of 2 records, starting from position 5

# Sorting

To sort the data before it is returned, you can apply sort methods. These can be Asc and Desc directions.

  • JavaScript
  • Python
  • cURL
const orders = ds.dataset("orders");

posts
  .select()
  .sortAsc("total")
//.sortDesc("total")
  .subscribe(records => { 
    // You've got sorted records here 
  }, err=>{});

# Aggregation Functions

There are a few aggregation functions you can use in order to complete calculations before obtaining data:

  1. max
  2. min
  3. sum
  4. avg
  5. count

WARNING

Please keep in mind that you can aggregate fields only from schema. We do not support aggregation for fields from schemaless.

You can combine output with other fields.

  • JavaScript
  • Python
  • cURL
const posts = ds.dataset("orders");

posts.select()
  .fields({ fn: "sum", field: "total", alias: "sum_total"})
  .subscribe(result => {
  }, err=>{});

After execution, you will receive an array similar to the following array of objects:

[
  {
    "sum_total": 202
  }
]

# REST API Errors

During REST API request you might get the following errors in responds:

Code Description
400 Bad request. The request was somehow malformed and was not executed.
401 Invalid authentication. Access token was not provided or incorrect.
403 Forbidden. Access token does not have permission to insert the record(s) into this dataset.
404 Dataset not found
500 There is an internal error

# Limitations

Dataset has a response size limit of 250Kb. If you receive the error response is too big, try to apply filters or fetch only specific fields for the response.