Data

DATA Introduction

Data objects in Cavaliba are best understood with the following concepts in mind.

Schema are top-level asset families : Devices, Applications, Laptops, Projects, Customers, Books, Facilities, Cooking Receipes, Invoices, etc.

Fields describes attributes for each Schema. Fields and Schema can be modified at any time during the life of the system: add or remove fields, change constraints.

FieldTypes - fields can be of a regular data types such as string, int, float, boolean, date, IP addresses, … They can have a more complex structure : other objects, enumerate (static) lists, users and groups, external and computed fields, etc. Fields can be single or multi-valued. Fields can be mandatory or optional. Fields can enforce constraints (eg. an integer field must be greater than 4)

Instance / asset / object : theses terms refer to individual items managed by the Cavaliba Data solution. Each Instance belong to one Schema, and has values for some or all fields related to their Schema.

Schemas (and their Instances) have relationships : a geographical sites is located in a region or country. A virtual machine is hosted on a physical server, etc.

YAML - Schema, Fields, Instances can be provided as regular YAML (or JSON) files. They can be uploaded to Cavaliba from UI, API or CLI command, with appropriate permissions. You can also import CSV file.

Enumerate is a special FieldType which provides predefined list of values. Think of enumerates as static lists. Enumerate are also regular objects with a Schema, etc. Enumerates can provide more than a single value per choice (eg. A country Enumerate could provide both the ISO code as well as as human-friendly name for the country). Enumerate can be seen as mimnal/static Schema.

Pipelines are compoents used to transform data at (bulk) import time. When uploading a CSV, rather than asking the sender to adapt to your exact schema, use a pipeline to define mapping and transformation. A large set of operators is available. In itself, a pipeline is a regular Cavaliba object, with a Schema and specific attributes. You can manage pipelines with the Web UI, CLI, YAML import files, etc.

Dataviews are object used to select a subset of Schema fields to be presented to your users in the Web UI for fast access. It helps to provide several dataviews for complex Schema. Dataviews are also regular objects with a Schema and specific attributes and can be managed as such.

Once Schema are created (or loaded from provided demo schema), Cavaliba Data Management automatically provides :

• a Responsive Web UI for humans : create, view, edit, import, export, delete, enable/disable assets.
• a REST API for machines: same operations, single instance/field level, or massive bulk transfer
• a CLI command to import/export in batch from/to external systems
• a relationship framework, including inheritance / propagation of fields and objects between Instances
• a Role/Permission model to handle authorization at Schema/Instance/Field level
• a Data storage at scale on standard relationnal databases (PostgreSQL, MariaDB/MySQL, …) with an Entity-Attribute-Value (EAV) dynamic datamodel.