Skip to content

Export

The Exporter builds a CSV, YAML or JSON file from a schema’s objects. It is reached from the sidebar (Tools > Exporter), or from a schema’s object list through Action > Export, which prefills it with that list’s current view and search query.

Requires the p_data_export permission, plus read permission on the schema being exported.

Source

Schema - the schema to export. The selector only lists schemas you may read.

Search query - the same syntax as the list page search box. Leave it blank to export every object. There is no * wildcard here: a blank query is “everything”. Only objects you are allowed to read are written to the file.

Columns

The panel on the right of the form lists every column the schema can export, one checkbox each, showing the field name and its displayname. The boxes you tick, in the order they are listed, are the columns of the file. There is no way to reorder them.

_schema and keyname are always ticked and cannot be unticked. _schema is what makes the file re-importable; keyname is what identifies the row.

Select all / Select none toggles every other box at once; the button label follows the current state.

What is offered, in order:

Group Columns
Builtin attributes displayname, is_enabled, last_sync, last_update
Schema fields every declared field, in schema order
injected subfields resolved through a schema reference or an external field, named <field>__<subfield> (for example myuser__email)

Injected columns start unticked. They are computed at read time: useful in a report, but not written back if the file is re-imported. Their names are derived from the schema and from one sample object, so treat the list as a hint - an object without the reference set exports an empty cell rather than failing.

Enumerate subfields are not listed. They come from the value each object selected, so the list differs from object to object. In YAML and JSON they are exported automatically, right after their parent, whenever the enumerate column itself is ticked; they are never exported to CSV, where one fixed header cannot depend on each row’s value.

Arriving through Action > Export from an object list pre-ticks the columns of the view that list was showing, instead of the default set. A view column the panel cannot offer is left unticked. Nothing here changes which view your list page shows next.

An export is a snapshot of the schema as it is now: a field added later is not picked up by a bookmarked URL or a repeated export - re-open the Exporter and tick the new box.

Format

CSV, YAML or JSON. The CSV delimiter and encoding selectors only apply to CSV.

Text fields (dataformat: text) are never written to CSV - they routinely contain newlines and delimiters. They are exported normally in YAML and JSON.

The schema column

Always present, and not an option: it is what makes an export re-importable. cavaliba load and the Import Tool need either this column, a pipeline, or a forced schema to know what each row is.

It is the leading _schema column in CSV, and the classname key in YAML/JSON - the YAML/JSON file carries classname only, never a second _schema key.

The CSV column is named _schema, with the underscore, on export. The underscore marks it as a directive rather than a real field, like _action.

On import, _schema and classname are both accepted - so an exported file loads back unchanged, and a file written by hand can use the same name as YAML and JSON. schema, without the underscore, is not recognised by the Import tool: such a column is treated as an ordinary field and the row is refused for having no schema. (cavctl load, which parses CSV on the client, additionally accepts schema.)

Options

Add an _action field - adds an _action column (CSV) or key (YAML/JSON) with the same value on every row, so the file is ready to be edited and re-imported. noop is the safe choice: re-importing it unchanged does nothing.

In CSV the column is optional and, when present, comes first - ahead of _schema - so the cell you edit before re-importing is the leftmost one in a spreadsheet.

Inline refs (YAML/JSON) - a comma-separated list of schema names, or * for all. Each referenced object is inlined under a _refs key. Only reference fields actually selected for export contribute.

Revisions (YAML/JSON) - include the last N revisions of each object under a _revision key.

Background exports

Every export runs as a background task, whatever its size. A progress window opens showing percent complete, total, done, success and error counts, elapsed and remaining time, the rate in objects per second, and the first few errors. When the task finishes, a download link appears in the same window.

While the export is running, the window offers two buttons:

  • Abort - stops the task; the partial file is deleted, never offered for download.
  • Hide - closes the window and leaves the task running. You can pick it up again from the Tasks menu, and stop it from there: following and stopping your own tasks needs no permission.

Clicking outside the window or pressing Escape does nothing: these buttons are the only way out while the export runs.

Once the export is finished (or aborted, or failed), the window shows Close instead: there is nothing left to follow, and the download link is right there.

An export aborts by itself if more than CAVALIBA_MAX_EXPORT_ERROR objects (default 20) fail.

Note that Total counts the objects matched by the query, while Success counts the ones actually written: objects you are not allowed to read are scanned but skipped.

An export needs a running task worker. If none can be reached, the window says Export failed and gives the reason instead of leaving you waiting.

Failed objects

The window lists the first few failed objects. When there is at least one, a Download the N error(s) link below the list gives the full list as a text file, one message per line - the same list the task’s detail page shows under errors.

A typical case is a value the chosen CSV encoding cannot represent: that object is counted as an error and skipped, and the rest of the export is written as usual.

Result files

An export’s file is kept for CAVALIBA_EXPORT_FILE_TTL_HOURS (default 24, so one day) and then removed by the nightly housekeeping. Since every export produces one, expect a day’s worth of export files to sit in the file store.

There are three ways to get to it:

  1. the download link in the progress window, as soon as the export finishes;
  2. the Tasks menu in the top bar - a finished export shows a download icon directly in the task list;
  3. the task’s own detail page, under Result.

Routes 2 and 3 are what you use after hiding the window or leaving the page. They need no task permission - the Tasks menu shows your own tasks to any logged-in user - but downloading the file still requires p_data_export, and only the task’s owner can do it, or anyone with p_task_all.

The handle is in the URL, so the direct link /data/private/exporter/result/<handle>/ also works if you kept it.

Download page

Tools > Download lists the files produced by the exporter.sh cron script in the export folder (CAVALIBA_EXPORT_FOLDER), for preview or download. This is a different area from the result files above, and requires p_data_export_viewer.

After an upgrade

The sidebar and dashboard are driven by the SIDEBAR_CONTENT / DASHBOARD_CONTENT configuration entries, which are only seeded from their defaults when no stored value exists yet. So on an install that already has them:

  • The existing Export entry needs no change - it keeps its app: export key and now opens the Exporter.
  • The new Download entry is not added automatically. Add - app: download to the Tools section of SIDEBAR_CONTENT (and of DASHBOARD_CONTENT if you want the tile) under Configuration > home. Until then the page is only reachable at /data/private/download/.

Spreadsheet formulas

Cell values are written exactly as stored. A value starting with =, +, - or @ is treated as a formula by Excel and LibreOffice when the CSV is opened directly.

Cavaliba deliberately does not escape them: escaping changes the value, and the whole point of the _schema / _action columns is that the file can be re-imported unchanged. For an export whose content you do not control, open it through the spreadsheet’s import dialog (as text) rather than by double-clicking it.

Notes

  • In a development setup with CAVALIBA_TASK_CELERY=sync, there is no worker: the export runs inside the request instead, so the progress window opens on an export that has already finished.
  • The worker must share the same CAVALIBA_FILESTORE volume as the web nodes, exactly like the Import Tool.
  • The column panel enumerates its selection, so the UI always asks for an explicit column list. The API and CLI (cavaliba get) instead accept the filter grammar the UI no longer shows - * for every column, !a,b for every column except those, or an explicit comma-separated list. In that filter mode, CSV never emits injected subfields: the header is resolved from the first exported object, and an object without the reference set would write one cell less and shift every following column. Naming an injected column explicitly works in every mode.