Manta Developers
GitHub
Menu

Upload

Send files into a datastore with the manta-upload CLI.

What it is

@manta-eu/upload is the manta-upload command. It uploads files from a script, a scheduled job, or another system’s export, using the same flow as a chat upload.

CSV, Parquet, Turtle, and N-Triples files are ingested into the datastore’s knowledge graph. Any other type is stored but not ingested. Pass --bundle to store a file without ingesting it, for files that are loaded together as a bundle.

The API accepts files up to 5 GiB. For the four ingested types it reads the start of the file (and, for Parquet, the end) and deletes a file whose bytes contradict its extension: a .parquet without Parquet’s magic bytes, or a .csv, .ttl, or .nt that is not UTF-8 text. Re-save a CSV exported as Windows-1252 or ISO-8859-1 as UTF-8 first. Ingest has its own limit and skips a file over 512 MiB by default.

Ingest runs in batches, a few minutes after the upload. The command targets hiremanta.com production unless --api-url, --keycloak-url, and --keycloak-realm name another deployment.

Import

The package is a CLI. Its binary is manta-upload.

pnpm manta-upload login

That login is stored and shared with manta-codegen. For a machine, set MANTA_CLIENT_SECRET in the environment (never as a flag). A service account only gets a datastore today if it holds the operator realm role.

Example

pnpm manta-upload people.csv orders.parquet --datastore your-datastore
pnpm manta-upload people.csv --datastore your-datastore --bundle

--datastore is required. A failed file does not stop the rest of the batch.

Request

Each file is posted to /graphql on the API base URL (default https://app.hiremanta.com) with a bearer token and this header:

x-manta-datastore: your-datastore

The CLI calls initiateUpload with the file name, byte size, and a purpose of FILE (or BUNDLE when you pass --bundle), uploads the parts, then calls completeUpload. The complete selection is fileIri and mediaType.

Response

The CLI prints one line per file to stderr, using the fileIri and mediaType from completeUpload and the byte size it read:

manta-upload: people.csv -> <fileIri> (<mediaType>, <byteSize> bytes)

A file that fails prints manta-upload: people.csv failed: <reason>. When any file failed, the process finishes with manta-upload: <failed> of <total> file(s) failed and exit code 1.

Each version the ingest handles then gets a File Ingest (ingest:IngestRun) node in the datastore:

  • prov:used points to the file
  • ingest:status is ingested, unparsable (the contents could not be read), gone (the file was deleted or replaced before it could be read), or too_large (over the size the knowledge graph accepts), with prov:endedAtTime
  • on success, ingest:recordsRead, ingest:recordsDropped, and prov:generated (the dataset the rows belong to)

A file whose key cannot be written as an IRI gets no node.

How a CSV becomes entities

A CSV or Parquet file needs no schema and no mapping. Manta reads it like this:

  • One entity per row. The file name, without its extension, names the class: reclamations.csv gives Reclamations and unit_reclamations.csv gives UnitReclamations. Name each file after the kind of record it holds, and keep one kind of record per file.
  • One property per column. A column’s header becomes a camelCase property in the datastore’s own upload namespace: unit_id and unit id both give unitId, and an all-caps header is lowercased first (UNIT_ID gives unitId). Capitals inside a header are kept, so Unit ID gives unitID. A repeated header gets a number (unitId2). Spell a header the same way in every file.
  • Types are inferred for a CSV. Manta samples the first 10,000 rows and gives a column the first type that reads at least 95% of its values: integer, double, boolean (true or false), date (2026-10-29), or dateTime (ISO 8601, taken as UTC without an offset). Anything else stays text, and so does a column with a leading zero, such as a postal code. A Parquet column keeps the type the file gives it.
  • Rows that do not fit are dropped. In a CSV, a row with a value its column’s type cannot read is left out whole. An empty row is left out of either kind of file. Both are counted in ingest:recordsDropped. A CSV’s cells are trimmed, and an empty cell gives no value.
  • Each row records its origin. It carries prov:wasDerivedFrom (the uploaded file) and ingest:sourceRow (its line, counting from 1).

The identifier column

A row’s IRI comes from the first column named id, uuid or key (in any case) whose values are all filled in and all distinct. Without one, Manta numbers the rows. The column also stays a property, so read the id from it rather than from the IRI.

Uploading the same file again

Every upload is stored as a new file, and the IRIs of its rows include a token for that upload. Uploading a corrected reclamations.csv therefore adds a second set of entities next to the first one. It does not replace them. A CSV suits a one-off load. For data you send again as it changes, upload Turtle or N-Triples with your own IRIs, as below.

Relationships

A CSV column holds values, not links. A unitId column in reclamations.csv stores the unit’s id as text or a number, and nothing connects the reclamation to the row for that unit in units.csv.

To express a relationship, upload Turtle or N-Triples instead. Name each entity with an IRI in your app’s own namespace, and point at another entity by its IRI:

@prefix acme: <https://acme.example/vocab#> .
@prefix schema: <https://schema.org/> .

<https://acme.example/id/unit/U-12> a acme:Unit ;
  schema:identifier "U-12" .

<https://acme.example/id/reclamation/R-301> a acme:Reclamation ;
  schema:identifier "R-301" ;
  acme:unit <https://acme.example/id/unit/U-12> ;
  acme:reportedOn "2026-10-01"^^<http://www.w3.org/2001/XMLSchema#date> .

An RDF file keeps its IRIs. A later upload replaces every triple of each subject it carries, so sending R-301 again with a new status changes that reclamation instead of adding a second one. A subject left out of the later file keeps what it had. Because a load replaces the whole subject, keep all of a subject’s triples in one file: a later file that describes the same subject replaces what an earlier one said about it. Blank nodes get IRIs tied to the upload that carried them, so give an IRI to anything you will send again. Declare your classes and properties with manta ontology push first.

A custom app allowed to write data can send the same triples as a SPARQL update instead of a file. An update changes only the triples it names, which suits a few records changing at a time. A file suits a full load.

First push for a new datastore

  1. Get the datastore from Manta, and pick it in your custom app’s settings.
  2. Push your app’s vocabulary and shapes to it, if your data uses your own classes: pnpm manta ontology push vocab.ttl --namespace https://acme.example/vocab# --name acme --datastore acme-homes and pnpm manta shapes push shapes.ttl --name acme --datastore acme-homes.
  3. Upload the files: pnpm manta-upload units.ttl reclamations.ttl --datastore acme-homes.
  4. A few minutes later, check each file’s ingest:IngestRun for ingest:status ingested and its ingest:recordsDropped.
  5. Run pnpm manta-codegen generate --datastore acme-homes to get typed schemas for what was loaded.