Skip to content

JIM File Connector

Overview

The JIM File Connector enables bi-directional synchronisation of identity data with CSV and delimited text files. It is ideal for integrating with systems that produce flat-file extracts, such as HR exports, batch feeds, or data migration files.

Capabilities: Full Import, Export

Supported Features

  • Configurable delimiters
    Comma (default), tab, pipe, semicolon, or any custom character
  • Multi-valued attributes
    Supported via duplicate column names in the header row, or via a configurable in-field delimiter (default: pipe |)
  • Schema auto-discovery
    Column headers are read automatically to build the schema
  • Type detection
    Attribute data types are inferred by inspecting up to 50 rows of data. Supported types: Text, Number, Long Number, Decimal, Boolean, Guid, DateTime
  • Culture-aware parsing
    An optional culture setting controls how numbers, dates, and other locale-sensitive values are parsed
  • Object type support
    Objects can be typed via a dedicated column in the file, or by specifying a fixed object type for the entire file
  • Operational modes
    Import Only, Export Only, or Bidirectional (export then confirming import from the same file)
  • Auto-confirm export
    An export counts as confirmed as soon as the file is written, and JIM records the values it wrote, so it knows what the file holds without reading it back (see Export Only mode)
  • Stop on first error
    Optionally halt processing on the first error encountered, useful for debugging data quality issues

Settings

Setting Description Default Example
File Path Path to the CSV file inside the container. Used for import, export, or both depending on mode. (required) /connector-files/Users.csv
Mode Operational mode: Import Only, Export Only, or Bidirectional. Import Only Bidirectional
Object Type Column Column in the file that contains the object type. Use when the file contains multiple object types. (one of these two is required) Type
Object Type Fixed object type name. Use when the file contains a single type of object. (one of these two is required) User
Delimiter Field delimiter character. , \t (tab)
Multi-Value Delimiter Character used to separate multiple values within a single field. \| ;
Culture Culture code for parsing locale-sensitive values. Uses invariant culture if not specified. (invariant) en-gb
Stop On First Error Stop processing when the first error is encountered. Useful for debugging. false true

Object type configuration

You must specify exactly one of Object Type Column or Object Type; JIM rejects the settings if you supply neither or both. If the file contains multiple object types in different rows, use Object Type Column to point to the column that identifies each row's type. If every row represents the same type, use Object Type to set a fixed value.

File access

JIM ships with a named volume (jim-connector-files-volume) mounted inside both the JIM Web and JIM Worker containers at /connector-files, on Docker and on Podman. The File Connector reads from and writes to paths under this directory. You will configure all File Connector File Path settings as /connector-files/<some-path>.

There are two patterns for getting files into and out of /connector-files. Most deployments use both at the same time, depending on the integration.

1. Default: write to the named volume

This is the simplest pattern and the recommended starting point. The volume is created and managed by Docker; ownership is set automatically so the JIM container's runtime user can read and write to it without any host-side permission tweaking.

You don't need to configure anything in your docker-compose.yml; the volume is part of the bundled JIM compose files. To put a file into the volume, stream it through the worker container so the resulting file is owned by the JIM runtime user:

docker exec -i -u app jim.worker sh -c 'cat > /connector-files/Users.csv' < ./Users.csv

The -u app flag is important: it runs the shell inside the container as the JIM runtime user (UID 1654), so the file lands with the correct ownership. This matters when JIM will later rewrite the same file, for example in Export Only or Bidirectional mode, or whenever schema discovery is triggered on an existing file.

On Podman, the worker container is jim-worker, and its default user is already app (for a rootless installation, see Rootless commands):

sudo podman exec -i jim-worker sh -c 'cat > /connector-files/Users.csv' < ./Users.csv
sudo podman cp jim-worker:/connector-files/Exports.csv - | tar -xf -   # pull an export out

Don't use docker cp to push files JIM will rewrite

docker cp ./file jim.worker:/path is tempting but preserves your host UID/GID. The resulting file is not writable by the JIM runtime user (UID 1654), so any subsequent JIM export against that file will fail with an "Access to the path … is denied" error. Use the docker exec … cat > form above instead.

Reading out of the volume with docker cp is fine; the file on the host takes your local UID, which is what you usually want:

docker cp jim.worker:/connector-files/Exports.csv ./Exports.csv

When to use this pattern:

  • Exports
    JIM writes the file; a downstream process or human pulls it out via docker cp, an SFTP sidecar, or similar.
  • Manual imports
    Admin pushes a file in once, JIM imports it.
  • Sidecar-orchestrated drops
    A helper service (cron job, file-transfer container) runs alongside JIM, drops a file into the volume, and JIM picks it up on the next scheduled import.

2. Integration: bind-mount over a subdirectory

Use this pattern when an external system writes files to a fixed location you cannot change, typically a network share (SMB/CIFS or NFS) the customer's HR system writes to nightly.

You bind-mount the host path over a subdirectory of /connector-files. JIM still sees a unified /connector-files filesystem; only the specific subdirectory's contents come from the host.

# docker-compose.override.yml or your production overlay
services:
  jim.worker:
    volumes:
      # Mount your network share (already mounted on the host) at a subdirectory
      - /mnt/hr-extracts:/connector-files/hr-input
  jim.web:
    volumes:
      - /mnt/hr-extracts:/connector-files/hr-input

Then configure the File Connector with File Path = /connector-files/hr-input/employees.csv.

Permissions on bind-mounted paths

The JIM container runs as a non-root user (UID 1654, named app). The default named volume is owned by this user, so writes work out of the box. Bind-mounted host paths preserve host UID/GID, so if your HR system writes files owned by a different UID, the JIM worker may not have read access (for imports) or write access (for exports). Either:

  • Make the host directory readable/writable by UID 1654 (chown 1654:1654 /mnt/hr-extracts), or
  • Adjust your network share's mount options (uid=1654,gid=1654 for CIFS, or apply NFS UID-mapping) so files appear to be owned by UID 1654 inside the container.

Permission errors during sync show up as RPEIs on the failing import or export Activity, with a clear "Access to the path … is denied" message.

On Podman

On Podman, add the host folder to the pod file, /opt/jim/jim.yaml: a hostPath volume, mounted in the worker container (and the web container, for the file browser):

      volumeMounts:
        - { name: hr-extracts, mountPath: /connector-files/hr-input }
  # ...
  volumes:
    - name: hr-extracts
      hostPath: { path: /mnt/hr-extracts, type: Directory }

Then restart JIM: sudo systemctl restart jim.service, or for a rootless installation, jim-systemctl restart jim.service (see Rootless commands). An upgrade replaces jim.yaml, so carry the addition into each new release's copy.

Two things can differ from Docker:

  • Ownership, when rootless
    A rootful JIM, the default, needs the folder owned by UID 1654, as on Docker. A rootless JIM runs as the jim account, and its container user, UID 1654, is a subordinate ID of that account on the host rather than UID 1654. Hand the folder to the container user from within the account's own user namespace, which picks the right host ID:

    chown jim: /mnt/hr-extracts
jim-podman unshare chown 1654:1654 /mnt/hr-extracts

    On the host, the folder then belongs to the account's first subordinate ID plus 1653 (for example 101653 when /etc/subuid gives jim the range from 100000). For a network share, mount it with that ID as the owner (uid=101653,gid=101653 for CIFS).

  • SELinux
    On a host with SELinux enforcing, as RHEL is by default, a container may read and write only files labelled for containers. Label the folder, persistently:

    semanage fcontext -a -t container_file_t '/mnt/hr-extracts(/.*)?'
restorecon -R /mnt/hr-extracts

    For an NFS or CIFS mount, set the label as a mount option instead: context="system_u:object_r:container_file_t:s0".

Where a network share cannot be mounted on the host with a suitable owner or label, a rootful installation can mount it as a Podman volume instead, which Podman mounts itself: sudo podman volume create --driver local --opt type=cifs --opt device=//fileserver/hr-extracts --opt o=username=jim-svc,uid=1654,gid=1654 hr-extracts, then use persistentVolumeClaim: { claimName: hr-extracts } in place of hostPath. Mounting a share needs root, so this does not work rootless.

Mounting both at once

The patterns combine naturally. A typical deployment looks like this:

services:
  jim.worker:
    volumes:
      # Default volume is already wired up; JIM uses it for exports and ad-hoc imports.
      # No line needed here; comes from the base docker-compose.yml.

      # Pull HR extracts directly from the network share they're written to.
      - /mnt/hr-extracts:/connector-files/hr-input

      # Push payroll exports to a different network share that the payroll team picks up.
      - /mnt/payroll-drop:/connector-files/payroll-output

JIM's File Connector configurations would then point at:

  • /connector-files/Users.csv
    Admin-pushed file in the named volume
  • /connector-files/hr-input/employees.csv
    Read from the HR share
  • /connector-files/payroll-output/payroll.csv
    Written to the payroll share

Schema Discovery

When you configure a File Connector Connected System and trigger schema discovery, JIM performs the following steps:

  1. Reads the header row
    Column names become attribute names in the schema.
  2. Detects multi-valued attributes
    If the same column name appears more than once in the header, the attribute is marked as multi-valued.
  3. Infers data types
    JIM reads up to 50 data rows and attempts to parse each column's values as Number, Long Number, Decimal, Boolean, Guid, or DateTime, in that order (narrowest numeric type first). Whole numbers within 32-bit range infer Number, larger whole numbers infer Long Number, and fractional or exponent-notation values (for example 1.5 or 1.5E3) infer Decimal. If none of these match, the attribute defaults to Text.
  4. Discovers object types
    If an Object Type Column is configured, JIM reads through the file to find all unique object type values. Otherwise, it uses the fixed Object Type setting.

In Export Only mode where no file exists yet, schema discovery creates a minimal schema with just the specified object type. Attributes are defined later by Synchronisation Rules.

Export Cell Formats

When JIM writes the export file, each attribute value is rendered as text in its cell:

  • Text, Number, Long Number, Guid
    Written as-is, using invariant culture.
  • Decimal
    Canonical invariant form: plain notation, no trailing zeros, never exponent notation.
  • DateTime
    ISO 8601 round-trip format (for example 2025-06-15T10:30:00.0000000Z).
  • Boolean
    Lowercase true or false.
  • Reference
    The referenced object's External ID value (for example the manager's employee ID). This is the same string form the import side parses, so a reference exported by JIM resolves back to the same object if the file is re-imported.
  • Binary
    Base64-encoded, as a CSV cell cannot carry raw bytes.

Binary is export-only

The File Connector cannot import Binary attribute values: schema discovery never infers a Binary type, and a schema-declared Binary attribute fails to parse on import. Binary cells written by an export are therefore one-way; do not use Bidirectional mode for a schema that includes Binary attributes.

Export Only mode

In Export Only mode JIM writes the file and never reads it back. It knows what each row holds because every export records the values it wrote against the row's Connected System Object. That is what lets JIM clear a value in the file when it is cleared in the Metaverse: it compares the new value with what it last wrote.

Three consequences are worth knowing:

  • Changes made to the file outside JIM go unseen. JIM compares against what it wrote, not against the file. If another process edits the file, use Bidirectional mode, so a confirming import reads the file after each export.
  • Exported objects stay Pending Provisioning in the Connector Space. Only an import moves an object to Normal, and Export Only never imports. This does not hold anything up: changes and deletions are exported as usual, and an object JIM deletes from the file is removed from the Connector Space at the same time.
  • Nothing is lost if JIM cannot record what it wrote. The export is left unconfirmed, the next export run writes the same rows again (which changes nothing in the file) and records them, and the export's Activity finishes with a warning saying so.

Bringing JIM up to date with an existing file

JIM records values only when it writes them. Point a new Export Only Connected System at a file that already has rows, or upgrade from a JIM release that did not record exported values, and JIM does not know what those rows hold: a value cleared in the Metaverse is not cleared from them until the value has changed at least once since. To have JIM learn the whole file at once:

  1. Set the Connected System's Mode to Bidirectional.
  2. Run a Full Import for it, then a Full Synchronisation, creating those Run Profiles if it has none.
  3. Set Mode back to Export Only.

The import reads every row into the Connector Space, and moves the objects it finds to Normal. Rows JIM did not write, and objects JIM expected but the file does not have, are reported by that import like any other.

Troubleshooting

File not found

The error File not found: '/connector-files/Users.csv' indicates the file is not accessible at the specified path inside the container.

  • For named-volume files: confirm the file is in the volume: docker exec jim.worker ls /connector-files.
  • For bind-mounted paths: verify the host path exists and is mounted in docker-compose.yml (docker compose config | grep connector-files).
  • Paths are case-sensitive on Linux. Make sure the File Path in JIM matches the actual filename exactly.

Access denied (permission error)

The error Access to the path '/connector-files/...' is denied means the JIM runtime user (UID 1654) does not have write permission on the target file or its parent directory. This happens in two common situations:

Bind-mounted host directories: host files preserve host UID/GID, which may not match UID 1654.

  • Fix the ownership: chown -R 1654:1654 /mnt/your-mount-point on the host.
  • For CIFS/SMB mounts, add uid=1654,gid=1654 to the mount options.
  • For NFS, use UID mapping or ensure the file owner UID matches.

Files pushed into the named volume with docker cp: docker cp preserves your host UID, so the file is owned by you, not by UID 1654. A subsequent JIM export or schema refresh against that file will fail. Either re-push the file with the docker exec … cat > form shown under File access, or fix the ownership directly:

docker exec -u 0 jim.worker chown app:app /connector-files/Users.csv

Column count mismatch

The error Row X has Y columns but the header defines Z columns means a data row has fewer or more fields than the header row.

  • Open the file and inspect the row number mentioned in the error.
  • Check for missing delimiters, extra delimiters, or unquoted fields that contain the delimiter character.
  • Fields containing the delimiter character should be enclosed in double quotes.

Object type column not found

The error Object type column 'Type' not found in file headers means the column name specified in Object Type Column does not match any column in the file.

  • Check the spelling and case of the column name -- the match is case-insensitive, but the column must exist in the header row.
  • If the file does not have an object type column, use the Object Type setting instead.

Encoding issues

If imported data contains garbled characters or unexpected symbols:

  • Set the Culture setting to match the file's locale (e.g. en-gb, de-de).
  • Ensure the file is saved in UTF-8 encoding.
  • If the file was exported from another system, check whether that system uses a different character encoding.

Multi-valued attribute not detected

If attributes that should be multi-valued are appearing as single-valued:

  • Duplicate column approach
    Ensure the column name appears more than once in the header row. The names must match exactly (case-insensitive).
  • In-field delimiter approach
    Check that the Multi-Value Delimiter setting matches the delimiter used within the field values.