Skip to content

Architecture

JIM implements an enterprise identity management system using the metaverse pattern. This page describes the layered architecture, the metaverse model, the service topology, and the key design decisions.

System Context

JIM System Context Administrators manage JIM through the web UI and PowerShell, and automation clients call its REST API. An OIDC Identity Provider authenticates sign-in. JIM synchronises with Directory Services over LDAP and LDAPS, imports from HR systems via CSV, reads and writes file systems, reads and writes enterprise databases over SQL, and synchronises with cloud applications over SCIM 2.0. Administrator configures & operates >_ Automation Client CI/CD · API keys Identity Provider OIDC single sign-on JIM Junctional Identity Manager self-hosted identity synchronisation Directory Services Active Directory · LDAP HR Systems authoritative source File Systems CSV & flat files Enterprise Databases SQL Server · Oracle Cloud Applications SCIM 2.0 web UI · PowerShell REST API OIDC LDAP/LDAPS imports · CSV reads & writes SQL SCIM 2.0

Administrators and automation clients work through JIM's UI and API; JIM synchronises with the surrounding systems. Moving dots trace identity data in flight.

Layered Architecture

JIM follows a strict N-tier layered architecture. Upper layers depend on lower layers, never the reverse.

Layer Project Responsibility
Presentation JIM.Web Blazor Server UI with integrated REST API at /api/
Application JIM.Application Business logic, domain servers, JimApplication facade
Domain JIM.Models Entities, DTOs, interfaces
Data JIM.Data / JIM.PostgresData Data access abstractions and PostgreSQL implementation
Integration JIM.Connectors External system connectors

Rules:

  • Respect layer boundaries: the UI/API layer must only call JimApplication, never repository classes directly
  • The application layer depends on IRepository, not concrete implementations
  • All models and POCOs live in JIM.Models, never inline in service files

Container Diagram

JIM Containers JIM deploys as a set of containers. Administrators use the Web Application's Blazor UI, or the PowerShell Module which wraps the REST API; automation clients call the REST API directly; an OIDC Identity Provider authenticates sign-in. The Web Application, Scheduler Service and Worker Service all use the PostgreSQL database, which also acts as the task queue: the Scheduler queues tasks and the Worker polls them. The Worker Service runs the synchronisation pipeline and hosts the Connectors, connecting to Directory Services, HR systems, file systems, enterprise databases and cloud applications. Administrator web UI & PowerShell >_ Automation Client REST API · API keys Identity Provider OIDC single sign-on PowerShell Module automation via the REST API Web Application Blazor admin UI + REST API Scheduler Service cron & interval triggers Worker Service runs the sync pipeline hosts the Connectors PostgreSQL configuration & identity data Directory Services LDAP · LDAPS HR Systems CSV File Systems CSV & flat files Enterprise Databases SQL Server · Oracle Cloud Applications SCIM 2.0 web UI REST API OIDC REST API reads & writes queues tasks task queue JIM

JIM's deployable containers. PostgreSQL doubles as the task queue: the Scheduler queues work and the Worker polls it, so the services coordinate through the database rather than calling each other. Moving dots trace identity data in flight.

Metaverse Pattern

The metaverse is the authoritative identity repository at the centre of JIM's architecture. All identity operations flow through the metaverse; there is never a direct sync between Connected Systems.

  • MetaverseObject
    Central identity entity (users, groups, custom types)
  • ConnectedSystem
    External system synchronised with the metaverse
  • SyncRule
    Bidirectional mappings between Connected Systems and the metaverse
  • Staging Areas
    Import/export staging for transactional integrity

The metaverse pattern Source systems on the left import identity data into the Metaverse, the single source of truth at the centre. Target systems on the right receive exports from it. Data always flows through the Metaverse; it never moves directly between systems. Source Systems HR · directories · files Metaverse single source of truth Target Systems directories · applications import export

Sources project identities into the Metaverse; targets receive them from it. The same Connected System can be both source and target (writeback). Moving dots trace import and export flows.

Component Diagrams

The component views use the docs site's hand-authored diagram system and are simplified for legibility (the Worker view also appears on the customer architecture page); the process diagrams below carry the exact runtime flows.

Application Layer

JIM.Application exposes a single entry point, the JimApplication facade, which delegates to the domain servers. The Worker's processors bypass the facade and use SyncServer and SyncEngine directly for performance-critical synchronisation work.

Inside the Application Layer JIM.Web and JIM.Scheduler call the JimApplication facade, the single entry point to the domain servers: Metaverse, ConnectedSystem, ObjectMatching, Search, Security, Auth, Scheduler, Tasking, Activity, ChangeHistory, Certificate, ServiceSettings, FileSystem, ExampleData and System, plus the configuration services. The JIM.Worker task processors bypass the facade and use SyncServer, which orchestrates the sync evaluation and execution servers (ExportEvaluation, ExportExecution, ScopingEvaluation, ScopeReconciliation and DriftDetection), and SyncEngine, the stateless sync domain logic with no I/O. Domain servers reach PostgreSQL through IJimRepository; the sync servers use the dedicated ISyncRepository. JIM.Web Blazor UI + REST API JIM.Scheduler schedule triggers JIM.Worker task processors JimApplication facade single entry point SyncServer sync orchestration SyncEngine stateless · no I/O Domain servers Metaverse · ConnectedSystem · ObjectMatching · Search Security · Auth · Scheduler · Tasking · Activity ChangeHistory · Certificate · ServiceSettings · FileSystem ExampleData · System · configuration services Sync evaluation & execution ExportEvaluation · ExportExecution ScopingEvaluation · ScopeReconciliation DriftDetection IJimRepository general data access ISyncRepository bulk sync data access PostgreSQL single data store JIM.Application

The facade fronts the domain servers for the Web and Scheduler services; the Worker's task processors use SyncServer and the stateless SyncEngine directly. Domain servers reach PostgreSQL through IJimRepository, the sync servers through the dedicated ISyncRepository (SyncServer also uses it directly; omitted for legibility). Moving dots trace calls and data in flight.

Web Application

Inside the Web Application Administrators use the Blazor admin pages and automation clients call the REST API controllers; every request enters through the authentication middleware, which validates OIDC sign-in against the Identity Provider and API keys. Both the pages and the controllers call the JimApplication facade, the Application Layer's single entry point. Administrator web UI >_ Automation Client REST API · PowerShell Identity Provider OIDC single sign-on Authentication middleware OIDC sign-in · API key validation Blazor pages admin UI API controllers REST at /api/ JimApplication facade Application Layer entry point OIDC validates requests JIM.Web

Every request enters through the authentication middleware, which validates OIDC sign-in against the Identity Provider and API keys, before reaching the Blazor pages or API controllers; both call the JimApplication facade. Moving dots trace requests in flight.

Worker Service

Inside the Worker Service The Worker Host polls the PostgreSQL task queue and dispatches processors for import, synchronisation and export; the service reads and writes staged and Metaverse data in the same database. Import processors stage data arriving through the Connectors; synchronisation processors use the Sync Engine to apply projection, attribute flow, and deletion and matching rules; the export processor sends outbound changes back through the Connectors to Connected Systems. The Connectors layer hosts the LDAP, File/CSV, SQL and SCIM 2.0 connectors. PostgreSQL task queue staged & metaverse data Worker Service Worker Host polls the task queue · dispatches processors Import full & delta imports Synchronise full & delta synchronisation Export pending export execution Sync Engine projection · attribute flow deletion & matching rules Connectors LDAP File / CSV SQL SCIM 2.0 Connected Systems dispatches uses staged imports outbound changes polls task queue

Inside the Worker Service: the host dispatches import, synchronise and export processors, and the Sync Engine makes the synchronisation decisions. The host polls the task queue in PostgreSQL, where the whole service reads and writes staged and Metaverse data; the Connectors carry data to and from Connected Systems. Moving dots trace data arriving through, and leaving via, the Connectors.

Connectors

Inside the Connectors library The JIM.Worker import and export processors invoke the connectors in the JIM.Connectors library. The LDAP Connector reads and writes Directory Services over LDAP and LDAPS; the File Connector reads and writes CSV and flat files, including HR exports; the SCIM 2.0 Connector reads and writes cloud applications that publish a SCIM 2.0 service provider interface; the SQL Connector reads and writes enterprise databases, Microsoft SQL Server and Oracle Database, through managed drivers. JIM.Worker import & export processors LDAP Connector LDAP · LDAPS File Connector CSV & flat files SQL Connector SQL Server · Oracle SCIM 2.0 Connector SCIM 2.0 · HTTPS Directory Services LDAP · LDAPS File Systems CSV · HR exports Enterprise Databases SQL Server · Oracle Cloud Applications SCIM 2.0 service providers JIM.Connectors

The Worker's import and export processors invoke the connectors, which carry data to and from the external systems. Moving dots trace data in flight.

Scheduler Service

Inside the Scheduler Service The Scheduler Host runs a 30-second polling loop that evaluates schedule due times and performs crash recovery. It queues tasks through the JimApplication facade, which stores schedules and the task queue in PostgreSQL for the Worker to pick up. JIM.Scheduler Scheduler Host 30-second polling loop due times · crash recovery JimApplication facade queues Worker tasks PostgreSQL schedules & task queue queues tasks

The Scheduler Host's polling loop evaluates schedule due times and performs crash recovery, queueing tasks through the facade into PostgreSQL for the Worker to pick up. Moving dots trace tasks being queued.

Technology Stack

Category Technology
Runtime .NET 10.0, C# 14
Database PostgreSQL 18 via Npgsql and EF Core 10.0
Web UI Blazor Server with MudBlazor 9.x
Authentication OpenID Connect (OIDC) with PKCE
Logging Serilog (structured logging)
Containers Docker and Docker Compose
CI/CD GitHub Actions
Testing NUnit, Moq, coverlet

Project Structure

src/
  JIM.Application/       -- Business logic, domain servers
  JIM.Connectors/        -- External system connectors
  JIM.Data/              -- Data access abstractions (interfaces)
  JIM.InMemoryData/      -- In-memory data layer (for testing)
  JIM.Models/            -- Domain models, DTOs, interfaces
  JIM.PostgresData/      -- PostgreSQL EF Core implementation
  JIM.Scheduler/         -- Schedule management service
  JIM.Utilities/         -- Shared utilities
  JIM.Web/               -- Blazor Server UI + REST API
  JIM.Worker/            -- Background task processor

test/
  JIM.Models.Tests/      -- Model and DTO tests
  JIM.Web.Api.Tests/     -- API controller tests
  JIM.Worker.Tests/      -- Worker and sync processor tests
  JIM.Workflow.Tests/    -- Multi-step workflow tests

Service Architecture

JIM runs as a set of Docker services:

Service Description
jim.web Blazor Server UI with integrated REST API at /api/. Listens on port 80 in-container; reached at http://localhost:5200 in the development Docker stack (HTTPS is terminated by a reverse proxy in production). Interactive Scalar API reference available at /api/reference in every environment, backed by a build-time OpenAPI document for instant loading.
jim.worker Background task processor. Polls the task queue, processes sync/import/export operations. Uses ISyncEngine/ISyncRepository separation for testability.
jim.scheduler Schedule management. Reacts to task completion instantly via PostgreSQL notifications, with a 30-second polling cycle as the fallback. Detects parallel step groups and queues them for concurrent worker dispatch.
jim.database PostgreSQL 18 database.
jim.keycloak Bundled Keycloak IdP for development SSO (port 8181). Not included in production deployments.

Real-Time Notifications

The services coordinate through the database, and the database also tells them when something changed: triggers on the Worker Task and Activity tables publish PostgreSQL NOTIFY events (on commit) that services receive over a dedicated LISTEN connection. The Scheduler uses them to react to task completion in under a second, and the web application pushes them to the browser, so the Operations page updates without frequent polling. A SignalR hub at /hubs/notifications broadcasts the same events for non-Blazor consumers.

Notifications are fire-and-forget hints carrying identifiers only; the database remains the source of truth. Every consumer keeps a polling fallback (the Scheduler's 30-second cycle, the UI's background refresh), so a dropped notification degrades latency, never correctness. No additional infrastructure is required, keeping JIM self-contained and air-gap deployable.

Worker Architecture

The Worker is the engine that processes all synchronisation operations. Its design separates pure domain logic from I/O for testability and performance.

Core Interfaces

  • ISyncEngine
    Stateless domain engine with methods for join resolution, projection, Attribute Flow, scoping, and more. Zero I/O dependencies; receives all data as parameters and returns results. Fully unit-testable without mocks.
  • ISyncRepository
    Data access boundary with approximately 80 methods. Production implementation: JIM.PostgresData.Repositories.SyncRepository. Test implementation: JIM.InMemoryData.SyncRepository.

Dependency Injection

The Worker and Scheduler use IJimApplicationFactory and IConnectorFactory for per-task context isolation. Each dispatched task gets its own DI scope with independent DbContext and connector instances.

Bulk Write Performance

  • ParallelBatchWriter
    Splits bulk writes across N concurrent PostgreSQL connections
  • COPY binary protocol
    Used for high-volume inserts (CSO creates, MVO creates, RPEIs, sync outcomes) via Npgsql's binary COPY API

Export Parallelism

Export parallelism operates on two independent axes:

  1. LDAP Connector Pipelining: Multiple LDAP operations execute concurrently within a single export batch using SemaphoreSlim-based throttling, tuned per directory type by Export Concurrency auto-tune (see LdapConnector.AutoTuneExportConcurrency)
  2. Parallel Batch Processing: Multiple export batches process concurrently with separate IRepository and IConnector instances per batch, gated by the SupportsParallelExport connector capability, and controlled by a Connected System's Max Export Parallelism setting

The two axes multiply: each parallel batch pipeline gets its own connector instance running its own Export Concurrency operations, so total in-flight operations = Max Export Parallelism x Export Concurrency.

A Connected System's Max Export Parallelism resolves via ExportParallelismResolver (JIM.Worker.Processors) in this order:

  1. An explicit Max Export Parallelism value on the Connected System always wins, respecting the administrator's choice.
  2. Otherwise, if the connector implements IConnectorRecommendedExportParallelism, its recommendation is used, clamped to 1-16. The LDAP Connector recommends a deliberately conservative 2 when the system's Export Concurrency signals a capable directory (8 or above; the auto-tune only sets 16, for Active Directory and OpenLDAP), keeping the multiplied total mild (2 x 16 = 32 in-flight operations); otherwise it makes no recommendation.
  3. Otherwise, JIM falls back to 1 (sequential), the pre-existing default.

Note that for directories with a single-writer storage backend (for example OpenLDAP's mdb), higher batch parallelism does not translate into proportionally higher write throughput; the recommendation is a starting point, not a guarantee.

Process Diagrams

Detailed Mermaid diagrams document the runtime behaviour of JIM's synchronisation engine, worker, and scheduler. These are viewable directly in GitHub, VS Code, or any Mermaid-compatible markdown renderer.

Synchronisation

  • Full Sync CSO Processing: Core per-CSO decision tree (scoping, join, projection, Attribute Flow, drift detection)
  • Delta Sync Flow: How delta sync differs from full sync (watermark, early exit, CSO selection)
  • Full Import Flow: Object import, duplicate detection, deletion detection, Pending Export reconciliation

Export

Worker and Scheduling

Supporting Concepts