GAMP 5 Category 4 for probabilistic decision engines — why "configurable" is the right classification

1. Executive summary

GAMP 5 — Good Automated Manufacturing Practice version 5, published by ISPE — is the de-facto industry standard for computer system validation (CSV) in regulated pharmaceutical environments. Its category framework (1 to 5) drives the depth and breadth of validation activity. Probabilistic decision engines for pharma operations are a new class of software that sits awkwardly in the existing categories: they are neither simple parameterised tools (Cat 3) nor fully bespoke developments (Cat 5).

This paper argues that Category 4 — configurable software — is the correct classification for engines like Synlogica's GRIP, where the deterministic rule layer (SOPs, GxP gates) and the probabilistic model parameters (Arrhenius coefficients, Bayesian priors, Monte Carlo distributions) are tenant-configurable, but the underlying engine — the probabilistic mathematics, the rule execution graph, the decision-package format — is qualified once by the vendor and used unchanged by every customer.

The classification matters because it scopes the validation effort. A Cat 5 (custom) classification would require the customer to validate the engine internals against their specific requirements (typically 200–400 person-days of effort). A correct Cat 4 classification confines customer-side validation to the configuration layer (typically 40–80 person-days) and accepts the vendor's qualification of the engine itself.

2. GAMP 5 categories — a 90-second refresher

GAMP 5 Second Edition (2022) describes five categories of computerised systems, each with a recommended validation approach:

(Category 2 was removed in the Second Edition; the numbering 1, 3, 4, 5 is retained for backward reference.) The framework is risk-based: the validation effort should match the system's GxP impact, complexity and novelty.

Category 4 deserves emphasis because it is the most operationally important: it covers the systems that pharma teams actually use day-to-day (ERP, LIMS, MES) and now increasingly the decisioning systems that sit on top of them.

3. The classification challenge for probabilistic engines

Probabilistic decision engines confuse the standard Cat 3 vs Cat 4 vs Cat 5 conversation in three ways:

3.1 The engine is the same; the parameters are not

Every customer of Synlogica Terminus uses the same GRIP engine version — the Monte Carlo sampler, the Bayesian update logic, the Arrhenius integrator are bit-identical across tenants. What differs is the configuration: per-tenant Arrhenius parameters for API portfolio, per-tenant Bayesian priors for supplier elasticity, per-tenant Monte Carlo input distributions for lane risk. This is the textbook Cat 4 pattern.

3.2 Probabilistic outputs are not bit-identical across runs (by design)

A Monte Carlo simulation seeded with a random initial state produces statistically equivalent but not bit-identical outputs run-to-run. Inspectors trained on deterministic software sometimes flag this as non-reproducibility, which would be a Part 11 concern. The correct response: the engine seeds the simulation deterministically per decision (using e.g. SHA-256 of input data lineage), so any decision can be replayed bit-for-bit even though two unrelated runs would differ.

3.3 The decision policy layer is risk-tolerance configuration, not custom code

"Recommend release if probability of exceeding threshold is below 5% AND batch is non-sterile AND product class is class B" looks like custom logic but is configuration of a generic policy DSL. As long as the DSL itself is qualified and the configuration is validated like any Cat 4 configuration, this is not Cat 5.

The risk in mis-classification is symmetric: under-classify (e.g. Cat 3) and the inspector asks for missing validation evidence; over-classify (Cat 5) and the team spends person-years validating what the vendor already qualified.

4. Why Category 4 (configurable) is the right fit

The Category 4 fit rests on three observations:

4.1 The product exists independently of any single customer

GRIP is developed and qualified by Synlogica as a product. Every customer receives the same engine binary, the same algorithm implementations, the same decision-package format. This is the definitional test for Category 4 versus Category 5: a Cat 5 system is built for one customer; a Cat 4 product is built once and used by many.

4.2 The configuration surface is structured, not arbitrary

Customers configure GRIP through a defined surface: rule sets (declared in a structured DSL), model parameters (typed inputs with validation rules), policy configurations (selected from a documented set of policy templates with risk-tolerance parameters), data integration endpoints (mapped through a connector layer). They do not write code into GRIP. They do not modify the engine internals.

4.3 Vendor qualification packages exist and are auditable

The vendor (Synlogica) maintains GAMP 5 Category 4 qualification documentation for GRIP — design qualification, code review records, unit and integration test reports, qualification protocols, release notes per version. These are made available to customers under DPA as part of the trust pack. The customer's validation effort builds on this evidence rather than duplicating it.

The combination — product-class software, structured configuration surface, vendor qualification evidence — is the exact pattern Category 4 was defined for.

5. What changes vs Category 5 (custom)

The practical differences between Cat 4 and Cat 5 effort levels are substantial:

The Cat 4 approach is not "less rigorous"; it is "rigour partitioned between vendor and customer". The customer's rigour is concentrated on configuration validation (which the vendor cannot do for them), and the vendor's rigour is concentrated on engine qualification (which would be wasteful to duplicate per customer).

6. Validation deliverables a customer should expect

From the vendor (as part of the trust pack):

• Supplier audit dossier — including ISO 9001 / ISO 27001 status, development lifecycle documentation, code review and change control records.
• GRIP qualification protocol and report — design qualification, unit test coverage, integration test coverage, performance qualification.
• Release notes per version — what changed in the engine, what changed in the rule DSL, what changed in the policy DSL, what changed in the decision-package format. Versioned by semver.
• Traceability matrix — requirement → design → code → test, maintained for every release.
• Risk assessment for probabilistic outputs — covering the seeded-determinism mechanism and the convergence guarantees for Monte Carlo / Bayesian updates.

From the customer (assembled with vendor support):

• User Requirements Specification (URS) — scoped to intended use: what business decisions Terminus will be used to make.
• Functional risk assessment — GxP impact per decision class, controls per impact level.
• Configuration specification — the customer's specific rules, parameters and policies, with rationale.
• IQ / OQ scripts — focused on the configuration layer and the integration to existing systems (eQMS, ERP, TMS).
• PQ (Performance Qualification) — end-to-end scenarios that confirm the configured system supports the intended business decisions.
• Validation summary report — closing the validation lifecycle and authorising production use.

7. Validating probabilistic logic — the specific challenges

7.1 "Same input, same output" — the reproducibility test

Probabilistic systems can be validated for bit-for-bit reproducibility by seeding the random sampler deterministically. The test: a decision package replayed against the same engine version with the same inputs must produce the same output. GRIP achieves this through SHA-256-derived seeding. The validation evidence is a reproducibility protocol that replays N production decisions and confirms identical outputs.

7.2 "What happens when inputs degrade?" — the robustness test

Probabilistic engines need explicit tests for what happens when input data is missing, noisy or out of expected ranges. The expected behaviour is graceful refusal to recommend rather than recommendation based on assumed inputs. GAMP 5 OQ scripts for probabilistic systems should include negative tests covering data unavailability, sensor anomaly, and parameter drift.

7.3 "How do we know the model is right?" — the calibration test

Distinguished from validation in the regulatory sense. Calibration is about whether the model's predictions match reality (e.g., do shipments classified as 95% likely to arrive on time actually arrive 95% of the time). This is an ongoing performance-monitoring concern, separately from validation. The validation question is "does the system implement the intended model correctly"; the calibration question is "is the intended model itself accurate for our operations".

7.4 "What changes when the engine version updates?" — the impact test

When the vendor releases a new engine version, the customer's validation team needs an impact assessment: did the algorithmic behaviour change, did the rule DSL semantics change, did the decision-package format change? GRIP versioning follows semver: patch versions are bit-equivalent for valid inputs, minor versions extend without breaking, major versions may change semantics and trigger customer re-validation of affected configurations.

8. Ongoing controls and change management

Cat 4 systems require ongoing validation discipline:

• Change control — every configuration change (new rule, modified parameter, updated policy) goes through change control with risk assessment, test plan, approval and post-change verification.
• Periodic review — typically annual; confirms configuration is still fit for purpose, the user list is current, integrations remain valid.
• Incident management — any decision-package anomaly is logged, investigated, root-caused. Anomalies that trace to configuration are customer-side; anomalies that trace to the engine are vendor-side and trigger a support escalation.
• Vendor change notification — the vendor notifies of every release. Customer's validation team assesses impact and decides on re-validation scope.
• Continuous performance monitoring — calibration metrics (predicted vs actual) are reported periodically; significant drift triggers re-tuning of the configuration.

The biggest mistake in Cat 4 ongoing validation is treating it as a one-time event. The system is qualified at go-live; ongoing controls keep it qualified across change. Validation maturity is measured by the discipline of those ongoing controls, not by the depth of the initial qualification.

9. Takeaways

1. GAMP 5 Category 4 (configurable) is the correct classification for probabilistic decision engines like GRIP where the engine is qualified by the vendor and customers configure rules, parameters and policies.
2. Cat 5 (custom) over-classification adds 3–5× validation effort without adding compliance value for product-class software with structured configuration surfaces.
3. Vendor-side qualification documentation (supplier audit, qualification protocol, release notes, traceability) must be available under DPA and trusted as evidence by customer-side validators.
4. Probabilistic logic requires specific validation patterns: seeded determinism for reproducibility, negative tests for input degradation, version-controlled engine releases with documented semver semantics.
5. Calibration (model accuracy in production) is separate from validation (correct implementation); both matter but require different controls.
6. Ongoing controls — change control, periodic review, incident management, vendor change notification — are where Cat 4 maturity is measured.