Define custom tests to validate clinical trial infrastructure for FDA compliance
Contract Research Organizations (CROs) spend months validating clinical trial infrastructure. OpenFactory enables CROs to define executable tests that validate:
Instead of manually validating each system, define tests once and run them automatically on every build. This reduces validation time from months to minutes.
| Assertion Type | Description | Use Case |
|---|---|---|
| service_running | Verify a systemd service is running | NTP sync, audit logging, EDC connectors |
| service_enabled | Verify a service starts at boot | Critical services like auditd |
| package_installed | Verify software is installed | PostgreSQL, 2FA libraries, DICOM tools |
| file_exists | Verify a file or directory exists | Config files, audit logs, encryption keys |
| file_contains | Verify file contains specific text | Configuration validation |
| file_permissions | Verify file permissions | Security hardening, log access controls |
| user_exists | Verify a user account exists | Service accounts, operator accounts |
| user_in_group | Verify user is in a group | Access control validation |
| user_password | Verify user password is set correctly | Default credentials validation |
| port_listening | Verify a port is listening | Database, web services, HL7 receivers |
| network_reachable | Verify network connectivity | Central lab connectivity, EDC access |
| http_responds | Verify HTTP endpoint responds | Web services, REST APIs |
| command_succeeds | Verify command exits with code 0 | Custom validation scripts |
| command_output | Verify command output matches | Data validation, format checks |
| GUI & Display Testing | ||
| gui_application_opens | Launch GUI application and verify window appears | EDC client startup, imaging viewers, CTMS interfaces |
| gui_window_visible | Check if window with specific title is visible | Verify EDC login screen, confirm dialogs, error modals |
| gui_execute_command | Execute command in terminal and capture screenshot | Visual regression testing, UI state validation |
| gui_application_process | Verify application process is running | Ensure EDC client remains active, detect crashes |
| gui_screenshot_matches | Visual regression testing with baseline comparison | Detect UI changes in EDC forms, verify layouts |
| gui_click_element | Click UI elements at coordinates | Navigate menus, interact with custom controls |
| gui_form_fill | Automated form field population and submission | Test eCRF data entry workflows |
| gui_text_visible | OCR-based text detection on screen | Verify error messages, form labels, dialog text |
Beyond command-line testing, OpenFactory captures real VNC screenshots during test execution to validate visual interfaces. This is critical for CROs validating GUI-based clinical trial systems like EDC clients, CTMS interfaces, and medical imaging viewers.
Every GUI assertion automatically captures a VNC screenshot alongside command output. This provides:
Each screenshot is stored with structured metadata to enable future visual comparison. The system captures two files per assertion:
assertion-123.png
Screenshot captured via VNC during test execution
assertion-123.json
Metadata: match areas, tags, assertion details, command output
{
"area": [
{"xpos": 100, "ypos": 50, "width": 300, "height": 200, "type": "match", "match": 95}
],
"tags": ["edc-login", "passed"],
"properties": {
"description": "EDC login screen displays",
"stdout": "Login form rendered successfully"
}
}OpenFactory now supports automated comparison of screenshots against reference images, detecting visual regressions automatically:
Compares captured screenshots to reference images, failing if visual differences exceed threshold:
{
"type": "gui_screenshot_matches",
"description": "EDC form layout matches approved design",
"params": {
"reference_id": "edc-form-v3.2-approved",
"threshold": 0.95,
"ignore_regions": [[10, 10, 200, 30]],
"match_area": [0, 0, 1920, 1080],
"save_as_reference": false
}
}Use case: Detect when a software update inadvertently changes EDC form layouts, button positions, or required field indicators — all visual regressions that could invalidate a validated system.
Define regions of interest (form fields, buttons, headers) with per-area match thresholds. Ignore dynamic content like timestamps or session IDs.
Perceptual image hashing detects visual changes while tolerating minor rendering differences (anti-aliasing, fonts, subtle color shifts).
Maintain approved reference screenshots for each validated screen. Update library when intentional design changes are made and re-validated.
Automatically detect unintended visual changes in EDC systems, imaging viewers, or dashboards before deploying to production sites.
Verify that electronic Case Report Forms (eCRFs) display correctly after software updates:
Validate DICOM viewer interface for radiology endpoints:
Verify adverse event reporting forms and MedDRA coding interfaces:
FDA 21 CFR Part 11 requires that computerized systems be validated to ensure accuracy, reliability, and consistent intended performance. Visual validation provides:
A central lab needs to receive HL7 messages from clinical sites, validate message format, and store results in a database. This example validates the entire data pipeline:
{
"name": "central-lab-receiver",
"base_image": "debian-bookworm",
"features": ["ssh", "headless", "gxp", "audit-logging"],
"packages": ["postgresql", "python3-hl7", "python3-psycopg2"],
"test_config": {
"custom_tests": [
{
"description": "Verify HL7 message queue is operational",
"category": "data-collection",
"assertions": [
{
"type": "service_running",
"description": "HL7 listener service is running",
"params": {"service": "hl7-receiver"}
},
{
"type": "port_listening",
"description": "HL7 port 2575 is listening",
"params": {"port": "2575"}
},
{
"type": "command_output",
"description": "Parse sample HL7 message",
"params": {
"command": "python3 /opt/scripts/validate_hl7.py /opt/test-data/sample_lab.hl7",
"expected_output": "VALID"
}
}
]
},
{
"description": "Verify database is configured for lab results",
"category": "data-storage",
"assertions": [
{
"type": "service_running",
"description": "PostgreSQL is running",
"params": {"service": "postgresql"}
},
{
"type": "command_succeeds",
"description": "Lab results table exists",
"params": {
"command": "psql -U labuser -d labdb -c '\\dt lab_results'"
}
}
]
},
{
"description": "Verify GxP compliance - audit trails",
"category": "compliance",
"assertions": [
{
"type": "service_running",
"description": "Audit daemon is running",
"params": {"service": "auditd"}
},
{
"type": "file_exists",
"description": "Audit log exists",
"params": {"path": "/var/log/audit/audit.log"}
},
{
"type": "file_permissions",
"description": "Audit log is read-only",
"params": {"path": "/var/log/audit/audit.log", "mode": "0600"}
}
]
}
]
}
}A locked-down workstation for site coordinators to access Electronic Data Capture (EDC) systems. This validates browser restrictions, 2FA, time sync, and encryption:
{
"name": "site-edc-workstation",
"base_image": "ubuntu-24.04",
"features": ["desktop", "gxp", "audit-logging"],
"packages": ["chromium-browser", "libpam-google-authenticator", "cryptsetup"],
"test_config": {
"custom_tests": [
{
"description": "Verify EDC access is restricted to approved sites",
"category": "security",
"assertions": [
{
"type": "package_installed",
"description": "2FA authentication library installed",
"params": {"package": "libpam-google-authenticator"}
},
{
"type": "command_output",
"description": "Browser whitelisting configured",
"params": {
"command": "grep 'trial.imedidata.com' /etc/chromium/policies/managed/url_allowlist.json",
"expected_output": "trial.imedidata.com"
}
},
{
"type": "command_succeeds",
"description": "USB storage is disabled",
"params": {
"command": "lsmod | grep -v usb_storage"
}
}
]
},
{
"description": "Verify 21 CFR Part 11 compliance - time sync",
"category": "compliance",
"assertions": [
{
"type": "service_running",
"description": "Time sync service is running",
"params": {"service": "chrony"}
},
{
"type": "command_output",
"description": "Time is synchronized",
"params": {
"command": "chronyc tracking | grep 'System time'",
"expected_output": "0.0"
}
}
]
},
{
"description": "Verify disk encryption is enabled",
"category": "security",
"assertions": [
{
"type": "file_exists",
"description": "Encrypted home directory",
"params": {"path": "/dev/mapper/crypt-home"}
},
{
"type": "package_installed",
"description": "Full disk encryption tools",
"params": {"package": "cryptsetup"}
}
]
}
]
}
}A workstation for safety physicians to review serious adverse events (SAEs) and generate CIOMS forms. This validates database access, MedDRA tables, and reporting tools:
{
"name": "safety-database-station",
"base_image": "debian-bookworm",
"features": ["desktop", "gxp", "audit-logging"],
"packages": ["postgresql-client", "libreoffice", "python3-reportlab"],
"test_config": {
"custom_tests": [
{
"description": "Verify MedDRA dictionary is loaded",
"category": "data-validation",
"assertions": [
{
"type": "command_succeeds",
"description": "MedDRA LLT table exists",
"params": {
"command": "psql -h safety-db -U safetyuser -d safetydb -c '\\dt meddra_llt'"
}
},
{
"type": "command_output",
"description": "MedDRA version is 26.1",
"params": {
"command": "psql -h safety-db -U safetyuser -d safetydb -c 'SELECT version FROM meddra_version'",
"expected_output": "26.1"
}
}
]
},
{
"description": "Verify CIOMS form generation",
"category": "reporting",
"assertions": [
{
"type": "command_succeeds",
"description": "Generate CIOMS I form from test SAE",
"params": {
"command": "python3 /opt/safety/generate_cioms.py --sae-id TEST001 --output /tmp/test.pdf"
}
},
{
"type": "file_exists",
"description": "CIOMS form PDF created",
"params": {"path": "/tmp/test.pdf"}
}
]
},
{
"description": "Verify electronic signature validation",
"category": "compliance",
"assertions": [
{
"type": "package_installed",
"description": "Digital signature library",
"params": {"package": "python3-cryptography"}
},
{
"type": "command_succeeds",
"description": "Sign test document",
"params": {
"command": "python3 /opt/safety/sign_document.py /tmp/test.pdf /tmp/test.signed.pdf"
}
}
]
}
]
}
}A workstation for biostatisticians to perform CDISC SDTM/ADaM validation and statistical analysis:
{
"name": "biostat-workstation",
"base_image": "ubuntu-24.04",
"features": ["desktop", "gxp"],
"packages": ["r-base", "python3-pandas", "python3-numpy"],
"test_config": {
"custom_tests": [
{
"description": "Verify CDISC validation tools are installed",
"category": "data-validation",
"assertions": [
{
"type": "command_succeeds",
"description": "R SDTM validator package",
"params": {
"command": "Rscript -e 'library(sdtm.oak)'"
}
},
{
"type": "command_output",
"description": "Validate sample SDTM dataset",
"params": {
"command": "python3 /opt/cdisc/validate_sdtm.py /opt/test-data/dm.xpt",
"expected_output": "PASS"
}
}
]
},
{
"description": "Verify statistical software is configured",
"category": "analysis",
"assertions": [
{
"type": "command_succeeds",
"description": "R can load test dataset",
"params": {
"command": "Rscript -e 'library(haven); read_sas("/opt/test-data/adsl.sas7bdat")'"
}
},
{
"type": "package_installed",
"description": "SciPy for Python analysis",
"params": {"package": "python3-scipy"}
}
]
}
]
}
}A workstation for radiologists to review medical images (CT, MRI, X-ray) with DICOM viewers. This validates GUI application startup, DICOM network connectivity, and image rendering:
{
"name": "dicom-imaging-station",
"base_image": "ubuntu-24.04",
"features": ["desktop", "gxp"],
"packages": ["dcmtk", "weasis", "orthanc"],
"test_config": {
"custom_tests": [
{
"description": "Verify DICOM viewer launches and displays images",
"category": "gui-validation",
"assertions": [
{
"type": "gui_application_opens",
"description": "Weasis DICOM viewer launches",
"params": {
"command": "weasis",
"window_title": "Weasis",
"timeout": 10
}
},
{
"type": "gui_window_visible",
"description": "DICOM viewer main window visible",
"params": {
"window_title": "Weasis Medical Viewer"
}
},
{
"type": "gui_application_process",
"description": "Weasis process is running",
"params": {
"process_name": "weasis"
}
}
]
},
{
"description": "Verify DICOM network services (PACS connectivity)",
"category": "network",
"assertions": [
{
"type": "service_running",
"description": "Orthanc DICOM server running",
"params": {"service": "orthanc"}
},
{
"type": "port_listening",
"description": "DICOM C-STORE port 4242 listening",
"params": {"port": "4242"}
},
{
"type": "command_succeeds",
"description": "Query PACS for test study",
"params": {
"command": "dcmqrscp -q -c AE_TITLE@localhost:4242 -k StudyInstanceUID=1.2.3.4"
}
}
]
},
{
"description": "Verify image rendering with screenshot validation",
"category": "visual-validation",
"assertions": [
{
"type": "gui_execute_command",
"description": "Load test DICOM image and capture screenshot",
"params": {
"command": "weasis /opt/test-data/chest-xray.dcm",
"wait_seconds": 5
}
}
]
}
]
}
}When gui_execute_command runs, OpenFactory:
This provides visual proof that the DICOM image rendered correctly, which is critical for FDA validation of imaging systems used in clinical trials.
Instead of creating 50 pre-baked system variants, create one base image and compose different test suites for specific workflows:
Each test suite validates different aspects of the same base system, reducing duplication and maintenance overhead. When you update the base image, all test suites automatically validate the new build.
These assertion types are in development and will be available in future releases.
Execute SQL queries and validate results
Example: Verify MedDRA tables, CDISC domains
Validate HL7, DICOM, CDISC format compliance
Example: Verify HL7 v2.5 message structure
Test REST API endpoints with auth
Example: EDC API connectivity checks
Verify data freshness and log rotation
Example: Ensure audit logs are recent
Validate electronic signatures (21 CFR Part 11)
Example: Verify signed CIOMS forms
Verify SSL/TLS certificates and expiry
Example: Check EDC site certificates
Contact us to discuss your CRO testing requirements and see how OpenFactory can reduce validation time from months to minutes.