Limits¶
What a result proves¶
VERIFIEDmeans each save was confirmed by a record check that was configured for that save. A field the check doesn't compare can still be wrong, and a value that was wrong in the source document is entered as given.- A qualification pass covers the environment it ran in: the application version, browser, screen size, and system of record it was tested against. A new application version or a changed screen needs a new qualification.
- Zero model calls doesn't mean zero network traffic. The application, the record check, and a Cloud connection can still use the network.
- Deterministic means the program fixes the steps, inputs, targets, and checks. It doesn't mean the application returns the same screens every time, or that a step is safe to repeat.
What it doesn't cover¶
- Steps without a right-record check. A compiled program arms that
check on a subset of its clicks; one OpenEMR program armed 4 to 7 of 12.
A click that isn't armed has no right-record check.
lintand every run report list which steps are armed. - Saves without a record check. Without one, a run can't end
VERIFIED, and Standard and Regulated refuse the save. Record checks are usually written per deployment, against the app's system of record. - Redesigned workflows. Re-finding a moved or restyled control is in scope. A changed sequence of screens or changed business rules needs a new demonstration or a taught fix.
- Unattended clinical judgment. OpenAdapt enters what it's given. It doesn't decide what belongs in the record.
Where it stops more often¶
- RDP and Citrix. With only pixels, identifiers with confusable
characters can't always be read with confidence, so runs stop more than in
a browser. On pixel-only screens the right-record check stops whenever the
right patient's own record number has look-alike characters, such as
Oand0. On our test corpora that was 47 to 61 in 100 correct-patient checks, re-measured 17 October 2026 (identity evidence). Those are extra stops, not wrong entries. A confirmed export of the system's record numbers (runtime.known_ids, openadapt-flow 2.2.0) lifts a stop when exactly one listed number fits; on a rendered record list, stops on the right row fell from 176 to 3 in 280 clicks whose neighboring row wasn't a look-alike twin (results). Teach the specific case, or use a surface with structure where the app offers one. Don't lower the check. - First runs of a new recording. Expect
lintwarnings andclinical-writerefusals until record checks and right-record checks are complete. - Screens that change between finding a target and acting, such as a late notification. The step stops rather than click a stale target.
Validated on¶
- OpenEMR, the open-source EHR whose 7.0 release is ONC-certified: test copies of OpenEMR 8.0.0.3 with synthetic patients.
- openIMIS, the open-source health insurance and claims system that runs Nepal's national health insurance scheme: a test copy with synthetic claims.
- RDP into a Windows 11 host over a real network.
- A Citrix stand-in: a client window with no DOM, driven from pixels.
- OpenAdapt's own test apps on Windows, macOS, and Linux.
A given application on a given surface is qualified in its own environment before production use.
Results by surface¶
| Surface | On record | Scope |
|---|---|---|
| Browser | OpenEMR referral entry, 14 to 15 Oct 2026: of 63 new-patient referrals, 62 saved and read back 12 of 12 fields; 9 of 9 possible duplicates with no new chart, 0 model calls (run data). openIMIS claims entry: 148 of 149 claims saved correctly by read-back, 0 silent wrong saves, 0 model calls (scoreboard) | Local test copies with synthetic patients and claims. In the OpenEMR run the test harness did the read-back, so the engine labeled the runs finished, not checked. |
| Windows | 3 of 3 saves in our WinForms app confirmed in SQLite; 3 of 3 refusals each for stale and ambiguous targets; 0 model calls (report) | Our own app on one Windows 11 ARM VM snapshot. RDP and Citrix recordings don't use local UI Automation. |
| macOS | 3 of 3 TextEdit saves matched byte for byte; refused an ambiguous two-window target without changing either file (adjudication) | One macOS 15.7.3 host and one app. The original report reads failed because of cleanup warnings; the adjudication accepts the saves and the refusal. |
| Linux (X11) | In required CI on Ubuntu 24.04: a GTK3 app, 3 confirmed saves, 3 ambiguous-target and 3 stale-target refusals | Our own app on Xvfb. |
| RDP | Windows 11 VM over Aardwolf: 3 of 3 file writes confirmed by a separate guest check (report). Record, compile and replay over FreeRDP: 3 of 3 saves, 3 of 3 stops under drift (report) | The FreeRDP run used a synthetic Linux app. Pixel-only, so the false-stop rate above applies. |
| Citrix | 3 confirmed saves and 3 stops under severe drift on a Citrix stand-in with no DOM; 0 model calls (report) | A stand-in window, not an ICA/HDX session. Each Citrix deployment is qualified in its own environment. Pixel-only, so the false-stop rate above applies. |
The full failure inventory, with evidence for each entry, is in
packages/flow/docs/LIMITS.md.
Lane boundaries¶
- Agent-driven recording on native Windows, RDP, and Citrix is tested against a mock Windows server and stand-in client windows.
- Qualification's fault staging works in the browser. On other surfaces,
qualifyrefuses targets other thanweb. problems_onlyapproval turns on only after a workflow passes a readiness gate. Until then, every save asks a person.- The managed Cloud runner runs browser workflows only. It isn't a lane for patient data.
What OpenAdapt doesn't configure for you¶
In an on-premises deployment, your team owns the firewall, disk encryption and key management, storage, identity provider, backups, retention, incident response, and compliance program. OpenAdapt supplies the run gates, egress checks, and audit log that sit inside them.