Service: mhv-du-rx-service-api
Success definition: Got any response back from Oracle Health (including ER:Error business rejections). Failure = HL7 send timeout / no response.
Day total: 341 orders | 312 success | 29 failure | 91.5% response rate
| Hour (ET) | OH Orders | Success | Failure | Response % |
|---|---|---|---|---|
| 12:00 AM | 16 | 7 | 9 | 43.8% |
| 2:00 AM | 1 | 1 | 0 | 100% |
| 4:00 AM | 5 | 5 | 0 | 100% |
| 6:00 AM | 18 | 17 | 1 | 94.4% |
| 7:00 AM | 2 | 2 | 0 | 100% |
| 8:00 AM | 10 | 10 | 0 | 100% |
| 9:00 AM | 16 | 16 | 0 | 100% |
| 10:00 AM | 10 | 10 | 0 | 100% |
| 11:00 AM | 31 | 31 | 0 | 100% |
| 12:00 PM | 30 | 29 | 1 | 96.7% |
| 1:00 PM | 26 | 26 | 0 | 100% |
| 2:00 PM | 9 | 9 | 0 | 100% |
| 3:00 PM | 21 | 21 | 0 | 100% |
| 4:00 PM | 17 | 17 | 0 | 100% |
| 5:00 PM | 14 | 13 | 1 | 92.9% |
| 6:00 PM | 16 | 16 | 0 | 100% |
| 7:00 PM | 16 | 16 | 0 | 100% |
| 8:00 PM | 22 | 22 | 0 | 100% |
| 9:00 PM | 8 | 8 | 0 | 100% |
| 10:00 PM | 23 | 23 | 0 | 100% |
| 11:00 PM | 30 | 14 | 16 | 46.7% |
Notes:
- 12:00 AM: 9 failures from a single request batch (
f4076af3) at 12:48 AM — all HL7 timeouts, 1 proxy timeout. - 6:00 AM, 12:00 PM, 5:00 PM: Isolated single-order timeouts (transient).
- 11:00 PM: 16-failure cluster from 11:36–11:54 PM — sustained HL7 endpoint outage lasting ~20 minutes.
Day total: 595 orders | 252 success | 343 failure | 42.4% response rate
| Hour (ET) | OH Orders | Success | Failure | Response % |
|---|---|---|---|---|
| 12:00 AM | 5 | 5 | 0 | 100% |
| 1:00 AM | 61 | 43 | 18 | 70.5% |
| 2:00 AM | 1 | 1 | 0 | 100% |
| 5:00 AM | 1 | 1 | 0 | 100% |
| 6:00 AM | 5 | 4 | 1 | 80.0% |
| 8:00 AM | 12 | 11 | 1 | 91.7% |
| 9:00 AM | 21 | 16 | 5 | 76.2% |
| 10:00 AM | 19 | 16 | 3 | 84.2% |
| 11:00 AM | 56 | 52 | 4 | 92.9% |
| 12:00 PM | 205 | 103 | 102 | 50.2% |
| 1:00 PM | 150 | 0 | 150 | 0.0% |
| 2:00 PM | 59 | 0 | 59 | 0.0% |
Notes:
- 1:00 AM: 18 failures from ~1:42–1:52 AM — burst of HL7 timeouts, then self-recovered.
- 9:00–11:00 AM: Elevated failures (3–5/hr) — HL7 endpoint was degraded before the major outage.
- 12:00 PM: Outage began — 50.2% success, 10 proxy timeouts.
- 1:00–2:22 PM: Complete outage — 0% HL7 success, 16 proxy timeouts. All messages timing out at 15–30s.
- After 2:22 PM: Zero OH orders — OH refill processing was intentionally disabled.
- All failures are
java.lang.Exception: Failed to send HL7 message after 1 attempts(HL7v2 MLLP timeouts to Oracle Health).