LLM Agents Hunt for Vulnerabilities in Medical Devices
AI systems now find software flaws faster than anyone can check them. A new preprint from the Syracuse University Institute for Artificial Intelligence puts a logic engine between the model and the analyst.
Using LLM’s to find software vulnerabilities has made big news this year. However, tools like Claude Mythos and ChatGPT have generally been shown to excel at vulnerability discovery on source code – where the model has more context than a decompiled binary. However, for medial device security, we need to find vulnerabilities in binaries before the attackers to secure the devices. This is what led to the ARPA-H UPGRADE program which Syracuse is doing work on. Researchers at Syracuse and HRL have introduced a new preprint on a system, EntailLLM, designed to guide an LLM in vulnerability discovery based on reverse engineering domain knowledge.
An LLM mistake caught by domain knowledge. The green path is the trajectory the model proposed; the flagged step in red is the assumption the logic engine refuses to accept. (Figure 1, Mukherj…
An LLM identifies potential vulnerabilities as a path through the relationship of functions in the decompiled binary. EntailLLM treats every proposed path as a hypothesis rather than an answer. The path is a walk through the binary's function call graph; the domain knowledge, drawn from CWE entries, MITRE material and device documentation, sits in a separate graph. Verification means lining the two up, and when no chain of rules justifies a step, the system rejects it and reports what was missing.
Pruning paths that do not fit a program's structure is statistical work that language models already do reasonably well. Rejecting a path that violates what is known about the domain, even when it is structurally fine, is semantic, and the paper argues that takes logic rather than probability.
Across three vulnerability classes, four language models and seven binaries ranging from 405 to 12,696 functions, adding domain knowledge lifted the share of logically supported paths from 78 percent to 98 percent, falling in only three of 94 configurations. The best setup raised the count of verified paths by 37 percent, from 1,054 to 1,445, which matters, because a filter that only throws work away is not much of a filter.
It held up outside the lab. Firmware pulled from deployed medical devices, which tends to be long-lived, awkward to patch and shipped without source code, started at 71.4 percent and reached 98 percent with no per-device tuning. The system runs in production on a modest cloud instance, handing verified paths to a fuzzer.
Image
Syracuse researchers presenting EntailLLM at an ARPA-H hackathon event.
The authors are candid about the limit. Entailment is not exploitability, and a path can be logically sound and lead nowhere dangerous. EntailLLM says nothing about containment either, and will not stop a model that leaves its box. What it offers is the posture the paper traces to shielding in safe reinforcement learning: let the model propose freely, and put something outside it, auditable and rule-governed, in charge of what gets through.
The paper is the work of Syracuse Ph.D. students Kaustuv Mukherji, Jaikrishna Manojkumar Patil, Syracuse MS student Colton Payne and faculty member Paulo Shakarian in addition to Dana Warmsley, Nigel Stepp and Evelyn Kim of HRL Laboratories. It was funded in part by ARPA-H.
The preprint, EntailLLM: Verifying LLM-Generated Vulnerability Discovery Paths with Domain Knowledge via Logic Programming, is posted as arXiv:2608.01763.
