Supporting Artifact Evaluation with LLMs: A Study with Published Security Research Papers

Artifact Evaluation (AE) is essential for ensuring the transparency and reliability of research, closing the gap between exploratory work and real-world deployment is particularly important in cybersecurity, particularly in IoT and CPSs, where large-scale, heterogeneous, and privacy-sensitive data meet safety-critical actuation. Yet, manual reproducibility checks are time-consuming and do not scale with growing submission volumes. In this work, we demonstrate that Large Language Models (LLMs) can provide powerful support for AE tasks: (i) text-based reproducibility rating, (ii) autonomous sandboxed execution environment preparation, and (iii) assessment of methodological pitfalls. Our reproducibility-assessment toolkit yields an accuracy of over 72% and autonomously sets up execution environments for 28% of runnable cybersecurity artifacts. Our automated pitfall assessment detects seven prevalent pitfalls with high accuracy (F1 > 92%). Hence, the toolkit significantly reduces reviewer effort and, when integrated into established AE processes, could incentivize authors to submit higher-quality and more reproducible artifacts. IoT, CPS, and cybersecurity conferences and workshops may integrate the toolkit into their peer-review processes to support reviewers’ decisions on awarding artifact badges, improving the overall sustainability of the process. ...

David Heye, Karl Kindermann, Robin Decker, Johannes Lohmöller, Anastasiia Belova, Sandra Geisler, Klaus Wehrle, Jan Pennekamp

MAC Aggregation over Lossy Channels in DTLS 1.3

Aggregating Message Authentication Codes (MACs) promises to save valuable bandwidth in resource-constrained environments. The idea is simple: Instead of appending an authentication tag to each message in a communication stream, the integrity protection of multiple messages is aggregated into a single tag. Recent studies postulate, e.g., based on simulations, that these benefits also spread to wireless, and thus lossy, scenarios despite each lost packet typically resulting in the loss of integrity protection information for multiple messages. In this paper, we investigate these claims in a real deployment. Therefore, we first design a MAC aggregation extension for the Datagram Transport Layer Security (DTLS) 1.3 protocol. Afterward, we extensively evaluate the performance of MAC aggregation on a complete communication protocol stack on embedded hardware. We find that MAC aggregation can indeed increase goodput by up to 50% and save up to 17% of energy expenditure for the transmission of short messages, even in lossy channels. ...

Eric Wagner, David Heye, Jan Bauer, Klaus Wehrle, Martin Serror

Poster: Transport Security Orchestration Using DNS

Communication networks enable the exchange of data with varying sensitivity, from non-sensitive public files to highly confidential healthcare or financial records. Cryptographic protection introduces significant computational and communication overhead. While lightweight ciphers have been proposed to reduce this burden, they compromise security and are unsuitable for sensitive data. We propose a system that enables adaptive security by embedding service sensitivity information in the Domain Name System (DNS), allowing peers to select appropriate cryptographic primitives based on data requirements. This approach ensures adequate protection while minimizing overhead. Additionally, it can be seamlessly integrated into existing networks without additional hardware. Initial results indicate improved throughput and reduced computational load on hosts. ...

David Heye, Sahi Islam, Jan Pennekamp, Klaus Wehrle

Madtls: Fine-grained Middlebox-aware End-to-end Security for Industrial Communication

Industrial control systems increasingly rely on middlebox functionality such as intrusion detection or in-network processing. However, traditional end-to-end security protocols interfere with the necessary access to in-flight data. While recent work on middlebox-aware end-to-end security protocols for the traditional Internet promises to address the dilemma between end-to-end security guarantees and middleboxes, the current state-of-the-art lacks critical features for industrial communication. Most importantly, industrial settings require fine-grained access control for middleboxes to truly operate in a least-privilege mode. Likewise, advanced applications even require that middleboxes can inject specific messages (e.g., emergency shutdowns). Meanwhile, industrial scenarios often expose tight latency and bandwidth constraints not found in the traditional Internet. As the current state-of-the-art misses critical features, we propose Middlebox-aware DTLS (Madtls), a middlebox-aware end-to-end security protocol specifically tailored to the needs of industrial networks. Madtls provides bit-level read and write access control of middleboxes to communicated data with minimal bandwidth and processing overhead, even on constrained hardware. ...

Eric Wagner, David Heye, Martin Serror, Ike Kunze, Klaus Wehrle, Martin Henze