AI & ML
JEV vs. LLMs: A Comparative Analysis for Phishing Email Detection
arXiv cs.CR0 views
arXiv:2611.23683v1 Announce Type: new Abstract: Phishing remains a major source of credential theft and fraudulent payments, and email security systems must classify large message volumes reliably, quickly, and at manageable cost. Large Language Models (LLMs) offer flexible text understanding, but their token-based generation increases response time and API cost, and the reliability of their confidence values requires measurement. Jev 1.13, a typed decision model from TypeSafe AI, instead returns a structured outcome with a confidence value. This paper compares Jev 1.13 with Gemini 3.5 Flash-Lite, NVIDIA Nemotron 3 Ultra, and Meta Llama 4 Maverick on a labelled dataset of 1500 emails, covering classification performance, decision latency, token consumption, API cost, and probability calibration. Jev achieves the highest accuracy (79.67%), precision (71.43%), recall (97.97%), and F1 score (82.62%). It also records the shortest batch time (173.88 seconds), the lowest token count (597,895), the lowest cost (USD 0.015103 per 1000 decisions), and the lowest Brier score (0.1698) and ECE (0.1594). These results suggest that Jev is a promising option for fast, structured, high-volume triage. LLMs retain the advantage of generating explanations, which we do not evaluate, and all four systems require further validation before standalone deployment.
Source:https://arxiv.org/abs/2611.23683PreviousFlowSH: Detecting Web Attacks by Correlating Anomalous Network Flows and Endpoint LogsNextData Poisoning Attacks to Locally Differentially Private Triangle Counting Protocols
Related posts
Krita is a sketching and painting program designed for digital artists
0Hacker News New
Agent Duel
0Hacker News New
AI can be tricked to target civilians as military targets by preteding it's game
0Hacker News New
If you're concerned about citation hallucination by LLMs, you're 2 years behind
0Hacker News New
US National Cyber Director to step down
0Hacker News New
ChatGPT for SNES
0Hacker News New