AI & ML
Beyond the Landing Page: Website Blocking, Tracker Visibility, and Human Alignment in Automated Web Crawls
arXiv cs.CR0 views
arXiv:2611.06197v1 Announce Type: new Abstract: Web tracking studies largely rely on automated crawlers, yet their measurements can differ from what people encounter while browsing. Conventional crawlers follow predefined navigation rules, whereas LLM-driven agents decide how to navigate and interact with websites, potentially changing the tracking they encounter. We compare agentic and non-agentic crawlers to examine how websites respond to them, what tracking they record, and how closely their observations match human browsing. Across 2,000 popular websites, we evaluate different navigation strategies, automation frameworks, and stealth configurations, alongside browsing data from 26 participants. We find that website denials increase beyond landing pages for standard browser automation, while a detection-resistant automation framework largely prevents this escalation without increasing recorded tracking. On the same URLs, agentic crawlers record tracking activity from 1.5-1.7 times as many third-party organizations, particularly on consent-managed pages. Agentic crawlers recover 82-87% of tracker organizations observed by participants, compared with 58-68% for non-agentic crawlers, yet roughly half of their own observations are absent from human participant visits. These findings show that avoiding website interventions, increasing tracking visibility, and achieving closer agreement with human observations do not necessarily go hand in hand, highlighting the need to select and validate crawlers according to the measurement objective.
Source:https://arxiv.org/abs/2611.06197PreviousNetwork Intrusion Forensic Detection System with Explainable Artificial Intelligence and Digital Chain of CustodyNextExplainable Anomaly Detection in 5G Networks through Continuous Time Graphs
Related posts
How Chiron Changes AI Incident Investigation on ORCA-bench
0Hacker News New
Ask HN: How do small teams pragmatically manage Dependabot fatigue?
0Hacker News New
How WeRateDogs Is Fighting A.I. Slop, One Perfect Puppy at a Time
0Hacker News New
Piston – small causal model for time series that can refuse
0Hacker News New
Novel AI framework runs on optical hardware
0Hacker News New
Nexus-Q: the first commits were a threat model and architecture, not code
0Hacker News New