Ike Nassi

Barbarians at the Gate (https://lnkd.in/gkEBX79s) is a very interesting new paper from some great folks at Berkeley, with some exciting results showing the potential for AI in systems research. But I think the authors aren't quite asking the hardest problem about where this takes systems as a field. I wrote a new blog post about it: https://lnkd.in/gxS3s4v4 The core point of the paper is that AI can be effective at automatically driving the 'number go up' part of systems research. Given an evaluator, AI can automate progress on implementation. I believe this is 100% correct. The paper also talks about how more focus will have to go into developing evaluators. Again, I 100% agree, but think they're understating their case. Building evaluators (basically tests) is, I believe, going to become the focus of systems and software over the next decade. Which leaves researchers with the hard problem of choosing which problems to go after. That's never been easy in academic systems, and is only going to become more of a challenge. But the best systems research is driven by choosing problems well! This vision for AI could lead to lots more of the least interesting systems research. The "number go up" kind. Which could make it harder than ever to identify the truly innovative and visionary work. We're already seeing that elsewhere in science. So, mostly, I think the paper is right, but not yet asking the biggest question: what should systems research be?

276

12 Comments

New work on Privacy Leakage Evaluation for Autonomous Web Agents: Paper: https://lnkd.in/e9iSkd-k LLM-powered AI agents hold great potential for enhancing productivity but risk inadvertent privacy leakage when given access to sensitive information. To address this, AI agents can be trained to better satisfy the privacy principle of data minimization, i.e. sharing private information only when strictly necessary for a specific task. This work also develops AgentDAM, a new benchmark that evaluates how well AI agents can limit unnecessary processing of potentially private information in realistic web scenarios. Our findings show that existing agents can often misuse sensitive data, but a prompting-based approach can substantially reduce this risk. With Arman Zharmagambetov, Chuan Guo, Ivan Evtimov, Maya Pavlova, and Kamalika Chaudhuri

57

1 Comment

Meet ZebraLogic: A Comprehensive AI Evaluation Framework for Assessing LLM Reasoning Performance on Logic Grid Puzzles Derived from Constraint Satisfaction Problems (CSPs) A research team from the University of Washington, Allen Institute for AI, and Stanford University introduced ZebraLogic, a benchmarking framework developed to rigorously test LLMs’ logical reasoning performance. ZebraLogic generates logic puzzles with quantifiable complexity, ensuring a controlled environment for systematic evaluation. The framework prevents data leakage and enables a detailed analysis of an LLM’s ability to handle increasingly complex reasoning tasks. ZebraLogic serves as a crucial step toward understanding the fundamental constraints of LLMs in structured reasoning and scaling limitations. The ZebraLogic framework constructs logic puzzles with varying difficulty levels based on two primary complexity measures: search space size and Z3 conflict count, a metric derived from an SMT solver. The study tested leading LLMs, including Meta’s Llama, OpenAI’s o1 models, and DeepSeekR1, and revealed significant accuracy declines as puzzle complexity increased. The framework allowed for a precise assessment of reasoning capabilities across different levels of problem difficulty, making it one of the most structured evaluations of LLMs to date. By systematically varying the constraints, researchers could determine the impact of problem size on logical reasoning performance..... Read the full article: https://lnkd.in/gRn4XSFC Paper: https://lnkd.in/gaziAss3 Project Page: https://lnkd.in/gkxYiwQc Bill Yuchen Lin Ronan Le Bras Kyle R. Ashish Sabharwal Radha Poovendran Peter Clark Yejin Choi

30