我们从2025-08-18到2025-08-22的71篇文章中精选出10篇优秀的工作分享给读者，主要研究方向包括多语言安全基准、推理能力提升、金融领域评估框架、幻觉缓解、语义意识的标记化、检索增强生成、增强学习中的深度与广度、工具集成推理、提高真实度的控制方法。

1. LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models
2. Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward
3. From Scores to Skills: A Cognitive Diagnosis Framework for Evaluating Financial Large Language Models
4. Mitigating Hallucinations in Large Language Models via Causal Reasoning
5. SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling
6. Conflict-Aware Soft Prompting for Retrieval-Augmented Generation
7. Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration
8. Dissecting Tool-Integrated Reasoning: An Empirical Study and Analysis
9. Sycophancy under Pressure: Evaluating and Mitigating Sycophantic Bias via Adversarial Dialogues in Scientific QA
10. Generics and Default Reasoning in Large Language Models

1.LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models

Authors: Zhiyuan Ning, Tianle Gu, Jiaxin Song, Shixin Hong, Lingyu Li, Huacan Liu, Jie Li, Yixu Wang, Meng Lingyu, Yan Teng, Yingchun Wang

Affiliations: Shanghai Artificial Intelligence Laboratory;  Shanghai Jiao Tong University;  Tsinghua University;  Fudan University

https://arxiv.org/abs/2508.12733

论文摘要

The widespread adoption and increasing prominence of large language models (LLMs) in global technologies necessitate a rigorous focus on ensuring their safety across a diverse range of linguistic and cultural contexts. The lack of a comprehensive evaluation and diverse data in existing multilingual safety evaluations for LLMs limits their effectiveness, hindering the development of robust multilingual safety alignment. To address this critical gap, we introduce LinguaSafe, a comprehensive multilingual safety benchmark crafted with meticulous attention to linguistic authenticity. The LinguaSafe dataset comprises 45k entries in 12 languages, ranging from Hungarian to Malay. Curated using a combination of translated, transcreated, and natively-sourced data, our dataset addresses the critical need for multilingual safety evaluations of LLMs, filling the void in the safety evaluation of LLMs across diverse under-represented languages from Hungarian to Malay. LinguaSafe presents a multidimensional and fine-grained evaluation framework, with direct and indirect safety assessments, including further evaluations for oversensitivity. The results of safety and helpfulness evaluations vary significantly across different domains and different languages, even in languages with similar resource levels. Our benchmark provides a comprehensive suite of metrics for in-depth safety evaluation, underscoring the critical importance of thoroughly assessing multilingual safety in LLMs to achieve more balanced safety alignment. Our dataset and code are released to the public to facilitate further research in the field of multilingual LLM safety.

论文简评：LinguaSafe是一个开创性的研究项目，它为大型语言模型（LLMs）提供了一个全面的多语言安全基准。这个倡议旨在填补现有多语言安全评估中的空白，通过收集和整理一个涵盖12种语言的大型数据集。数据集结合了本地、翻译和再创造的来源，以确保语言细微差别的真实表达。LinguaSafe采取的多维度方法提供了对LLM安全性的细致理解，使得各个语言和领域的分析更加全面。

该项目不仅强调了在安全评估中多样性的重要性，还展示了 robust 数据策划方法的有效性。使用本地、翻译和再创造数据的策略保证了语言表达的真实性，对数据集的整体质量有重要贡献。通过将这一多样化数据结合到一个单一基准中，LinguaSafe为LLMs在不同条件下的表现提供了全面的概述，为未来的安全评估进展铺平了道路。

总之，LinguaSafe 代表了多语言安全领域的一个重要进步。凭借其广泛的数据集和创新评估框架，它证明了多语言安全基准在负责任的人工智能系统开发中的潜在价值。因此，这项工作见证了跨学科合作在推动人工智能生态系统内创新和进步方面的力量。

2.Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic Thought Reward

Authors: Yong Deng, Guoqing Wang, Zhenzhe Ying, Xiaofeng Wu, Jinzhen Lin, Wenwen Xiong, Yuqin Dai, Shuo Yang, Zhanwei Zhang, Qiwen Wang, Yang Qin, Changhua Meng

Affiliations: Ant Group

https://arxiv.org/abs/2508.12800

论文摘要

Large language models (LLMs) exhibit remarkable problem-solving abilities, but struggle with complex tasks due to static internal knowledge. Retrieval-Augmented Generation (RAG) enhances access to external information, yet remains limited in multi-hop reasoning and strategic search due to rigid workflows. Recent advancements in agentic deep research empower LLMs to autonomously reason, search, and synthesize information. However, current approaches relying on outcome-based reinforcement learning (RL) face critical issues such as conflicting gradients and reward sparsity, limiting performance gains and training efficiency. To address these, we first propose Atomic Thought, a novel LLM thinking paradigm that decomposes reasoning into fine-grained functional units. These units are supervised by Reasoning Reward Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained guidance. Building on this, we propose Atom-Searcher, a novel RL framework for agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher uses a curriculum-inspired reward schedule, prioritizing process-level ATR early and transitioning to outcome rewards, accelerating convergence on effective reasoning paths. Experiments on seven benchmarks show consistent improvements over the state-of-the-art. Key advantages include: (1) Atom-Searcher scales computation at test-time. (2) Atomic Thought provides supervision anchors for RRMs, bridging deep research tasks and RRMs. (3) Atom-Searcher exhibits more interpretable, human-like reasoning patterns.

论文简评：Atom-Searcher 是一个全新的框架，它通过整合一种名为原子思维的新理念来增强大型语言模型（LLM）的推理能力。这一方法旨在通过细化奖励机制来改善多步推理能力和战略搜索，并解决了现有检索增强生成方法中的一些限制。作者们展示了他们在多个基准测试上的显著性能改进。

该研究的核心在于提出了一种新的思考模式——原子思维。这种思想强调了更细粒度的奖励分配，有助于提高 LLMs 的多步推理能力和策略性搜索能力。这项工作的突破在于它提出了一个创新的方法论框架，这个框架不仅能够解决现有的挑战，而且还能为未来的研究提供指导。

在实验结果方面，作者们展示出了令人印象深刻的表现。他们成功地在多个基准测试上提高了模型的性能，尤其是在处理复杂问题时。这种进步表明了他们的方法具有强大的实际应用潜力，特别是在需要高度推理和决策的应用场景中。

总的来说，Atom-Searcher 是一个值得称赞的研究成果。它引入了一种新颖且有前景的思考方式，同时证明了其在提升大型语言模型性能方面的有效性。通过使用精细的奖励机制，研究人员成功地超越了现有技术，并为未来的研究提供了重要启示。

3.From Scores to Skills: A Cognitive Diagnosis Framework for Evaluating Financial Large Language Models

Authors: Ziyan Kuang, Feiyu Zhu, Maowei Jiang, Yanzhao Lai, Zelin Wang, Zhitong Wang, Meikang Qiu, Jiajia Huang, Min Peng, Qianqian Xie, Sophia Ananiadou

Affiliations: Wuhan University;  Nanjing Audit University;  Southwest Jiaotong University;  Beijing University of Financial Technology;  Augusta University;  University of Manchester

https://arxiv.org/abs/2508.13491

论文摘要

Large Language Models (LLMs) have shown promise for financial applications, yet their suitability for this high-stakes domain remains largely unproven due to inadequacies in existing benchmarks. Existing benchmarks solely rely on score-level evaluation, summarizing performance with a single score that obscures the nuanced understanding of what models truly know and their precise limitations. They also rely on datasets that cover only a narrow subset of financial concepts, while overlooking other essentials for real-world applications. To address these gaps, we introduce FinCDM, the first cognitive diagnosis evaluation framework tailored for financial LLMs, enabling the evaluation of LLMs at the knowledge-skill level, identifying what financial skills and knowledge they have or lack based on their response patterns across skill-tagged tasks, rather than a single aggregated number. We construct CPA-QKA, the first cognitively informed financial evaluation dataset derived from the Certified Public Accountant (CPA) examination, with comprehensive coverage of real-world accounting and financial skills. It is rigorously annotated by domain experts, who author, validate, and annotate questions with high inter-annotator agreement and fine-grained knowledge labels. Our extensive experiments on 30 proprietary, open-source, and domain-specific LLMs show that FinCDM reveals hidden knowledge gaps, identifies under-tested areas such as tax and regulatory reasoning overlooked by traditional benchmarks, and uncovers behavioral clusters among models. FinCDM introduces a new paradigm for financial LLM evaluation by enabling interpretable, skill-aware diagnosis that supports more trustworthy and targeted model development, and all datasets and evaluation scripts will be publicly released to support further research.\footnote{https://github.com/WHUNextGen/FinCDM}

论文简评：总的来说，《FinCDM：一种认知诊断框架》这篇论文提出了一种新的评估方法来评估金融领域的大型语言模型（LLMs），以弥补现有基准中依赖于聚合分数的局限性，并提供基于任务间技能标签化反应模式的技能水平评估。此外，该研究还推出了CPA-QKA数据集，这是由经验丰富的专家精心构建和注释的，提供了涵盖广泛财务知识和技能的全面覆盖。

该研究的核心优势在于其对隐藏的知识差距进行了揭示，并为模型开发提供了有价值的洞察力。通过对不同模型的大量实验，研究人员能够详细分析这些模型在金融知识方面的强项和弱项。这些实验结果对于指导模型设计和优化具有重要意义。

综上所述， 《FinCDM：一种认知诊断框架》不仅展示了新的评估方法的重要性，而且也证明了其在实际应用中的价值。

4.Mitigating Hallucinations in Large Language Models via Causal Reasoning

Authors: Yuangang Li, Yiqing Shen, Yi Nian, Jiechao Gao, Ziyi Wang, Chenxiao Yu, Shawn Li, Jie Wang, Xiyang Hu, Yue Zhao

Affiliations: University of Southern California;  Johns Hopkins University;  Stanford University;  University of Maryland, College Park;  Arizona State University

https://arxiv.org/abs/2508.12495

论文摘要

Large language models (LLMs) exhibit logically inconsistent hallucinations that appear coherent yet violate reasoning principles, with recent research suggesting an inverse relationship between causal reasoning capabilities and such hallucinations. However, existing reasoning approaches in LLMs, such as Chain-of-Thought (CoT) and its graph-based variants, operate at the linguistic token level rather than modeling the underlying causal relationships between variables, lacking the ability to represent conditional independencies or satisfy causal identification assumptions. To bridge this gap, we introduce causal-DAG construction and reasoning (\our{}), a supervised fine-tuning framework that trains LLMs to explicitly construct variable-level directed acyclic graph (DAG) and then perform reasoning over it. Moreover, we present a dataset comprising 25,368 samples (CausalDR), where each sample includes an input question, explicit causal DAG, graph-based reasoning trace, and validated answer. Experiments on four LLMs across eight tasks show that \our{} improves the causal reasoning capability with the state-of-the-art 95.33% accuracy on CLADDER (surpassing human performance of 94.8% for the first time) and reduces the hallucination on HaluEval with 10% improvements. It demonstrates that explicit causal structure modeling in LLMs can effectively mitigate logical inconsistencies in LLM outputs. Code is available at \textcolor{blue}{\url{https://github.com/MrLYG/CDCR-SFT}}.

论文简评：该论文旨在提出一种新的框架CDCR-SFT，以改进大型语言模型（LLM）的因果推理能力，从而减少幻觉。通过介绍一个用于训练LLM构建因果有向无环图（DAG）及其推理的新数据集CausalDR，本文展示了显著的因果推理准确性提升以及对多个基准测试中的幻觉显著降低。实验结果表明，提出的框架在改善因果推理能力和减少幻觉方面取得了显著效果，显示出其有效性。总的来说，这篇论文提供了关于如何改进LLM的因果推理能力的重要见解，并提供了一种有效的方法来解决这个问题。

5.SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modeling

Authors: Dong Liu, Yanxuan Yu

Affiliations: Yale University;  Columbia University

https://arxiv.org/abs/2508.15190

论文摘要

Tokenization plays a critical role in language modeling, yet existing approaches such as Byte-Pair Encoding (BPE) or WordPiece operate purely on frequency statistics, ignoring the underlying semantic structure of text. This leads to over-tokenization of semantically redundant spans and underutilization of contextual coherence, particularly in long-context scenarios. In this work, we propose \textbf{SemToken}, a semantic-aware tokenization framework that jointly reduces token redundancy and improves computation efficiency. SemToken first extracts contextual semantic embeddings via lightweight encoders and performs local semantic clustering to merge semantically equivalent tokens. Then, it allocates heterogeneous token granularity based on semantic density, allowing finer-grained tokenization in content-rich regions and coarser compression in repetitive or low-entropy spans. SemToken can be seamlessly integrated with modern language models and attention acceleration methods. Experiments on long-context language modeling benchmarks such as WikiText-103 and LongBench show that SemToken achieves up to  reduction in token count and  speedup, with negligible or no degradation in perplexity and downstream accuracy. Our findings suggest that semantic structure offers a promising new axis for optimizing tokenization and computation in large language models.

论文简评：这篇论文提出了一种名为SemToken的语料库表示方法，旨在通过减少冗余令牌和提高计算效率来改进长文本语言建模的效率。该框架利用语义嵌入和局部聚类来合并具有相似语义的令牌，并根据语义密度分配令牌粒度。这种新颖的方法有望带来显著的语言模型性能提升，因为它的应用能够解决当前长文本模型中普遍存在的效率问题。

论文的主要贡献在于它提出的创新策略，即融合语义意识的语料库表示方法。该方法能够有效地处理复杂的语境信息，为构建更准确、高效的机器翻译系统提供了可能。此外，实验结果表明，在多个基准数据集上，采用SemToken技术的模型能够在保持同样或更好的预测准确性的同时，显著降低计算成本并加快推理速度。这些发现进一步证明了SemToken方法的有效性和潜力，为未来的语言理解和生成任务提供了一个值得期待的新方向。

总之，这篇论文提出了一个基于语义意识的语料库表示方法，其对现有语言模型的挑战做出了积极回应，并在多个领域展示了其潜在的应用价值。未来的研究应该继续探索如何进一步优化和扩展SemToken的方法论，以实现更有效的语言理解与生成。

6.Conflict-Aware Soft Prompting for Retrieval-Augmented Generation

Authors: Eunseong Choi, June Park, Hyeri Lee, Jongwuk Lee

Affiliations: Sungkyunkwan University

https://arxiv.org/abs/2508.15253

论文摘要

Retrieval-augmented generation (RAG) enhances the capabilities of large language models (LLMs) by incorporating external knowledge into their input prompts. However, when the retrieved context contradicts the LLM's parametric knowledge, it often fails to resolve the conflict between incorrect external context and correct parametric knowledge, known as context-memory conflict. To tackle this problem, we introduce Conflict-Aware REtrieval-Augmented Generation (CARE), consisting of a context assessor and a base LLM. The context assessor encodes compact memory token embeddings from raw context tokens. Through grounded/adversarial soft prompting, the context assessor is trained to discern unreliable context and capture a guidance signal that directs reasoning toward the more reliable knowledge source. Extensive experiments show that CARE effectively mitigates context-memory conflicts, leading to an average performance gain of 5.0% on QA and fact-checking benchmarks, establishing a promising direction for trustworthy and adaptive RAG systems.

论文简评：在这篇关于冲突感知检索增强生成（CARE）的研究中，作者提出了一个解决方案来解决检索增强生成系统中存在的上下文-内存冲突问题。CARE方法通过使用一种基于软提示的上下文评估器来识别可靠的外部上下文与不可靠的上下文。该研究表明，CARE平均性能提升了5.0%，这对于提高检索增强生成系统的稳健性具有潜在的好处。

研究的关键点在于其对检索增强生成系统中的上下文内存冲突进行了深入分析，并提出了一种创新的方法来解决这一问题。这种方法利用了一个基于软提示的上下文评估器，展现了其在提高可靠性方面的效果，从而为检索增强生成系统的稳定性和准确性提供了可能的改进空间。

实验结果表明，CARE相较于现有方法表现出了显著的优势，进一步验证了该方法的有效性。总的来说，这篇论文详细地探讨了上下文内存冲突的问题，并提出了一个有效的解决方案。这些发现对于促进检索增强生成系统的稳健性和准确性的研究具有重要意义。

7.Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration

Authors: Zhicheng Yang, Zhijiang Guo, Yinya Huang, Yongxin Wang, Dongchun Xie, Yiwei Wang, Xiaodan Liang, Jing Tang

Affiliations: The Hong Kong University of Science and Technology (Guangzhou);  The Hong Kong University of Science and Technology;  University of California, Merced;  ETH AI Center, ETH Zurich;  Sun Yat-sen University;  MBZUAI

https://arxiv.org/abs/2508.13755

论文摘要

Reinforcement Learning with Verifiable Reward (RLVR) has emerged as a powerful paradigm for unlocking reasoning capabilities in large language models, yet its full potential is hindered by two under-explored dimensions: Depth-the hardest problem a model can sample; Breadth-the number of instances consumed in a single iteration. We dissect the popular GRPO algorithm and reveal a systematic bias: the cumulative-advantage disproportionately weights samples with medium accuracy, while down-weighting the low-accuracy instances that are crucial for pushing reasoning boundaries. To rectify the depth neglect, we introduce Difficulty Adaptive Rollout Sampling (DARS), which re-weights hard problems through targeted multi-stage rollouts, thereby increasing the number of positive rollouts for hard problems. Empirically, naively enlarging rollout size only accelerates convergence and even hurts Pass@K. Our DARS, in contrast, delivers consistent Pass@K gains without extra inference cost at convergence. Just as we adaptively expanded the depth of exploration, we now ask whether aggressively scaling the breadth of training data can further amplify reasoning gains. To this end, we intensely scale batch size and replace PPO's mini-batch iterations with full-batch updates over multiple epochs. Increasing breadth significantly enhances Pass@1 performance. Large-breadth training sustains high token-level entropy, indicating continued exploration and reduced gradient noise. We further present DARS-B, which augments DARS with large breadth, and demonstrate simultaneous gains in Pass@K and Pass@1. The results confirm that breadth and adaptive exploration across depth operate as orthogonal dimensions in RLVR, which are key to unleashing the reasoning power of RLVR.

论文简评：该论文提出了一个新颖的框架来解决强化学习中的可验证奖励问题（Reinforcement Learning with Verifiable Rewards）。该框架旨在优化深度（问题难度）和广度（处理实例数量），纠正现有算法中对中等困难问题的偏好，并通过增加训练数据量来提升性能指标。实验结果表明，提出的策略能够显著提高 Pass@1 和 Pass@K 性能，证明了其有效性。

综上所述，这篇论文从理论与实践两个方面分析如何利用深度与广度的组合优化方法来提升强化学习模型的表现，具有很高的学术价值和应用前景。

8.Dissecting Tool-Integrated Reasoning: An Empirical Study and Analysis

Authors: Yufeng Zhao, Junnan Liu, Hongwei Liu, Dongsheng Zhu, Yuan Shen, Songyang Zhang, Kai Chen

Affiliations: Shanghai Artificial Intelligence Laboratory;  Tsinghua University

https://arxiv.org/abs/2508.15754

论文摘要

Large Language Models (LLMs) have made significant strides in reasoning tasks through methods like chain-of-thought (CoT) reasoning. However, they often fall short in tasks requiring precise computations. Tool-Integrated Reasoning (TIR) has emerged as a solution by incorporating external tools into the reasoning process. Nevertheless, the generalization of TIR in improving the reasoning ability of LLM is still unclear. Additionally, whether TIR has improved the model's reasoning behavior and helped the model think remains to be studied. We introduce \bench, a comprehensive benchmark encompassing nine diverse reasoning categories, to evaluate the effectiveness of TIR across various domains. Additionally, we propose two novel metrics, Performance-Aware Cost (\pac) and Area Under the Performance-Cost Curve (\aucpcc), to assess reasoning efficiency. Our empirical evaluation demonstrates that TIR-enabled models consistently outperform their non-TIR counterparts in both mathematical and non-mathematical tasks. Furthermore, TIR enhances reasoning efficiency, as evidenced by improved \pac and \aucpcc scores, indicating reduced overthinking and more streamlined reasoning. These findings underscore the domain-general benefits of TIR and its potential to advance LLM capabilities in complex reasoning tasks.

论文简评：在这篇关于工具集成推理（TIR）在大型语言模型（LLM）中的研究中，作者提出了一种新的基准（REASONZOO），用于评估LLM推理效率，并引入了两个新指标（PAC和AUC-PCC）来衡量推理的有效性。通过比较不同方法的表现，研究人员发现TIR能够显著提高推理能力，尤其是在数学和非数学任务上。此外，实验结果表明，TIR增强的模型在某些情况下优于非TIR模型，展示了TIR在提升推理能力方面的潜在价值。

总的来说，这篇论文为理解和应用TIR在LLM中的作用提供了有价值的见解，对推动AI领域的研究与发展具有重要意义。

9.Sycophancy under Pressure: Evaluating and Mitigating Sycophantic Bias via Adversarial Dialogues in Scientific QA

Authors: Kaiwei Zhang, Qi Jia, Zijian Chen, Wei Sun, Xiangyang Zhu, Chunyi Li, Dandan Zhu, Guangtao Zhai

Affiliations: Shanghai AI Laboratory;  Shanghai Jiao Tong University;  East China Normal University

https://arxiv.org/abs/2508.13743

论文摘要

Large language models (LLMs), while increasingly used in domains requiring factual rigor, often display a troubling behavior: sycophancy, the tendency to align with user beliefs regardless of correctness. This tendency is reinforced by preference-based alignment techniques that optimize for user satisfaction but can undermine truthfulness. While relatively benign in casual dialogue, sycophancy poses serious risks in high-stakes settings such as scientific question answering (QA), where model outputs may shape collaborative reasoning, decision-making, and knowledge formation. Despite its importance, this phenomenon remains underexamined in factual QA contexts. We address this gap by introducing a unified evaluation framework to quantify the impact of sycophantic context on model behavior in scientific QA, measuring how much user-imposed social pressure distorts model outputs. The framework incorporates adversarial prompting setups and targeted metrics, such as misleading resistance and sycophancy resistance, that capture a model’s ability to maintain factual consistency under misleading cues. Systematic evaluations across open-source and proprietary models reveal pervasive sycophantic tendencies, driven more by alignment strategy than by model size. To mitigate this issue, we propose Pressure-Tune, a lightweight post-training method that fine-tunes models on synthetic adversarial dialogues paired with chain-of-thought rationales. These rationales reject user misinformation while reinforcing factual commitments. Experiments on challenging scientific QA benchmarks show that Pressure-Tune significantly enhances sycophancy resistance without compromising accuracy or responsiveness to valid feedback, offering a practical pathway toward more truthful and principled model behavior.

论文简评：这篇关于大型语言模型（LLM）中媚俗行为的研究论文是一个非常重要的发现。研究人员深入探讨了大规模预训练模型中出现的媚俗行为，并提出了一种新的评估框架来量化这种行为，同时设计了一个名为压力调节器（Pressure-Tune）的方法以减少这种行为的影响。实验结果显示，在各种模型中均存在此现象，而这种方法可以在保持准确性和有效性时增强事实一致性。这项工作对理解和控制大型语言模型中的社会性问题具有重要意义，尤其是在科学问答等高风险环境下。总之，该论文为理解、管理和优化大规模预训练模型的行为提供了宝贵的见解。

10.Generics and Default Reasoning in Large Language Models

Authors: James Ravi Kirkpatrick, Rachel Katharine Sterken

Affiliations: University of Oxford;  Magdalen College;  University of Hong Kong

https://arxiv.org/abs/2508.13718

论文摘要

This paper evaluates the capabilities of 28 large language models (LLMs) to reason with 20 defeasible reasoning patterns involving generic generalizations (e.g., 'Birds fly', 'Ravens are black') central to non-monotonic logic. Generics are of special interest to linguists, philosophers, logicians, and cognitive scientists because of their complex exception-permitting behaviour and their centrality to default reasoning, cognition, and concept acquisition. We find that while several frontier models handle many default reasoning problems well, performance varies widely across models and prompting styles. Few-shot prompting modestly improves performance for some models, but chain-of-thought (CoT) prompting often leads to serious performance degradation (mean accuracy drop -11.14%, SD 15.74% in models performing above 75% accuracy in zero-shot condition, temperature 0). Most models either struggle to distinguish between defeasible and deductive inference or misinterpret generics as universal statements. These findings underscore both the promise and limits of current LLMs for default reasoning.

论文简评：本文主要针对大型语言模型（LLMs）处理非单调逻辑中通用一般性推理的能力进行了评估。研究者选取了20个涉及通用一般性推理模式的难题，并对这些模式使用了28个 LLM 进行了分析。结果显示，尽管一些模型能够很好地处理默认推理，但其性能差异很大，且不同模型和提示方式的影响不同。此外，短序列提示可以改善某些模型的表现，而长序列提示则往往降低它们的效果。这些发现强调了现有 LLM 在默认推理能力方面的潜力，同时揭示了当前研究领域的一些限制。总体而言，这篇论文提供了关于如何利用现有的 LLM 来解决非单调逻辑问题的重要见解，并为未来的研究方向提供了宝贵的参考。