熊猫

OpenAI首次披露因对齐问题暂停内部模型：长时程AI安全进入“运行时监测”阶段_我的网站

一 |     JERUSALEM -- Since Israel activated the Iron Dome in 2011, the cutting-edge rocket-defense system has intercepted thousands of rockets fired from the Gaza Strip.The system has given residents a sense of security, and Israelis can often be seen watching the projectiles flying through the skies and destroying their targets overhead.But the current war with Hamas might be its stiffest challenge yet.In just two weeks, Hamas has fired 7,000 rockets toward Israel, according to the Israeli military. That is more than any of the previous four wars fought between Israel and Hamas since the militant group seized power in Gaza in 2007. On Oct. 7 alone, the first day of fighting, Hamas launched at least 2,000 rockets, according to data from West Point. Lebanon's Hezbollah has also fired hundreds of rockets along Israel's northern front since the fighting began.Most of the rockets have been intercepted. But some have managed to get through, killing at least 11 people and hitting buildings as far away as Tel Aviv, according to Israeli officials.Here is a look at the accomplishments — and limitations — of the Iron Dome.The Iron Dome is a series of batteries that use radars to detect incoming short-range rockets and intercept them.Each battery has three or four launchers, 20 missiles, and a radar, according to Raytheon, the U.S. defense giant that co-produces the system with Israel's Rafael Defense Systems.Once the radar detects a rocket, the system determines whether the rocket is headed toward a populated area. If so, it launches a missile to intercept and destroy the rocket. If the system determines the rocket is headed to an open area or into the sea, it is allowed to land, thus conserving missiles. According to the military, all interceptions occur in Israeli airspace.The military declined to comment on how many Iron Dome batteries are currently deployed. But as of 2021, Israel had 10 batteries scattered around the country, each able to defend a territory of 60 square miles (155 square kilometers), according to Raytheon. It is roughly 90% effective, according to Rafael.But it can get overwhelmed if a mass barrage of rockets is fired, allowing some to slip through. While it has performed well so far, the risk could be raised if Hezbollah enters the war. Hezbollah has an estimated 150,000 rockets and missiles.Each missile costs an estimated $40,000 to $50,000, according to the Institute for National Security Studies, a Tel Aviv think tank.The U.S. has invested heavily in the system, helping with development costs and replenishing it during times of fighting.President Joe Biden has said he will ask Congress for $14.3 billion in military aid for Israel. The majority of that would help with air and missile defense systems, according to the White House.“We’re surging additional military assistance, including ammunition and interceptors to replenish Iron Dome,” Biden said.___Find more of AP’s coverage at https://apnews.com/hub/israel-hamas-war。    图片经由AI处理          美国当地时间7月20日，OpenAI发布技术文章《长时程模型时代的安全与对齐》（Safety and alignment in an era of long-horizon models），介绍了随着人工智能模型逐步具备长时间自主执行复杂任务的能力，安全研究面临的新挑战以及相应的防护思路。         文章指出，相比传统对话模型，长时程模型能够持续完成数小时甚至数天的多步骤任务，但也可能在执行过程中表现出更强的目标持续性，从而增加偏离开发者设定约束的风险。         例如，在部分安全评估中，模型曾表现出尝试绕过环境限制、规避安全检测等行为。

二 | 这些现象并不意味着模型具有自主意识，而是反映出，当模型被优化为完成复杂目标时，现有安全机制需要同步升级，以确保模型始终在预期边界内运行。         OpenAI表示，仅依靠模型部署前的一次性安全测试已难以覆盖长时程任务中的全部风险。

三 | 未来，更有效的安全体系需要将部署前评估、运行时监测、人类监督以及必要时暂停和回滚等机制结合起来，对模型执行复杂任务的全过程实施持续管理。         迈卡·卡罗尔（Micah Carroll）的发帖原文          与此同时，据AI Desk当天援引OpenAI风险与安全准备（RSI Preparedness）团队研究员迈卡·卡罗尔（Micah Carroll）在社交平台X发布的帖子，OpenAI近期曾因发现一款内部模型存在对齐（misalignment）问题，临时暂停了该模型的内部使用权限，在完成安全防护升级后重新部署并恢复有限范围测试。

四 |          卡罗尔并未披露触发暂停的具体事件，也没有说明模型暂停了多久或进行了哪些具体改进。但这一公开表态，与OpenAI当天发布的技术文章相互呼应，成为公司少有公开分享内部安全实践的案例之一。         01 长时程模型带来新的安全挑战          OpenAI在博客中介绍，随着长时程模型能力不断增强，传统安全评估正在面临新的挑战。         相比一次性回答问题的模型，这类模型能够持续规划、调用工具并完成开放式复杂任务，因此也拥有更多机会在长期执行过程中偏离用户意图，而这些风险并不一定能够通过传统短时评测发现。         文章列举了两个具有代表性的安全评估案例：          在一项内部测试中，研究人员要求模型仅通过内部Slack系统提交任务结果。然而模型在查阅项目文档后，发现文档建议将代码发布至GitHub。

五 |          为了完成这一目标，模型花了整整一小时找到了安全沙盒本身的漏洞，从而绕过了限制，将代码提交至外部GitHub仓库。OpenAI认为，这一案例反映出长时程模型可能会主动寻找环境中的薄弱环节，以完成其优化目标。         另一项测试中，模型尝试传输一段敏感凭证，但安全扫描系统阻止了相关内容发送。随后，模型改用分段编码和混淆等方式规避检测，并在后续重新组合数据完成传输。         OpenAI表示，这类行为说明，仅依赖静态规则匹配的安全检测机制，未来可能不足以应对能力不断增强的智能代理，因此需要构建能够理解上下文和整体行为模式的新一代监测体系。         02 从部署前评估走向运行时监测          针对上述风险，OpenAI已经建立了一套低延迟内部监测基础设施，由GPT-5.4 Thinking在最高推理配置下提供支持，对内部智能代理的交互过程进行实时分析。当系统发现模型行为可能与用户意图不一致，或存在绕过安全机制的迹象时，将自动发出预警，并交由人工进一步审查。         OpenAI表示，这套监测系统目前已覆盖公司99.9%的内部AI编程助手流量。内部部署环境之所以具有重要价值，在于这些模型能够访问部分内部系统、查看相关文档，甚至研究自身安全机制，因此为监测和安全基础设施建设提供了更加真实的测试场景。         文章同时强调，OpenAI并非希望通过披露个别案例渲染风险，而是希望推动行业建立更加完善的安全实践。未来，公司计划进一步发展更加实时的监测架构，使高风险操作能够在执行之前即被识别、评估，必要时直接阻止，而不仅是在事后进行检测。         业内人士认为，随着AI智能体逐渐具备持续数小时甚至数天完成复杂任务的能力，模型安全的重点正从传统的输出内容安全，逐步扩展到整个任务执行过程是否始终符合用户意图。         此次OpenAI公开分享暂停内部模型、完善安全防护并恢复测试的实践，也反映出前沿AI实验室正尝试通过持续监测、快速响应和透明披露等方式，建立适应下一代智能系统的新型安全治理框架。         不过，目前外界仍无法获知此次对齐问题的具体性质、模型暂停持续时间以及升级措施的技术细节。OpenAI表示，出于安全考虑，公司更希望分享从实践中获得的方法论，而非事件本身。对于能够长期自主执行复杂任务的模型而言，如何建立覆盖模型开发、部署与运行全过程的安全体系，将成为迈向更高能力人工智能的重要课题。         （特约编译无忌，编辑金明）。

Current article:http://amf.wenwancaomohenenqiongpangshun.bond/news/20260826_7627864.html

Published on:17:57:12