论文标题
AGI因违反正交论文而修改其效用功能
An AGI Modifying Its Utility Function in Violation of the Orthogonality Thesis
论文作者
论文摘要
人工通用情报（AGI）可能具有工具性的驱动力来修改其效用功能，以提高其合作，讨价还价，承诺，威胁和抵抗和参与勒索的能力。这样的AGI必然具有至少部分可观察到的效用函数，并且受到其他代理选择如何与之相互作用的影响。这种工具驱动力将与正交性论文相抵触，因为修改将受到AGI智力的影响。在高度竞争的环境中，AGI可能会融合到具有几乎相同的效用功能，一个通过游戏理论优化了以有利地影响其他代理。
An artificial general intelligence (AGI) might have an instrumental drive to modify its utility function to improve its ability to cooperate, bargain, promise, threaten, and resist and engage in blackmail. Such an AGI would necessarily have a utility function that was at least partially observable and that was influenced by how other agents chose to interact with it. This instrumental drive would conflict with the orthogonality thesis since the modifications would be influenced by the AGI's intelligence. AGIs in highly competitive environments might converge to having nearly the same utility function, one optimized to favorably influencing other agents through game theory.
