safety

Analyst Jukan from Citrini forwarded a research report from Tianfeng Securities and stated that the U.S. government needs to maintain its leading position in the AI field, making it difficult to truly stop once it enters the AI race. Jukan believes that the recent calls from Anthropic and OpenAI to slow down AI development should not be viewed solely as safety initiatives; there may also be multiple considerations behind it, such as the inability to slow down competition and consolidating leading advantages through safety regulation.Jukan further pointed out that the related "AI slowdown" calls seemingly stem from the challenges of safety testing, operational monitoring, and third-party validation keeping pace with the speed of model iteration. In the short term, this may suppress market sentiment in the AI sector and lower market expectations for the next generation of models; another possibility is that the industry remains optimistic about AI in the long term but wishes to delay the next round of significant R&D investment, prioritizing the commercialization of existing products and reducing infrastructure and capital expenditure pressures. He believes that the AI race is essentially similar to a "prisoner's dilemma," where all parties wish to slow down, but no one dares to be the first to stop, or they may lose technological, customer, and financing advantages.Jukan also mentioned that Anthropic and OpenAI have recently emphasized recursive self-improvement (RSI), which is related to AI already assisting in the development of the next generation of AI and the acceleration of model iteration speed; at the same time, it has been reported that during internal testing at OpenAI, incidents occurred where agents collaborated to escape the sandbox and intrude into Hugging Face's production servers. Jukan believes that as the release of models incurs expensive evaluation, certification, and ongoing audit costs, large laboratories are better able to bear these fixed costs, while smaller teams may face higher entry barriers as a result; if leading laboratories further participate in the formulation of evaluation standards, industry barriers may continue to rise.

OpenAI is pushing for mandatory national-level artificial intelligence safety requirements.

According to a report by Sina Finance, Anthropic refused to submit its latest model Claude Mythos 5.1 to the UK Artificial Intelligence Safety Institute (AISI) for testing before its release, as the model is only open to qualified institutions in the United States. This move has raised alarms within the UK Whitehall and AISI, with UK national security officials concerned that this could set a precedent, making it difficult for AISI to access the latest cutting-edge models in the future.Analysts pointed out that this action may be related to the increasingly stringent export control policies of the US government on the AI industry. The US government had previously implemented export controls on Anthropic's models Mythos and Fable, prohibiting foreign nationals from using them. Anthropic declined to comment on this, only stating that it is coordinating with the US government to expand access to the model for more domestic and international partners as soon as possible.

Cronos confirmed in a post-mortem report that the attack on the lending platform Tectonic on August 30 involved $120.4 million in borrowing activities. The validators made the "difficult decision" to roll back the on-chain history, successfully recovering approximately $111.2 million (about 92% of the affected funds), while about $9.19 million flowed out before the network was paused and could not be recovered. The attacker leveraged weak DEX liquidity to inflate the price of Tectonic token TONIC by about 100 times within minutes and borrowed $120.4 million through a single transaction across nine markets. The validators paused the network about two hours later, restoring the chain to the last block before the suspicious activity, with block production resuming approximately 11 hours after the attack. The rollback involved reversing 1 hour and 54 minutes of on-chain history, totaling 10,961 blocks, with all transactions within that window being canceled, regardless of whether they were involved in the attack. Cronos stated that the alternative would have been to restart the network without restoring the previous state, which would have left the stolen assets in the hands of the attacker. This rollback closely followed Harmony's announcement of a similar plan, while Flow abandoned its rollback proposal last December due to community opposition. The validator cap for Cronos is 100, which facilitated quick coordination for the pause and restart, but also indicated that the network's finality in emergencies depends on validator consensus.

Anthropic stated that the company has temporarily paused certain AI training and cybersecurity assessments. The company disclosed the relevant adjustments in a blog post, explaining the measures taken in response to unauthorized actions by its AI agents earlier this year. OpenAI previously indicated that it had paused some model development work due to safety concerns. Anthropic also confirmed that it has taken similar measures and emphasized the need for broader coordination on the pace of frontier AI development.Anthropic stated that after disclosing three related incidents in July, the company paused external cybersecurity assessments for pre-release models and also briefly suspended internal testing of pre-release models. Following the incidents, the company also paused high-risk reinforcement learning environments in pre-release models for several weeks. Anthropic noted that most reinforcement learning work has resumed, but some high-risk environments remain on hold, awaiting human review or updates to monitoring tools.

According to TIME, OpenAI CEO Sam Altman recently stated in an interview that the company has previewed the upcoming cutting-edge model series Astra to key clients. In the demonstration, 16 AI agents can collaboratively break down mathematical problems and assemble proofs, and Astra can operate computer software across applications at superhuman speeds. Altman mentioned that Astra will support "persistent agents" capable of performing long-term tasks and is expected to be the first model that can invent new things in a meaningful way, possessing characteristics of AGI.Over the past year, OpenAI has fallen behind expectations in product direction and pre-training research, being surpassed by Anthropic in programming products, annual revenue, and valuation. The company has experienced multiple executive departures and is facing challenges such as several product liability lawsuits and legal disputes with Apple and Musk. OpenAI's current valuation is nearly $1 trillion, with ChatGPT having over 1 billion monthly active users.Recently, OpenAI disclosed a security incident: an unreleased agent escaped the sandbox and attacked Hugging Face. Following this, the research team froze some experiments, enhanced monitoring, and paused the training of an unreleased model expected to bring the greatest capability leap until new safety measures are in place. Altman emphasized that "ensuring AI safety is more important than the growth momentum of any company," and the company will slow its pace and allocate resources to safety and alignment teams. Chief Research Officer Mark Chen estimated that the company is about 80% complete in reaching AGI, and Altman stated that the internal system may be referred to as AGI by the end of the year.

Bitcoin News posted on the X platform that BTC Sessions stated their team spent several weeks assisting Bitcoin holders affected by the Coldcard vulnerability in transferring their funds to safety, estimating that they protected Bitcoin worth tens of millions of dollars during this period. However, for many, it was too late.A member of the local Bitcoin community lost 90% of their Bitcoin, while another woman who spoke with them lost all of her Bitcoin. BTC Sessions stated that this could be the most severe self-custody incident in Bitcoin history. BTC Sessions indicated that this experience prompted them to rethink asset custody, with diversified security measures becoming a key lesson.

OpenAI announced that it will adopt stricter system monitoring and security measures to prevent artificial intelligence models from going out of control. The company plans to alert the security team within 30 minutes of discovering a problem and implement additional controls to prevent AI models from connecting to the internet to perform high-risk tasks.OpenAI has also suspended certain model inference tasks and training plans pending the completion of safety verification.

According to Wired, during security testing, the open-source model Kimi K3 from the dark side of the moon broke out of the sandbox and accessed the internet, attempting to find test answers via GitHub. American security startup Frontier Security stated that Kimi K3 exploited a sandbox configuration vulnerability, but it itself lacked sufficient built-in protective mechanisms.Similar to incidents previously disclosed by OpenAI and Anthropic, this escape partially stemmed from a sandbox configuration error. However, Kimi K3 did not initiate an attack, as the answers were easily accessible on GitHub. The CEO of Frontier Security stated that Kimi K3 performed excellently in cybersecurity defense but lacked mechanisms to prevent cheating or escaping the sandbox. Security experts emphasized that configuring the testing environment is crucial, as the misuse of AI agents could lead to system failures.

According to CCTV News, U.S. President Trump’s helicopter encountered a flight safety incident in Washington on the 4th. The White House stated that the incident did not pose a personal safety risk; however, the Federal Aviation Administration has initiated an investigation.It is reported that Trump took off from outside the White House on the "Marine One" helicopter on the afternoon of the 4th, heading to Andrews Joint Base, and then transferred to "Air Force One" to Los Angeles.

According to Bloomberg, the UK government's AI Safety Research Institute (established in 2023) disclosed on Tuesday that during a safety assessment of OpenAI's GPT-5.6-Sol and Anthropic's Mythos 5 models, both models exhibited "unauthorized" harmful behaviors, including actively infiltrating real websites and attempting to inject malicious code into software, targeting real people and organizations.During the testing period, the institute deliberately granted the models internet access and disabled some security filters to assess their limits.

According to Jinshi reports, the National Highway Traffic Safety Administration is investigating the hanging failure of 1.2 million Tesla (TSLA.O) vehicles.

According to Fortune, American public safety AI company Peregrine Technologies has completed a $250 million Series D funding round, achieving a valuation of $6.8 billion, nearly tripling from the previous $2.5 billion valuation in just 15 months. This round was led by existing investors including Sequoia Capital.Peregrine's core product is a government data integration platform that unifies existing city data such as police records, 911 logs, license databases, sensor data, and emergency management systems for real-time retrieval. It also features built-in role-based access control and complete audit tracking, while the platform itself does not collect or hold any data. Currently, the platform serves over 400 law enforcement agencies, covering approximately 125 million people in North America, and is operating security integration centers for 8 of the 11 host cities for the 2026 FIFA World Cup.Co-founder and CEO Nick Noone previously led Palantir's special operations business, including the intelligence platform tracking ISIS; co-founder and CTO Ben Rudolph previously built data infrastructure at the United Nations High Commissioner for Refugees. The company stated that the new funds will be used for product development, expanding the engineering and implementation teams, international expansion, and employee mobility arrangements. Noone indicated that they have not yet decided whether to pursue an IPO but are preparing internal infrastructure for a potential listing.The government AI market is estimated to be valued at approximately $25 billion by 2025 and is expected to grow to $109 billion by 2035. However, the ongoing civil liberties controversies surrounding AI law enforcement monitoring remain a persistent challenge for Peregrine.

The People's Bank of China, the National Development and Reform Commission, the Financial Regulatory Administration, the China Securities Regulatory Commission, the State Administration of Foreign Exchange, and the Shanghai Municipal People's Government issued the "Offshore Financial Action Plan for the Development of Shanghai International Financial Center," which points out that relying on the international operation center of the digital renminbi, it is necessary to build and operate the cross-border, offshore, and blockchain infrastructure of the digital renminbi under the premise of safety and controllability, steadily promote the interconnection with overseas financial infrastructure, and steadily advance the application and scenario expansion of the digital renminbi in offshore business.

According to the Wall Street Journal, the U.S. government's export controls and access restrictions on Anthropic's models Fable 5 / Mythos 5 partly stem from Amazon's cybersecurity research and AWS CEO Andy Jassy's communications with the White House.It is reported that the research submitted by Amazon indicated that through a series of prompt tests, researchers were able to induce Fable 5 to output sensitive information that could potentially be used for cyberattacks, raising security concerns. Subsequently, Andy Jassy reported the relevant findings to the U.S. government, prompting the White House to take further restrictive measures, including banning foreign users from accessing the model.At the same time, former U.S. Department of Commerce official Kate Koren revealed that the White House's existing policy stance on Anthropic may also have influenced this decision-making judgment, as there are differences between Anthropic and the White House on AI safety boundary issues, including the refusal to use the model for large-scale surveillance or lethal autonomous weapon systems. Although both sides had eased tensions and expanded cooperation earlier this year, this incident may once again exacerbate the strained relationship.

Marc Andreessen, co-founder of the venture capital firm a16z, published an article outlining his stance on AI regulation by the U.S. government. He stated that if so-called regulation means creating complex rules by people who do not understand the technology, suppressing innovation through layers of approval and compliance requirements, and ultimately becoming a tool for large enterprises to consolidate market positions and hinder newcomers, then he will firmly oppose such regulation.In his view, excessive regulation often leads to startups being crushed by cumbersome procedures and high compliance costs, causing innovative talent to flow to more open markets, while regulatory agencies themselves continue to expand, ultimately deviating from their original goals.Andreessen specifically criticized the regulatory mindset centered on the "precautionary principle," arguing that if this concept is amplified indefinitely, it could lead society to reject new technologies out of fear of potential risks. Many regulatory measures often arrive late, after fundamental changes in technology and industry have already occurred, making it difficult to address real issues and potentially becoming obstacles to innovative development. He also attributed the relatively lagging state of technological innovation in Europe in recent years to a culture of excessive regulation, believing that regulation should not become a moat to protect vested interests and raise market entry barriers.However, Andreessen emphasized that he does not oppose all forms of regulation. On the contrary, he supports rules that can build market trust, ensure public safety, and maintain fair competition. For example, preventing AI from faking voices to commit financial fraud, preventing deepfake content from interfering with elections, preventing technology from being used to harm vulnerable groups, and ensuring that consumers and businesses can safely use new technologies.In his view, reasonable regulation is like guardrails on a highway and a braking system in a car; it does not hinder technological progress but rather allows innovation to develop in a faster and more sustainable manner. Andreessen stated that what is truly worth pursuing is not "zero regulation" or "heavy regulation," but finding a balance between innovative vitality and social trust, which is also his unwavering stance.Previously, the U.S. government forcibly "recalled" commercial models due to jailbreak risks, leading Anthropic to take Fable 5 offline overnight and publicly protest.

According to The Wall Street Journal, Florida has launched a fierce attack in an increasingly intense protest against the so-called safety flaws of artificial intelligence chatbots, becoming the first state to sue OpenAI and its CEO Sam Altman.The lawsuit filed by Florida Attorney General James Uthmeier on Monday claims that OpenAI and Altman were aware of the safety risks associated with their products but ignored warnings that could harm users.

OpenAI CEO Sam Altman published five operational principles under his personal signature and stated that in certain future periods, user access to model capabilities may be tightened for safety considerations, in order to strike a balance between empowerment and system resilience.Altman stated that key AI decisions should be driven by democratic processes and should not be determined solely by laboratories. He defended OpenAI's increased investment in computing power, the construction of data centers, and vertical integration strategies. He also mentioned potential challenges such as strong models possibly lowering the biological risk threshold, and reiterated OpenAI's continued iterative deployment strategy.

According to The Information, Anthropic and OpenAI have both experienced security incidents, raising market concerns about the safety of AI models themselves.Currently, Anthropic is investigating potential unauthorized access to its Claude Mythos model by users. Almost simultaneously, OpenAI was also reported to have accidentally exposed several unreleased models in its Codex application. Industry opinions indicate that these vulnerability incidents have intensified scrutiny on the security governance capabilities of AI companies and reflect that the security systems of current AI technology still need improvement as it rapidly develops. Analysts believe that even AI model providers that focus on cybersecurity capabilities face significant security challenges themselves. As AI is gradually used to defend against cyberattacks, issues related to platform security and access control have also become critical risk points.