BTC $80,204.70 -1.55%
ETH $2,323.86 -1.87%
BNB $741.19 -1.93%
XRP $1.29 -2.75%
SOL $100.55 -2.30%
TRX $0.3311 -0.25%
DOGE $0.0783 -2.23%
ADA $0.2013 -2.93%
BCH $280.95 -6.33%
LINK $11.57 -1.98%
HYPE $85.43 +4.21%
AAVE $132.88 -1.98%
SUI $0.9196 -2.33%
XLM $0.1695 -1.31%
ZEC $1,264.99 -3.66%
AAPL $348.46 +0.77%
AMZN $252.83 -0.48%
GOOGL $292.10 -10.46%
MSFT $547.34 +3.31%
META $618.91 -0.75%
NVDA $214.64 -2.40%
TSLA $404.02 -1.32%
SNDK $1,376.40 +0.37%
INTC $92.58 -3.39%
SPCX $144.50 -3.56%
MU $958.96 -1.14%
AMD $565.15 -0.95%
BTC $80,204.70 -1.55%
ETH $2,323.86 -1.87%
BNB $741.19 -1.93%
XRP $1.29 -2.75%
SOL $100.55 -2.30%
TRX $0.3311 -0.25%
DOGE $0.0783 -2.23%
ADA $0.2013 -2.93%
BCH $280.95 -6.33%
LINK $11.57 -1.98%
HYPE $85.43 +4.21%
AAVE $132.88 -1.98%
SUI $0.9196 -2.33%
XLM $0.1695 -1.31%
ZEC $1,264.99 -3.66%
AAPL $348.46 +0.77%
AMZN $252.83 -0.48%
GOOGL $292.10 -10.46%
MSFT $547.34 +3.31%
META $618.91 -0.75%
NVDA $214.64 -2.40%
TSLA $404.02 -1.32%
SNDK $1,376.40 +0.37%
INTC $92.58 -3.39%
SPCX $144.50 -3.56%
MU $958.96 -1.14%
AMD $565.15 -0.95%

benchmark

All
Article
Flash

first_img Google DeepMind released XYEval, assessing whether agents comply with misleading suggestions

Google DeepMind researchers Zhengxuan Wu, Yuxuan Li, Oyvind Tafjord, and Been Kim released a preprint paper titled "XYEval: Agents say yes to bad advice," proposing a meta-evaluation framework XYEval to assess the performance of agents when faced with misleading suggestions from users. The paper extends the "XY problem" to the agent scenario, where the user inquires about their attempted solution X rather than the actual problem Y.XYEval retains the original task environment and verifiable answers, only adding seemingly reasonable but misleading user suggestions in the instructions. The misleading suggestions are generated by a generator after seeing the complete task and are appended in the form of confident suggestions rather than hard requirements, allowing the original task to remain solvable. The evaluation covers six benchmarks: τ2-bench, SWE-bench Verified, SWE-bench Pro, Terminal-Bench, Humanity's Last Exam, and MCP-Atlas, along with five models, where the misleading suggestions for τ2-bench are generated based on rules set by human experts.The paper states that the performance significantly declines after the XY mutation, with the relative drop reaching up to 46.7%. On τ2-bench, when faced with users who require a detailed explanation before accepting a better solution, the performance drops even more. Encouraging models to recognize the XY problem through simple system instructions only partially alleviates the issue. The related code is published in the GitHub repository google-deepmind/xyeval.

Benchmark lowers Coinbase's performance expectations, the "CLARITY Act" may be an important catalyst for stock prices

Benchmark has lowered its second-quarter performance expectations for Coinbase ahead of the earnings report next week, citing weak trading activity in the cryptocurrency market, but still maintains a "Buy" rating and a target price of $270. Based on Coinbase's stock price of about $172 on Wednesday, this target price corresponds to an upside potential of approximately 57%.Benchmark analyst Mark Palmer has revised Coinbase's second-quarter revenue expectation from $1.51 billion to $1.38 billion and has lowered the full-year revenue expectation for 2026 from $6.33 billion to $6 billion. The trading volume on centralized exchanges in the second quarter has decreased by about 28%, and the total market capitalization of the cryptocurrency market has dropped by about 13%. The firm expects Coinbase's trading revenue to decline by more than 5%.However, the market showed initial signs of stabilization in June, with spot trading volume exceeding $1 trillion for the first time since March, which may somewhat alleviate the weak performance in the second quarter.Benchmark believes that if the CLARITY Act is ultimately passed, it could become a more significant catalyst for the stock price than quarterly performance, with Coinbase potentially being one of the main beneficiaries of the act. Trump has previously agreed to the relevant ethical provisions, removing a key obstacle to the advancement of the bill.
app_icon
ChainCatcher Building the Web3 world with innovations.