Today for AI
HOT RADAR
llmHEAT 0.0°

AI CURATED HOT STORY · 10/25/2026

DeepMind Uncovers Hidden Bugs in Softmax and Cross-Entropy Implementations

1 reports archived0 independent sourcesupdated 10/25/2026, 13:28:00
Synthesis & Latest Updates

Google DeepMind researchers identified numerical precision bugs in standard Softmax and Cross-Entropy implementations across major frameworks like PyTorch and JAX, which can cause training instability. They proposed a fix involving computational adjustments without increasing floating-point precision, with Gemini playing a key role in discovering the issue and validating experiments.

LATEST/Google DeepMind researchers identified numerical precision bugs in standard Softmax and Cross-Entropy implementations across major frameworks like PyTorch and JAX, which can cause training instability. They proposed a fix involving computational adjustments without increasing floating-point precision, with Gemini playing a key role in discovering the issue and validating experiments.

Hourly heat curve

92 fully observed hours
Now
0.0
Peak
9.922:00
24h change
–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

Coverage timeline

Total 1 reports · Latest first
  1. 机器之心 (微信公众号)T2·85 pts

    DeepMind Uncovers Hidden Bugs in Softmax and Cross-Entropy Implementations

    • Standard Softmax and Cross-Entropy implementations in PyTorch, JAX, and TensorFlow contain hidden numerical precision bugs
    • These bugs can alter gradients during specific training phases, causing loss spikes or capability degradation
    • The issue is fixable by adjusting computation methods without increasing floating-point precision or redesigning loss functions
    • Gemini played a crucial role in discovering the bug and running extensive validation experiments