Meta details migrating their petabyte-scale data ingestion system from legacy customer-owned pipelines to a self-managed warehouse service. 100% of workloads transitioned, improving reliability and efficiency. Key strategies included automated correctness verification via row count and checksum comparisons.
SAP and Snowflake announced GA of joint offerings for bidirectional zero-copy integration, unifying SAP business data with the Snowflake AI Data Cloud without duplication. This eliminates complex ETL and provides near real-time access to semantically modeled SAP data for AI and analytics.
Octopus Energy re-architected its margin data pipelines to handle the UK's 48x data surge from Market-wide Half-Hourly Settlement (MHHS). By splitting a monolithic pipeline into three specialized streams, a team of three engineers cut costs 50x, saving $1M/year, while scaling for half-hourly granularity.
Smartsheet built a Real-time Dynamic Filtering system on Amazon Managed Service for Apache Flink, cutting messaging costs by $40K/month and improving live collaboration latency by 1.8x. The solution dynamically filters hundreds of thousands of events per second, eliminating wasteful over-consumption where 90% of events were discarded.
Our World in Data analyzes the 4.7 million annual child deaths under five through interactive visualizations. Infectious diseases and birth complications dominate globally, with stark differences between low- and high-income countries, highlighting that most of these deaths are preventable.
Disney has removed the FiveThirtyEight archive, redirecting all old links to ABC News. Founder Nate Silver called it "200,000 hours of work erased," noting Disney told him to "get lost" when he tried to acquire the IP. The data journalism site's legacy now lives on only through the Internet Archive and Silver Bulletin.
Our World in Data released an interactive simulation that lets anyone adjust fertility, life expectancy, and migration to model population changes through 2100. The browser-based tool visualizes how different demographic assumptions reshape a country's future size and age structure.
Our World in Data built an interactive population simulation tool exploring how much fertility, life expectancy, and migration must change to halt South Korea's projected decline from 52M to 22M by 2100. The analysis reveals that even dramatic shifts in a single variable would barely slow the reduction.
Apache Iceberg 1.11.0 introduces a new File Format API enabling pluggable formats like Vortex and Lance without engine internals changes. It also brings production maturity to V3 spec features: deletion vectors, the Variant type, native geospatial support, and nanosecond-precision timestamps.
Apache Iceberg V3 adds deletion vectors, Variant types, geospatial support, and nanosecond precision over V2. These features make row-level operations more efficient and expand native data types, moving beyond V2's position-delete limitations for modern lakehouse workloads.
MinIO co-founders discuss eliminating AI storage bottlenecks that leave GPUs underutilized, highlighting their NVIDIA partnership on the new STX reference architecture and why modern AI infrastructure is converging on S3-compatible object storage.
A May 2026 AWS thermal event in Northern Virginia knocked Coinbase, FanDuel, and CME offline for hours, revealing that multi-AZ HA alone is insufficient. SingleStore argues that cross-region disaster-recovery architecture with low RPO targets is the only hedge against physical-infrastructure failures.
Snowflake unveiled multimodal analytics capabilities via Cortex AI, enabling organizations to extract structured intelligence from video, audio, and images using SQL queries. The platform supports brand safety monitoring, deepfake detection, and sports analytics by decoding visual and audio signals alongside traditional data.
The unit distance conjecture, open for 80 years, has been disproven by an OpenAI reasoning model. The proof provides an infinite family of examples improving on the square-grid construction, marking the first time AI has autonomously solved a prominent open problem central to a mathematical subfield.
Google's ERA uses Gemini to write and optimize expert-level scientific code, described in a new Nature paper. It helped build Computational Discovery, now rolling out via Gemini for Science. ERA searches literature and iterates over thousands of code options using tree search across genomics, public health, and more.
NVIDIA released Nemotron-Labs Diffusion, a family of diffusion language models (3B, 14B) that generate multiple tokens in parallel rather than one at a time, enabling faster, editable text generation. The models support three modes, autoregressive, diffusion, and hybrid, and are open-source under the Nemotron license.
IBM released Granite Embedding Multilingual R2, two Apache 2.0 multilingual embedding models built on ModernBERT. The compact 97M model beats all sub-100M multilingual embedders on MTEB retrieval (60.3), while the 311M model scores 65.2 (#2 among sub-500M open models). Both support 200+ languages, 32K context (64x R1), and include code retrieval across 9 programming languages.
Airbyte launched its Agent MCP in the ChatGPT app marketplace, giving ChatGPT live access to 50+ business data sources through a single connection. Its Context Store continuously replicates and indexes data for fast, cross-source search queries within ChatGPT.
Allen AI released OlmoEarth v1.1, a family of Earth observation models that cuts compute costs by up to 3x while maintaining v1's performance. The efficiency gains come from optimizing token sequence lengths in the transformer architecture, making satellite imagery analysis faster and cheaper at scale.
IBM Research launched the Open Agent Leaderboard, an open benchmark for comparing full AI agent systems across diverse tasks. It measures both quality and cost, paired with the Exgentic framework for reproducible evaluations. Everything is open source from day one.
Microsoft Research releases an open dataset of realistic U.S. power grid transmission topology built entirely from public data, spanning 48 states and multi-state interconnections. The models support AC optimal power flow analysis, enabling research into congestion, capacity, and demand siting without restricted data.
You’re caught up.
That was all 21 stories in the 26 May issue.