LodeHQSubscribe →

Comet cuts latency, Microsoft's Ontology, AI agents need context

Data · 2026-07-20

Data Engineering
Comet cuts Spark‑Iceberg query latency and fuels Rust development9 MIN

Apache DataFusion’s Comet layer runs Spark reads on Iceberg tables through the native Rust engine, keeping Java planning but executing with Arrow. In a 3 TB TPC‑DS benchmark it sped up 102 of 103 queries and gave Iceberg Rust a massive differential‑testing harness, catching bugs that flow back to Java.

Microsoft releases Ontology Playground: no‑code visual builder for Fabric IQ7 MIN

Microsoft’s Ontology Playground is a free, open‑source web app that lets users visually design, import, and export RDF/OWL ontologies specifically for Microsoft Fabric IQ without any backend infrastructure. By providing a drag‑and‑drop editor, a community catalogue, and one‑click embedding, it speeds semantic modeling and makes Fabric‑IQ data pipelines accessible to analysts and developers alike.

LinkedIn, Walmart, Zendesk rebuild stack for sub‑second AI agent response4 MIN

Enterprise AI agents were hitting latency walls not from model quality but from legacy stacks built for human workflows. LinkedIn shifted 80% of orchestration to async, pre‑provisioned containers; Walmart created a unified data‑access layer to curb duplication; Zendesk rewired its data plane for sub‑second replies. The fixes let agents move from pilot to production at scale.

Analytics & Visualization
Global Mycorrhizal Web Mapped: 110 Quadrillion Kilometers Underground1 MIN

Researchers from SPUN and data‑visualizer Moritz Stefaner visualized the hidden mycorrhizal network that links plants worldwide. Their map suggests the fungal infrastructure spans roughly 110 quadrillion kilometres within the top ten centimetres of soil, acting as a subterranean circulatory system for carbon, water and nutrients.

AI uses 0.5% of world electricity, roughly Germany’s annual output10 MIN

Data centers account for about 1.5% of global electricity, and AI workloads make up roughly a third of that, or 0.5% of total power, about the same as Germany’s yearly consumption. The demand is tightly clustered in a few regions, meaning local grids could feel outsized pressure as AI expands.

ML & AI for Data
Enterprise AI agents need governed data-product context, not just more RAG14 MIN

Retrieval can fetch relevant passages, but it strips source boundaries, ownership, and usage rules. Byakod proposes compiling a governed data‑product context, graphs, markdown wikis, and open knowledge formats, alongside vector search so agents understand business meaning, constraints, and provenance. This shifts AI from raw retrieval to knowledge cataloging.

Airbnb Cuts LLM Evaluation from Weeks to a Day with Smart Caching10 MIN

Airbnb slashed its LLM evaluation loop from weeks to under a day by caching generated references and judge scores. Since over 50% of model outputs repeat verbatim and judge drift stays around 1%, the cache eliminates redundant work, letting engineers iterate quickly and ship reliable improvements.

Scribd’s Human-in-the-Loop LLM Pipeline Classifies Hundreds of Millions of Docs in Two Months7 MIN

The team used a fast LLM to generate candidate labels and rationales, a judge model to flag disagreements, and SMEs to review only the toughest cases, building a reusable human-in-the-loop loop. This workflow let a three‑person crew classify hundreds of millions of Scribd documents in about two months, cutting costs and scaling accuracy across projects.

Netflix Deploys Its Own End-to-End LLM Serving Platform11 MIN

Instead of relying on external APIs, Netflix runs the full LLM stack, deployment, routing, GPU inference and OpenAI‑compatible HTTP endpoint, on a JVM‑based serving system backed by NVIDIA Triton and vLLM. The architecture lets the company keep traffic in‑house, enforce latency and policy constraints, and iterate on custom models at production scale.

Databases & Storage
Meta’s Trellis makes agents’ search histories queryable, slashing cost by 50% while preserving performance37 MIN

Trellis treats an agent’s experience graph, artifacts, rewards, causal lineage, as first‑class database state, exposing SQL, graph, vector, and temporal queries. Meta showed that reusing this structured history matched solution quality while halving token cost and delivering up to ten‑fold speedups, turning agents into stateless, crash‑resilient services.

Postgres 19 Makes LZ4 the New Default for TOAST Compression8 MIN

PostgreSQL 19 will replace the legacy pglz algorithm with LZ4 as the default TOAST compression method. The change keeps the single compression framework across heap tables, TOAST, and B‑tree indexes, giving better compression ratios and faster CPU performance for TEXT, VARCHAR, BYTEA, JSONB, and other variable‑length columns.

Get Data in your inbox, every issue.
Subscribe free
Privacy · Terms · About · Contact
© 2026 LodeHQ