Two tables both contain user_id; why do they produce different active-user counts? Enterprise analytics often fails before SQL generation: the system chose the wrong table, timestamp, filter, or definition.
OpenAI’s January 2026 account of its internal data agent centers the design on context engineering. The article’s snapshot described more than 3,500 internal users, 600 PB of data, and roughly 70,000 datasets. The more transferable lesson is how the system narrows that search space.
Six sources are not interchangeable knowledge
| Source | Contribution | Failure mode |
|---|---|---|
| Table usage | Schema, lineage, prior queries | Popular does not mean correct |
| Human annotations | Meaning, known issues, owner knowledge | Can be stale or local |
| Pipeline code | Derivations, filters, freshness clues | Needs runtime interpretation |
| Institutional knowledge | Documents, discussions, decisions | Authority and time vary |
| Runtime feedback | Results, errors, distributions | Describes one execution |
| Memory and corrections | Validated fixes and failure lessons | Must be scoped and versioned |
These materials should not become one enormous prompt. The system retrieves candidate assets, narrows them by permission, time, grain, and relevance, and retains evidence for why a context package was selected.
Query generation sits in the middle
A fuller loop is: understand intent; retrieve metadata and institutional knowledge; select candidate tables and trusted queries; generate and execute; adapt to errors and results; validate independently; record reusable corrections.
Runtime feedback matters. An empty result can indicate a bad filter or a real business event. Successful SQL proves only that the engine accepted the program—not that the business meaning is correct.
Memory needs an applicability boundary
“Use activated_at instead of created_at” is not a global rule. A reliable correction binds to a domain, metric, table version, time range, and validation record. Otherwise yesterday’s fix becomes tomorrow’s error.
For an enterprise pilot, start with a bounded domain whose definitions and owners are known and whose failure cost is manageable. Build searchable asset descriptions, trusted queries, runtime authorization, and a regression task set. Review corrections before promoting them into durable memory.
This internal case is not an off-the-shelf product specification. Its central lesson is that, at enterprise scale, the bottleneck moves from generating SQL to finding, constraining, and validating the right context.
Reference: Inside OpenAI’s in-house data agent