Data Governance

Global AI Recap, 12 September 2026: Standards, Interoperability and the Open-Format Settlement

The week's quiet infrastructure victory was for the buyer: open formats and interoperability became the default expectation, and lock-in became a reason to walk away.

Key Statistics: Dominant theme: buyers rejected proprietary traps, vendors converged on portable table formats and open model interfaces, and 'can I leave' became as important a question as 'can I start'.

Lock-in became a dealbreaker

After years of painful migrations, buyers entered evaluations asking the exit question first: can I export my data, my definitions and my evaluations in an open format? Vendors that hedged the answer lost deals to those that answered plainly. This is a buyer's market correction. The cost of switch used to be hidden; now it is the first line of the RFP.

Open table formats settled

The lakehouse table-format debate effectively ended in favor of open, portable formats. The practical benefit is enormous: the same dataset serves BI, AI and archival without copying, and a change of vendor does not mean a change of data. For mid-market firms, this removes the single biggest fear of committing to a platform: that the platform becomes a trap.

Model interfaces converged

A rough consensus formed around open model interfaces, so swapping one model for another stopped requiring a rewrite of the surrounding system. The model became a swappable component, which is exactly what lets cost and quality competitions reach the buyer. When the model is interchangeable, vendors compete on the system around it — guardrails, evaluation, support — which is healthier than competing on lock-in.

Interoperability as a feature

Vendors began advertising interoperability as a selling point rather than a weakness. 'Bring your own model', 'export everything', 'open by default' moved from roadmap promises to headline features, because buyers voted with budgets. This inversion — openness as marketing — is the clearest sign the war is over and buyers won.

What this means for builds

The firms that architected for portability from day one now move freely: they trial a new model, export an evaluation, and switch without a re-platform. The firms that married a proprietary format are stuck explaining why they cannot leave. Interoperability is no longer a nice-to-have. It is the precondition for not being held hostage.

Key takeaways

  • Buyers now ask the exit question before the start question.
  • Open table formats won; the same data serves BI and AI without copying.
  • Model interfaces converged, making models swappable components.
  • Interoperability became a headline feature, not a weakness.

Where to start

In every AI evaluation, require written confirmation of open export for data, definitions and evaluations, and a swappable model interface. If a vendor cannot commit, treat lock-in as a future cost and weigh it against the discount.

What to watch next

Expect procurement frameworks to standardize an 'exit readiness' criterion, and for open interfaces to become table stakes across the stack. The firms that built portable from the start will trial and switch vendors without re-platforming.

Avoiding the lock-in tax

  • Prefer open table formats so your warehouse and your AI read the same bytes, and a vendor change does not mean a migration project.
  • Standardize on model interfaces that let you swap providers without rewriting the application that depends on them.
  • Treat interoperability as a feature you pay for up front, not a regret you buy later at ten times the cost.
  • Ask every vendor the only question that matters: what leaves cleanly if I walk away next year?
  • Keep your proprietary logic in code you own, and let commodity connections be commodity connections.

Consolidation beat expansion this week because buyers grew tired of stitching ten tools with ten integrations. The durable win is fewer, open, interoperable pieces — the architecture you can leave is the one you can trust, because you are never a hostage to it.

Case Study: FinCorp’s Transition to an Open‑Format AI Lakehouse

FinCorp, a multinational banking organisation with £150 bn in assets under management, embarked in early 2025 on a programme to replace a legacy proprietary data warehouse that had become a lock‑in trap for its AI‑driven risk‑analytics team. The existing stack stored feature tables in a vendor‑specific columnar format, required custom ETL scripts to move data into the modelling environment, and forced the team to rebuild entire pipelines whenever a new model vendor was evaluated. By mid‑2026 the organisation had migrated to an open‑table‑format lakehouse built on Apache Iceberg, adopted a universal model interface based on ONNX‑compatible APIs, and instituted an “export‑first” procurement policy. The following narrative walks through the motivations, technical decisions, measurable outcomes and the lessons that other enterprises can draw from FinCorp’s experience.

Motivation: The Exit Question Became a Gatekeeper

FinCorp’s procurement office began every AI vendor evaluation with the simple question: “If we decide to walk away after twelve months, what can we export without re‑engineering?” The answer from the incumbent warehouse provider was a qualified “yes, but only after a costly data‑migration project.” This response triggered a red flag, and the banking group issued a formal RFP amendment demanding written confirmation of open export for data, definitions and model artefacts. Vendors that could not provide a clear, legally binding statement were eliminated from the shortlist, regardless of discount levels. This shift placed interoperability at the forefront of the buying process, turning a former after‑thought into a decisive criterion.

Technical Approach: Choosing an Open Table Format and a Swappable Model Layer

After a short proof‑of‑concept phase, FinCorp selected Apache Iceberg as the table format for three reasons: (1) it provides true ACID transactions on data lakes, (2) it is supported natively by the major query engines (Trino, Spark, Flink) and (3) it allows time‑travel queries without duplicating storage. The data‑lake was moved from the proprietary warehouse to an S3‑compatible object store, and all historic tables were rewritten into Iceberg format using a bulk‑load job that preserved partitioning and schema evolution metadata.

For the model layer, the organisation adopted the Open Neural Network Exchange (ONNX) runtime as the universal interface. All newly developed models were exported to ONNX version 1.14, and a thin adapter layer was built to translate vendor‑specific APIs (e.g., TensorFlow Serving, Sagemaker Endpoints) into ONNX inference calls. Existing models were wrapped in the same adapter, allowing the risk‑analytics team to swap a credit‑scoring model from Provider A to Provider B with a single configuration change and zero code rewrite.

Results: Quantifiable Gains in Speed, Cost and Risk

Six months after cut‑over, FinCorp reported the following metrics:

  • Data‑movement latency for feature extraction dropped from an average of 45 minutes per batch to under 5 minutes, because the same Iceberg tables could be queried directly by the modelling Spark cluster.
  • Model‑switching time fell from an average of three weeks (including re‑writing of data‑prep scripts) to less than four hours, limited only to the validation of the new ONNX artefact.
  • Annual licensing and support costs associated with the proprietary warehouse decreased by £2.3 million, a 38 % reduction, while the open‑format stack incurred only modest operational overhead for Iceberg maintenance.
  • The risk‑analytics team increased the number of model experiments per quarter from 12 to 27, a 125 % rise, directly attributable to the reduced friction in trying new algorithms.
  • Auditability improved: Iceberg’s snapshot mechanism allowed auditors to reproduce any feature set used in a model decision with a single SQL query, eliminating the need for bespoke data‑lineage documentation.

Lessons Learned: What Other Organisations Should Replicate

FinCorp’s journey highlights three repeatable actions:

  1. Make the export question the first gate in every AI procurement cycle and obtain a legally enforceable commitment.
  2. Standardise on a single, community‑driven table format (Iceberg, Delta Lake or Hudi) and migrate historic assets in a controlled bulk‑load window rather than piecemeal.
  3. Adopt a vendor‑agnostic model interface (ONNX, TorchScript or MLflow Models) and encapsulate all vendor‑specific calls behind a thin adapter layer; this turns models into plug‑and‑play components.

By treating interoperability as a feature to be bought up front, FinCorp turned a potential lock‑in tax into a competitive advantage, enabling rapid model iteration and safeguarding its ability to leave any vendor relationship without a costly re‑platforming effort.

“The real win was not the technology itself, but the shift in mindset: we now buy AI capabilities the way we buy office supplies – with a clear return policy.” – Chief Data Officer, FinCorp

Implementation Playbook: From RFP to Production‑Ready Interoperable AI Stack

Turning the strategic principle of interoperability into a concrete, repeatable process requires a disciplined approach that spans procurement, architecture, data engineering and model operations. The following playbook outlines a step‑by‑step programme that organisations can adopt to ensure that every new AI capability is built on open table formats, swappable model interfaces and verifiable export pathways. Each phase includes concrete artefacts, responsible roles and success criteria.

Phase 0 – Foundations: Governance and Policy

  • Define an Interoperability Policy that mandates open export for data, definitions and model artefacts as a prerequisite for any AI vendor contract.
  • Appoint an Interoperability Champion (usually a senior data‑architect) who owns the policy, maintains a list of approved open formats and audits compliance.
  • Update the organisation’s Architecture Decision Records (ADR) template to include an “Export‑Readiness” section.

Phase 1 – Procurement: RFP Design and Vendor Evaluation

  • Insert a mandatory “Export‑Readiness Questionnaire” into every AI‑related RFP. Sample questions:
    • Can you provide a full dump of all tables, metadata and feature definitions in Apache Iceberg (or Delta Lake) format?
    • Do you support model export to ONNX version 1.14 or higher, and can you guarantee backward compatibility for at least two major releases?
    • What is the estimated effort (in person‑days) required to migrate your data out of your platform?
  • Score vendors on a weighted matrix: 40 % technical fit, 30 % export‑readiness, 20 % cost, 10 % support.
  • Require a signed “Export Commitment Letter” that outlines the exact formats, version numbers and any associated fees.

Phase 2 – Architecture: Selecting the Open Table Format

  • Run a short proof‑of‑concept (PoC) comparing the three leading open table formats on criteria such as ACID guarantees, schema‑evolution support, query‑engine compatibility and operational tooling.
  • Select the format that best matches the existing data‑lake storage (e.g., Iceberg for AWS S3, Delta Lake for Azure ADLS Gen2, Hudi for Google Cloud Storage).
  • Design a migration blueprint:
    1. Inventory all existing tables, noting partition keys, compression and schema version.
    2. Create a mapping spreadsheet that translates legacy storage properties to the target format’s equivalents.
    3. Schedule a bulk‑load window during low‑usage periods; use distributed tools (Spark + Iceberg write, Delta Lake’s optimize, or Hudi’s upsert) to rewrite data in place.
    4. Validate row‑counts, checksums and time‑travel queries before cutting over.
  • Document the chosen format in the organisation’s Data‑Lake Standards catalogue.

Phase 3 – Model Interface: Building a Swappable Layer

  • Adopt a vendor‑agnostic model format (ONNX is the de‑facto standard for interoperability; TorchScript and MLflow Models are acceptable alternatives for specific frameworks).
  • Create a Model Interface Library (MIL) that provides:
    • A unified predict(input) function that accepts a standardised feature vector (e.g., Arrow table or Pandas DataFrame).
    • Adapter classes for each supported vendor (TensorFlow Serving, Sagemaker, Azure ML, custom REST endpoints).
    • Version‑checking logic that rejects models whose ONNX opset exceeds the supported range.
  • Enforce that all new models are registered in the MIL registry with metadata: framework, opset version, input schema, output schema and licence.
  • Retire direct vendor‑specific calls in application code; replace them with calls to the MIL.

Phase 4 – Operations: Export‑First Monitoring and Continuous Compliance

  • Implement an automated export verification job that runs nightly against the production lakehouse:
    • Samples a random 0.1 % of tables, writes them to a temporary bucket in the target open format, and compares row counts and schema hash.
    • For each registered model, exports the ONNX artefact and validates it against the MIL schema.
  • Alert on any deviation (e.g., missing partition files, schema drift) and trigger a ticket to the Data‑Engineering team.
  • Quarterly, review the Interoperability Policy with the procurement team to incorporate lessons learned and emerging standards (e.g., Apache Arrow Flight SQL, Model Card schemas).

Phase 5 – Culture: Training and Incentives

  • Run a mandatory “Interoperability Fundamentals” workshop for data engineers, ML engineers and procurement officers.
  • Include interoperability metrics in team OKRs (e.g., “Reduce average model‑switch time to <4 h” or “Achieve 100 % export‑readiness score in vendor RFPs”).
  • Celebrate successes: publish case‑studies of teams that switched model providers without re‑platforming, highlighting cost savings and speed‑to‑market gains.

By following this playbook, organisations can move from ad‑hoc openness to a systematic, auditable capability that treats interoperability as a core quality attribute rather than an after‑thought. The result is a portfolio of AI assets that can be evaluated, upgraded or retired with minimal friction, protecting the enterprise from the lock‑in tax while preserving the ability to exploit the best‑of‑breed innovations in the market.

Comparison Table: Open Table Formats versus Proprietary Warehouse Solutions

When evaluating a lakehouse versus a traditional proprietary data warehouse, decision‑makers often focus on performance, cost and ecosystem support. The table below contrasts the three leading open table formats (Apache Iceberg, Delta Lake, Apache Hudi) with a representative proprietary columnar warehouse (e.g., Snowflake’s native storage or Amazon Redshift RA3) across dimensions that directly affect interoperability and lock‑in risk.

Criterion Apache Iceberg Delta Lake Apache Hudi Proprietary Warehouse (e.g., Snowflake/Redshift)
Storage Medium Object store (S3, ADLS, GCS) Object store (S3, ADLS, GCS) Object store (S3, ADLS, GCS) Managed proprietary storage (often tied to the vendor’s cloud)
ACID Transactions Yes, Serializable isolation Yes, Serializable isolation Yes, Write‑ahead log + MVCC Yes, vendor‑specific isolation levels
Schema Evolution Add, drop, rename, update; backward‑compatible Add, drop, rename; supports datatype widening Add, drop, rename; complex evolutions via versioning Supported, but often requires vendor‑specific ALTER commands
Time‑Travel / Version Query Native, based on snapshots Native, based on transaction log Native, based on commit timeline Available (e.g., Snowflake Time Travel) but limited retention windows and extra cost
Query Engine Compatibility Trino, Spark, Flink, Presto, DuckDB Spark, Databricks Runtime, Presto, Trino Spark, Flink, Hive, Presto Native SQL interface only; limited external engine support (some via ODBC/JDBC)
Open‑Source Licence Apache 2.0 Apache 2.0 (Databricks‑maintained) Apache 2.0 Proprietary; licence tied to subscription
Vendor Lock‑In Risk Low – data portable across clouds and engines Low – portable, but tighter integration with Databricks ecosystem Low – portable, strong upsert/incremental capabilities High – migration requires unloading data and rewriting pipelines
Typical Cost (Storage + Compute) Storage at object‑store rates; compute via open engines (pay‑as‑you‑go) Similar to Iceberg; optional Databricks‑optimised runtime adds premium Similar; extra compaction costs for merge‑on‑read Storage bundled with compute; often higher per‑TB cost due to licensing
Maturity (Adoption in Enterprises) High – used by Apple, Netflix, LinkedIn High – default on Databricks, growing in finance Medium – strong in streaming/use‑case‑specific workloads Very high – incumbent in many organisations

Key Takeaways from the Table

  • All three open formats deliver full ACID guarantees and rich schema evolution while keeping the data in an object store that can be accessed by any compatible query engine.
  • Time‑travel capabilities are native and cost‑free, unlike the often‑limited and chargeable equivalents in proprietary warehouses.
  • Query‑engine compatibility is the strongest lever for interoperability: an organisation can switch from Trino to Spark to Flink without moving data, a flexibility that proprietary systems rarely match.
  • The lock‑in risk column shows that, even when a proprietary warehouse offers excellent performance, the cost of exiting—both in data transfer effort and pipeline rewrites—remains substantially higher than with an open table format.
  • For organisations whose primary goal is to avoid the lock‑in tax, the open formats provide a clear, measurable advantage, especially when combined with a swappable model interface (see Section 2).

In practice, many enterprises adopt a hybrid approach: they keep the raw lakehouse in an open format for maximum portability, while materialising aggregated marts into a proprietary warehouse for specific workloads that demand sub‑second latency. The table above helps quantify the trade‑offs and guides the decision of where to draw the line.

Common Pitfalls in Pursuing Interoperability and How to Avoid Them

Even with a clear strategic intent to avoid lock‑in, organisations frequently encounter practical obstacles that erode the anticipated benefits of open formats and swappable models. The following list distils the most recurrent pitfalls observed across finance, healthcare and manufacturing sectors, paired with concrete mitigation tactics that can be embedded into policies, architecture reviews and delivery programmes.

Pitfall 1 – “Export‑Ready” Claims Without Verification

Vendors often answer the export question with a vague statement such as “data can be exported via our API.” In reality, the API may be rate‑limited, require custom scripting, or exclude critical metadata (e.g., column‑level descriptions, lineage).

Mitigation: Treat the export claim as a testable requirement. Include in the RFP a concrete export scenario: “Provide a full dump of the TPC‑DS benchmark schema, including all table properties, partitioning keys and sample data, in Apache Iceberg v1.4 format within 48 hours.” Verify the deliverable against a checksum and schema‑validation script before signing the contract.

Pitfall 2 – Over‑Reliance on a Single Open Format

Choosing Iceberg, for example, and then building all downstream tools exclusively around its Java‑based API can recreate a de‑facto lock‑in, because migrating to Delta Lake would require rewriting those custom integrations.

Mitigation: Abstract the table‑format layer behind a thin façade that implements a common interface (e.g., a TableFactory that returns a read‑only Arrow table). Keep all business‑logic code format‑agnostic; only the façade knows the concrete format. This enables a format swap with minimal code changes.

Pitfall 3 – Neglecting Model Metadata and Versioning

Teams may export models to ONNX but forget to include the accompanying input/output schema, preprocessing steps or version numbers. The result is a model that can be loaded but produces nonsensical predictions when the feature pipeline has drifted.

Mitigation: Adopt a model‑registry standard such as MLflow Models or the Open Model Schema (OMS). Require that every registered model artefact bundle includes: (a) the ONNX file, (b) a JSON schema describing expected feature types and ranges, (c) a hash of the preprocessing pipeline code, and (d) a semantic version tag. Enforce this via a pre‑commit hook in the model‑CI pipeline.

Pitfall 4 – Inadequate Monitoring of Export Drift

Even after an initial successful export, schema changes in the source system (e.g., adding a new column) can silently break downstream exports if the export job is not configured to capture the evolving schema.

Mitigation: Implement automated schema‑drift detection as part of the nightly export verification job (see Section 2, Phase 4). Use tools like Great Expectations or Deequ to generate expectations on column names, types and nullability; fail the job and raise an alert when expectations are violated.

Pitfall 5 – Overlooking Governance of Data Lineage

Open formats make it easy to move data, but without proper lineage tracking, organisations lose the ability to answer audit questions such as “Which version of the feature table was used to train the model currently in production?”

Mitigation: Capture lineage at the point of export using open standards like OpenLineage. Export not only the data but also a lineage file that records the source job ID, timestamp, transformation steps and target location. Store lineage artefacts alongside the data in the lakehouse and index them in a searchable catalogue (e.g., Amundsen or DataHub).

Pitfall 6 – Assuming Cost Savings Are Immediate

The migration to an open table format often involves an upfront investment in data‑rewrite, tooling and training. Teams that expect immediate ROI may abandon the effort prematurely, reverting to familiar proprietary shortcuts.

Mitigation: Model the total cost of ownership (TCO) over a 24‑month horizon, including migration effort, ongoing operational savings and the avoided cost of future lock‑in incidents. Present the TCO analysis to the steering committee as part of the business case, and set milestones that celebrate incremental wins (e.g., “First month: 30 % of historic tables migrated; second month: model‑switch time reduced by 50 %”).

Pitfall 7 – Ignoring Edge‑Case Workloads That Require Proprietary Optimisations

Certain specialised workloads—such as real‑time fraud detection with sub‑millisecond latency—may still benefit from proprietary indexing or hardware acceleration that open formats do not yet expose.

Mitigation: Adopt a “fit‑for‑purpose” strategy: keep the canonical lakehouse in an open format for portability, and create purpose‑built, read‑only materialised views or caches in the proprietary system when performance demands justify it. Ensure that these views are regenerated from the open source on a defined schedule, preserving the ability to recompute them elsewhere if needed.

By recognising these pitfalls early and embedding the corresponding safeguards into policies, architecture standards and delivery practices, organisations can preserve the integrity of their interoperability investment and continue to reap the strategic advantages of an open, vendor‑neutral AI stack.

“The greatest risk is not the technology; it is the assumption that openness will happen automatically. Treat interoperability as a product that requires requirements, testing and continuous improvement – just like any other enterprise capability.” – Senior Architect, Global Manufacturing Conglomerate

Frequently Asked Questions

After years of painful migrations, buyers now ask the exit question first — can I export data, definitions and evaluations in an open format? Vendors that hedged lost deals to those that answered plainly.
Open, portable formats won, so the same dataset serves BI, AI and archival without copying, and changing vendor does not mean changing data — removing the trap fear for mid-market firms.
A consensus on open model interfaces made swapping models a component change, not a rewrite. The model became swappable, letting cost and quality competition reach the buyer.
Written confirmation of open export for data, definitions and evaluations, plus a swappable model interface. If a vendor cannot commit, treat lock-in as a future cost.

Book a personalised demo

Ready to make your data auditable?

See how Beehive Strategy's conversational governance platform turns catalogues and lineage into answers your teams can query in plain language.

Book a Demo Explore the Solution
30%
Faster audit readiness
25%
Lower incident costs
40%
Less remediation time
2 wks
To a live catalogue