Fivetran says Managed Data Lake Write Credentials are now generally available, letting query engines run DDL and DML against lake tables. Engines authenticate via OAuth2 client credentials—one Client ID and Secret per destination—so each gets a distinct identity against the hosted catalog. For teams running a dual-format Iceberg and Delta lake, the real question is how to scope write access tightly enough that an engine can delete rows or drop a table without broad bucket permissions, while Fivetran retains sole authority over metadata and schema evolution. This article explains how Fivetran strikes that balance, and what to set up before you rely on it.

Why it matters

For data engineers, the practical consequence is that permission management collapses into a single policy layer. Fivetran's catalog returns per-user access rules through the Iceberg REST protocol, so an organization defines governance once rather than re-encoding it for every engine. In Fivetran's post, Snowflake's Spitzer describes this as one policy definition and one catalog access point feeding all connected engines simultaneously. Engineers no longer maintain separate ACLs per query tool.

For AI and data engineers who provision write paths, the writable catalog credentials are visible only to Destination or Account Admins in the Fivetran dashboard. That means a data scientist spinning up an ad-hoc Spark session receives read-only access by default; only an admin can mint the short-lived, directory-scoped S3 credentials that unlock DDL and DML against a specific table's storage. The engine never holds standing broad bucket permissions, and because Fivetran retains sole authority over metadata publication, a stray schema edit in one engine cannot desynchronize what another engine reads.

What happened

Fivetran announced its Managed Data Lake service, which pulls data through more than 750 connectors and lands it in commodity object-storage buckets such as S3, ADLS, and GCS. The pipeline normalizes, compacts, and deduplicates records before writing them in both Apache Iceberg and Delta Lake open table formats, granting warehouse-grade ACID transactions on top of a lake. Alongside the core ingestion path, Fivetran introduced a write-credentials mechanism: short-lived tokens that let external compute engines execute DDL and DML directly against the lake's data files while Fivetran retains sole authority over metadata updates. According to Fivetran, this decoupling of storage from compute lets customers query and modify data through engines such as Databricks, Snowflake, or Starburst without Fivetran mediating every write.

The managed data lake write credentials feature supports destinations on S3, Azure Data Lake Storage, OneLake, and Google Cloud Storage. It includes native catalog integrations for AWS Glue, Databricks Unity Catalog, and Apache Polaris. Fivetran manages schema evolution and performance optimization on the backend, allowing engine-side writers to use vended credentials for standard SQL or Python operations.

This is an excerpt. The full article continues with:

  • Managing write access
  • Before you turn it on
  • What data and AI engineers should do
  • Limitations and open questions
  • Key takeaways

Read the full article on AI Data Legends → — with diagrams, code and sources.

Originally published on AI Data Legends. Each post here is an excerpt; the full articles live at aidatalegends.com.