Originally published on joshuafields.uk.

Django's permissions answer "can this user change documents?". They don't answer "can this user change this document?". Pass an object to has_perm and the default ModelBackend returns False, even for a user who holds the permission. django-guardian adds per-object grants for users and groups, and that goes a long way. But "share this folder with the design team, and everything in it follows" is a graph, and you end up writing the walk yourself.

I've built production permissions on SpiceDB, and it handles that graph well. It's also a second system, so every grant is a second write. The document commits, the grant call fails, and now you have a document nobody can open. Grant first, roll back the transaction, and you have a permission pointing at nothing. transaction.on_commit narrows the gap but doesn't close it: if the process dies between the commit and the callback, the grant never happens.

Keep the relationships in the same database

Zanzibar is the model Google published for Drive-style sharing. You store facts as relationships, such as document:roadmap#viewer@team:design#member (members of the design team can view the roadmap), and a schema says how they combine into permissions. SpiceDB and OpenFGA run that model as services. pemba, a library I wrote, runs it inside your process and keeps the relationships in your own database. A grant is then rows in the transaction you already have, so it commits or rolls back with everything else.

The code below is copied from a working project, which I ran on Postgres 18 and SQLite 3.49 with pemba 0.3.0 from PyPI, on Django 5.2 and 6.1, with psycopg 3 and psycopg2. Its tests also pass on Postgres 15.

Setting up 

pip install pemba django "psycopg[binary]"

Here's the model: users, teams with members, folders, and documents with an owner, editors and viewers. A viewer can be a user or everyone in a team, and parent->view means anyone who can view the folder can view the document.

definition user {} definition team { relation member: user } definition folder { relation viewer: user | team#member permission view = viewer } definition document { relation parent: folder relation owner: user relation editor: user relation viewer: user | team#member permission edit = owner + editor permission view = viewer + edit + parent->view }

pemba's tables live in their own Postgres schema, and pemba migrates them on its own connection. So the schema has to be committed before pemba opens, which takes two Django migrations: one that runs CREATE SCHEMA IF NOT EXISTS authz, then this one.

def setup_store(apps, schema_editor): migrate_store() class Migration(migrations.Migration): # pemba migrates on its own connection, so this can't sit inside a transaction.  atomic = False dependencies = [("documents", "0002_authz_schema")] operations = [migrations.RunPython(setup_store, migrations.RunPython.noop)]

Running it from a migration also means Django's test runner sets up the store in the test database. At runtime the engine opens with migrations off:

def _location(): # Read on every call, because the test runner renames the database.  s = connection.settings_dict if connection.vendor == "sqlite": return str(s["NAME"]) user, password = quote(s["USER"], safe=""), quote(s["PASSWORD"], safe="") sslmode = s["OPTIONS"].get("sslmode", "prefer") return f"postgres://{user}:{password}@{s['HOST']}:{s['PORT']}/{s['NAME']}?sslmode={sslmode}" def _open(location, auto_migrate): if connection.vendor == "sqlite": return pemba.Engine.sqlite(location, auto_migrate=auto_migrate) return pemba.Engine.postgres(location, schema="authz", auto_migrate=auto_migrate) @lru_cache def _engine_for(location): return _open(location, auto_migrate=False) def get_engine(): # Opened on first use, so each gunicorn worker gets its own: Engine isn't fork-safe.  return _engine_for(_location()) def write_conn(): # Postgres takes Django's connection. SQLite takes the sqlite3.Connection inside it.  return connection if connection.vendor == "postgresql" else connection.connection

The engine has its own connection pool (8 by default) next to Django's, so count those connections when you size Postgres. It also has its own TLS (rustls, using the platform's trust roots), so set sslmode in Django's OPTIONS for production. The docs say to open the store before the transaction starts. Opening it lazily inside one worked on both databases with migrations off, but with auto_migrate=True on SQLite it failed with database is locked after 5 seconds. write_conn exists because the two databases differ. On Postgres pemba takes django.db.connection directly. On SQLite it only accepts a sqlite3.Connection, and passing Django's wrapper raises TypeError.

Creating a document and its owner together 

def create_document(title, owner, folder=None): engine = get_engine() with transaction.atomic(): document = Document.objects.create(title=title, owner=owner, folder=folder) relationships = [f"document:{document.id}#owner@user:{owner.pk}"] if folder is not None: relationships.append(f"document:{document.id}#parent@folder:{folder.id}") pending = engine.write(touch=relationships, conn=write_conn()) return document, pending

Passing conn= to engine.write runs it inside your open transaction. pemba doesn't commit or roll back, Django does. The call returns a PendingRevision, which I'll come back to.

The rollback

Because create_document uses atomic(), it nests inside a bigger transaction like any other Django code. This test fails a payment after the document is created, then looks for the row, the stored relationship and the permission:

def test_rollback_removes_the_row_and_the_permission(self): with self.assertRaises(PaymentFailed): with transaction.atomic(): document, pending = create_document("Secret", self.anne) raise PaymentFailed self.assertFalse(Document.objects.filter(pk=document.pk).exists()) self.assertEqual(stored_relationships(document), 0) self.assertFalse(can_view(self.anne, document))

It passes on both databases. stored_relationships is a raw SELECT count(*) on pemba's table, so that's zero rows rather than a denied check. The same thing from the example's demo command on Postgres:

rolled back, row exists: False rolled back, anne can view: False rolled back, pending revision: WRITE_ROLLED_BACK

A rollback to a savepoint behaves the same. In an outer block that commits and an inner one that raises, the outer document kept its relationship and the inner one had neither row nor relationship. To check the idea under load, I ran six threads doing 40 transactions each and failing 30% at random. On both databases the set of rows and the set of permissions were identical afterwards: 169 of each from 240 attempts, with no errors.

Checking and listing 

def can_view(user, document, at_least=None): return get_engine().check( f"document:{document.id}", "view", f"user:{user.pk}", at_least=at_least ) def visible_documents(user, at_least=None): ids = get_engine().iter_resources("document", "view", f"user:{user.pk}", at_least=at_least) return Document.objects.filter(pk__in=list(ids))

A denial is False. A problem during evaluation raises an exception, so a failed check never looks like a denial. I use iter_resources rather than lookup_resources here because lookup_resources returns one page of up to 1,000 ids, while iter_resources walks every page at one revision. Filtering through Document.objects also means a stale id can't come back as a row.

Reading your own write

The engine reads on its own connections, so it can't see anything you haven't committed. Inside the atomic() block a check says False, even for the owner. After the commit, pass the PendingRevision as at_least= and the read is guaranteed to include your write:

def test_owner_can_view_after_commit(self): document, pending = create_document("Roadmap", self.anne) self.assertTrue(can_view(self.anne, document, at_least=pending)) self.assertFalse(can_view(self.bob, document, at_least=pending))

If you run with ATOMIC_REQUESTS, your view body is inside the transaction, so at_least=pending raises there. Use on_commit, which worked on both databases:

def test_read_your_write_from_on_commit(self): seen = [] with transaction.atomic(): document, pending = create_document("Roadmap", self.anne) transaction.on_commit( lambda: seen.append(can_view(self.anne, document, at_least=pending)) ) self.assertEqual(seen, [True])

To carry it across requests, keep the number. It's a bound check rather than a wait: if the store is behind that revision, the read raises REVISION_NOT_YET_AVAILABLE instead of answering from older data.

def test_keep_the_revision_for_a_later_request(self): document, pending = create_document("Roadmap", self.anne) revision = int(pending.revision()) # after the commit, for example in the session  self.assertTrue(can_view(self.anne, document, at_least=revision))

Testing

Use TransactionTestCase for anything that reads a permission back. TestCase wraps each test in a transaction that always rolls back, so the engine never sees the write:

class WhyNotTestCase(TestCase): def test_the_engine_cannot_see_a_write_that_never_commits(self): anne = User.objects.create(username="anne") # TestCase wraps this test in a transaction that always rolls back, and the  # engine reads on its own connection, so it never sees this write.  document, _ = create_document("Roadmap", anne) self.assertFalse(can_view(anne, document))

Even captureOnCommitCallbacks(execute=True) doesn't help, because it runs the callback without a real commit: at_least=pending still raised. pemba's docs say TestCase can also fire the commit-time trigger early, through check_constraints(), and I saw that in the Postgres log as a WARNING for each test that wrote. The docs don't otherwise compare the two test classes. That advice is from running them.

The test database needs one more thing. Engine has no close(), and its pooled connections stop Django dropping the test database (database "test_app" is being accessed by other users). Releasing it first fixes that, with TEST_RUNNER pointing at this class:

def close_engines(): # Engine has no close(). Dropping it closes its connections, which the test runner  # needs before it can drop the test database.  _engine_for.cache_clear() gc.collect() class Runner(DiscoverRunner): def teardown_databases(self, old_config, **kwargs): close_engines() super().teardown_databases(old_config, **kwargs)

What doesn't work, or will catch you out

  • Forgetting conn=. A plain engine.write() inside atomic() isn't part of your transaction. On Postgres it committed on its own and would have survived a rollback. On SQLite it waited for the lock, then raised TransactionConflictError. Outside atomic(), conn=connection raises ValueError, and allow_autocommit=True doesn't change that for Django.
  • Deletes. document.delete(), cascades and the admin don't know about pemba, so the relationships stay behind and check still says True. delete_relationships() can't join a transaction, so removing everything takes a second step after the commit (the example does this). You can also pass delete=[...] with conn= for relationships you can list. If the process dies in between, the leftovers point at a row that no longer exists, which is why I use UUID keys and filter through the ORM.
  • manage.py flush. It empties your tables and leaves pemba's alone. It also resets sequences, so a new user can get an old primary key. In one test run on SQLite a new user with pk 1 inherited 7 documents from before the flush.
  • check_constraints() on Postgres. It runs SET CONSTRAINTS ALL IMMEDIATE, which fires pemba's trigger early and holds a store-wide lock until your transaction ends, so other pemba commits wait. The answers stay correct, and you get a PembaWarning when the revision resolves.
  • Pending revisions on SQLite. Postgres raises WriteRolledBack after a rollback. SQLite can't detect one: pending.revision() just returned a number, and the next commit reused it. Only trust that number if your commit succeeded.
  • SQLite settings. Django's default in-memory test database doesn't work, because pemba needs a real file. Django's default deferred transactions are a problem too: with six threads reading before writing, 165 of 180 transactions failed with database is locked, and with this setting none did. transaction_mode needs Django 5.1. 
"ENGINE": "django.db.backends.sqlite3", "NAME": BASE_DIR / "app.sqlite3", # Take the write lock at BEGIN (Django 5.1+), so a transaction that reads # first can't fail with "database is locked" when it later writes. "OPTIONS": {"transaction_mode": "IMMEDIATE"}, # pemba needs a real file, not Django's default in-memory test database. "TEST": {"NAME": BASE_DIR / "test_app.sqlite3"},
  • Query logging. pemba uses the raw driver connection, so its statements don't appear in connection.queries or CaptureQueriesContext. Expect your query-count assertions and any tooling built on them to miss it.
  • Retries. At REPEATABLE READ or SERIALIZABLE, retry the whole atomic() block with backoff, as the transactions guide describes. I didn't run this under Django. My runs were all at the default READ COMMITTED.
  • Coverage. I ran all of this on Windows, so Linux and macOS are untested here for the Django parts. The docs list no async route for Django's connection.

And what pemba doesn't do at all: there's no server mode, no caveats or CEL conditions, no expiring relationships, and no Watch API. It's a young library, and the engine lives in one application's database. If several services need to share permissions, or you need those features, use SpiceDB or OpenFGA. That's the trade: you give up a shared service, and you drop the second write.

How it's tested

pemba runs against SpiceDB on every fixture and at least 500 generated cases, and every known difference is written down with a repro. That testing found a nondeterminism bug in SpiceDB's WriteSchema, where the same schema was sometimes accepted and sometimes rejected, and AuthZed fixed it (authzed/spicedb#3418).

For cost, benches/RESULTS.md has Postgres figures from a Linux CI runner, with the server in a service container and a SELECT 1 round trip of about 119 µs. At p50, a direct check takes 608 µs, a check through 10 nested folders 2.7 ms, and a lookup over 10,000 documents 15.9 ms. Those come from one run just before the 0.3.0 release, so treat them as rough. Three later runs of the release candidate, on other runners, put the direct check at 551, 621 and 641 µs. On Postgres, most of that time is network round trips, so yours will follow your network.

Where would this break for you?

The whole project, with its tests, is in examples/django, and the repo has everything else. The browser playground runs the real engine, so you can try the schema above without installing anything, though it uses a memory store and can't show a transaction.

Even if you never install pemba, I'd take two things from this: decide what happens when the second write fails, and test permission reads against real commits. If you run per-object permissions in Django, I'd like to know where this approach would break for you. Open an issue and tell me.