Microsoft Open-Sources “pg_durable” Extension for In-Database Workflows

0
1

Microsoft has open-sourced pg_durable, a PostgreSQL extension that lets developers build long-running, fault-tolerant software workflows directly inside the database.

The extension introduces durable execution natively to PostgreSQL, eliminating the need for developers to patch together external tools like background workers, message queues, and external orchestration engines. Workflows are defined using standard SQL, allowing PostgreSQL to automatically track execution progress, handle retries, and recover states following unexpected server crashes or database restarts.

By shifting workflow state tracking directly into PostgreSQL tables, pg_durable guarantees that long-running data processes can survive unexpected system failures and automatically pick up from their last recorded checkpoint. The extension provides developers with dedicated SQL operators designed specifically for managing sequential steps and parallel tasks, making it highly optimized for asynchronous tasks. Key target workloads for the tool include heavy data processing pipelines—such as parsing data for AI vector embeddings using pgvector—as well as automated database maintenance tasks and scheduled API calls.

Architecturally minimal, the open-source extension runs via a native Postgres background worker with no external control plane requirements. The underlying orchestration runtime is powered by two new Microsoft-developed Rust libraries: duroxide, which coordinates deterministic event tracking, and duroxide-pg, which handles state persistence in a dedicated schema.

The pg_durable extension is available on GitHub and ships natively as part of Microsoft’s Azure HorizonDB cloud service.

LEAVE A REPLY

Please enter your comment!
Please enter your name here