Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ A durable, resumable workflow engine for Elixir. Similar to Temporal/Inngest.
- **Compensations** - Saga pattern with automatic rollback
- **Cron Scheduling** - Recurring workflows with cron expressions
- **Reliability** - Automatic retries with exponential/linear/constant backoff
- **Orchestration** - Parent/child workflow composition
- **Persistence** - PostgreSQL-backed execution state

## Installation
Expand Down Expand Up @@ -439,6 +440,21 @@ hours(2) # 7_200_000 ms
days(7) # 604_800_000 ms
```

### Orchestration

```elixir
use Durable.Orchestration

# Synchronous: call child and wait for result
case call_workflow(MyApp.PaymentWorkflow, %{"amount" => 100}, timeout: hours(1)) do
{:ok, result} -> {:ok, assign(data, :payment, result)}
{:error, reason} -> {:error, reason}
end

# Fire-and-forget: start child and continue
{:ok, child_id} = start_workflow(MyApp.EmailWorkflow, %{"to" => email}, ref: :welcome)
```

### API

```elixir
Expand All @@ -449,6 +465,7 @@ Durable.list_executions(workflow: Module, status: :running)
Durable.cancel(id, "reason")
Durable.send_event(id, "event", payload)
Durable.provide_input(id, "input_name", data)
Durable.list_children(parent_id)
```

## Guides
Expand All @@ -457,10 +474,10 @@ Durable.provide_input(id, "input_name", data)
- [Parallel](guides/parallel.md) - Concurrent execution
- [Compensations](guides/compensations.md) - Saga pattern
- [Waiting](guides/waiting.md) - Sleep, events, human input
- [Orchestration](guides/orchestration.md) - Parent/child workflow composition

## Coming Soon

- Workflow orchestration (parent/child workflows)
- Phoenix LiveView dashboard

## License
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# DurableWorkflow Implementation Plan

> **⚠️ ARCHIVED (2026-01-23):** This document is no longer maintained.
> See `WORKPLAN.md` for current status and `arch.md` for technical reference.
>
> Key changes since this document was written:
> - ForEach primitive was **removed** (use `Enum.map` instead)
> - Parallel execution uses new results model (`__results__`, `into:`, `returns:`)
> - Loop primitive was **never implemented** (use step retries or `Enum` functions)

## Executive Summary

This document outlines the complete implementation plan for **Durable**, a durable, resumable workflow engine for Elixir.
Expand Down
108 changes: 70 additions & 38 deletions agents/WORKPLAN.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# Durable Workflow Engine - Work Plan

**Last Updated:** 2026-01-03
**Last Updated:** 2026-02-27
**Overall Progress:** ~75% Complete
**Reference:** See `IMPLEMENTATION_PLAN.md` for detailed design and code examples
**Reference:** See `arch.md` for technical architecture

---

## Quick Stats

| Metric | Value |
|--------|-------|
| Source Modules | 41 |
| Passing Tests | 214 |
| Source Modules | 40 |
| Passing Tests | ~268 |
| Documentation Guides | 6 |
| Lines of Code | ~8,500 |

Expand All @@ -24,7 +24,7 @@
| 0 | Project Foundation | Complete | 100% |
| 1 | Core MVP | Complete | 100% |
| 2 | Observability | Partial | 40% |
| 3 | Advanced Features | Mostly Complete | 85% |
| 3 | Advanced Features | Mostly Complete | 90% |
| 4 | Scalability | Not Started | 0% |
| 5 | Developer Experience | Partial | 35% |

Expand Down Expand Up @@ -86,7 +86,7 @@

---

## Phase 3: Advanced Features [85%]
## Phase 3: Advanced Features [90%]

### 3.1-3.3 Wait Primitives [COMPLETE - 46 tests]

Expand Down Expand Up @@ -117,26 +117,27 @@

### 3.5 Loops [SKIPPED]

Intentionally skipped - use step-level retries or `foreach` instead.
Intentionally skipped - use step-level retries or Elixir's `Enum` functions instead.

### 3.6 Parallel Execution [COMPLETE - 13 tests]

| Feature | Status |
|---------|--------|
| `parallel do` macro | Complete |
| Context merge strategies | Complete |
| Results model (`__results__`) | Complete |
| `into:` custom merge function | Complete |
| `returns:` option | Complete |
| Error strategies | Complete |
| Resume durability | Complete |

### 3.7 ForEach [COMPLETE - 13 tests]
See `guides/parallel.md` for comprehensive documentation.

| Feature | Status |
|---------|--------|
| `foreach` macro | Complete |
| `current_item/0`, `current_index/0` | Complete |
| Sequential/Concurrent modes | Complete |
| `:collect_as` option | Complete |
| `:on_error` handling | Complete |
### 3.7 ForEach [REMOVED]

**Decision (2026-01-23):** The `foreach` primitive was removed. Users should use
Elixir's built-in enumeration tools (`Enum.map`, `Task.async_stream`) for batch
processing instead. This simplifies the DSL while providing the same functionality
through idiomatic Elixir.

### 3.8 Switch/Case [NOT STARTED]

Expand Down Expand Up @@ -165,12 +166,24 @@ Low priority - `branch` macro covers most cases.
| Manual trigger | Complete |
| Telemetry events | Complete |

### 3.11 Workflow Orchestration [COMPLETE - 12 tests]

| Feature | Status |
|---------|--------|
| `call_workflow/3` (synchronous) | Complete |
| `start_workflow/3` (fire-and-forget) | Complete |
| Idempotent resume | Complete |
| Cascade cancellation | Complete |
| Parent notification on child complete/fail | Complete |
| Nested workflows (A → B → C) | Complete |
| `Durable.list_children/2` API | Complete |

See `guides/orchestration.md` for comprehensive documentation.

### Remaining Phase 3 Work

| Feature | Priority | Complexity |
|---------|----------|------------|
| Workflow Orchestration (`call_workflow`) | High | Medium |
| Parent-child tracking | High | Low |
| Switch/Case macro | Low | Low |
| Pipe-based API | Low | Medium |

Expand Down Expand Up @@ -204,7 +217,7 @@ Note: Multi-node scheduling already works via `FOR UPDATE SKIP LOCKED`.
| Module docs (@moduledoc) | Complete |
| Function docs (@doc) | Complete |
| Typespecs (@spec) | Complete |
| 6 Documentation Guides | Complete |
| 7 Documentation Guides | Complete |

### Remaining

Expand All @@ -229,25 +242,24 @@ Note: Multi-node scheduling already works via `FOR UPDATE SKIP LOCKED`.
### High Priority

1. Guide: Getting Started
2. Workflow Orchestration (`call_workflow`)
3. HexDocs Publishing
4. `mix durable.status`
2. HexDocs Publishing
3. `mix durable.status`

### Medium Priority

5. Guide: Testing Workflows
6. `Durable.TestCase`
7. Graph Generation
8. `mix durable.list`
9. pg_notify Message Bus
4. Guide: Testing Workflows
5. `Durable.TestCase`
6. Graph Generation
7. `mix durable.list`
8. pg_notify Message Bus

### Lower Priority

10. Switch/Case macro
11. Redis Queue Adapter
12. Phoenix Dashboard
13. Example Project
14. Pipe-based API
9. Switch/Case macro
10. Redis Queue Adapter
11. Phoenix Dashboard
12. Example Project
13. Pipe-based API

---

Expand All @@ -259,30 +271,50 @@ Note: Multi-node scheduling already works via `FOR UPDATE SKIP LOCKED`.
| wait_test.exs | 46 | Wait primitives |
| decision_test.exs | 13 | Decision steps |
| parallel_test.exs | 13 | Parallel execution |
| foreach_test.exs | 13 | ForEach iteration |
| log_capture_test.exs | 13 | Log/IO capture |
| integration_test.exs | 11 | End-to-end flows |
| branch_test.exs | 10 | Branch macro |
| durable_test.exs | 8 | Core API |
| compensation_test.exs | 6 | Saga pattern |
| Other | ~36 | Queue, handlers, etc. |
| **Total** | **214** | |
| orchestration_test.exs | 12 | Workflow orchestration |
| **Total** | **~268** | |

---

## Known Limitations

1. Wait primitives not supported in parallel/foreach blocks
1. Wait primitives not supported in parallel blocks
2. No backward jumps in decision steps (forward-only by design)
3. Context is single-level atomized (top-level keys only)
4. No workflow versioning
5. No foreach/loop DSL primitives (use Elixir's `Enum` functions)

---

## Next Steps

1. **Documentation** - Getting Started guide and HexDocs publishing
2. **Workflow Orchestration** - Child workflow support (`call_workflow`)
3. **Graph Visualization** - Understanding complex workflows
2. **Graph Visualization** - Understanding complex workflows
3. **Testing Helpers** - `Durable.TestCase` for easier workflow testing

The existing ~268 tests provide good confidence in implemented features. Suitable for internal use; additional documentation needed before public release.

---

The existing 214 tests provide good confidence in implemented features. Suitable for internal use; additional documentation needed before public release.
## Changelog

### 2026-02-27
- Added workflow orchestration: `call_workflow/3` (synchronous) and `start_workflow/3` (fire-and-forget)
- Added `Durable.Orchestration` module with `use Durable.Orchestration` macro
- Added cascade cancellation (cancelling parent cancels active children)
- Added parent notification on child completion/failure
- Added `Durable.list_children/2` API
- Added `guides/orchestration.md` documentation
- 12 new tests for orchestration (total: ~268)

### 2026-01-23
- Removed `foreach` primitive (use `Enum.map` or `Task.async_stream` instead)
- Updated parallel execution with new results model (`__results__`, `into:`, `returns:`)
- Updated documentation in `guides/parallel.md`
- Archived `IMPLEMENTATION_PLAN.md` (now `IMPLEMENTATION_PLAN_ARCHIVED.md`)
Loading