Crate grafos_batch 

grafos-batch — Task graph executor for DAG-structured batch jobs.

This crate provides a mini-Spark/Airflow-style batch job executor built on fabricBIOS. Tasks are organized into a directed acyclic graph (DAG) with explicit data dependencies. The executor computes a topological sort, groups independent tasks into execution waves, and runs them with automatic retry and failure propagation.

Quick start

use grafos_batch::*;

let mut graph = TaskGraph::new();
let a = graph.add_task(TaskDef {
    name: "produce".into(),
    task_fn: Box::new(|_ctx| {
        let mut data = std::collections::HashMap::new();
        data.insert("out".into(), b"hello".to_vec());
        Ok(TaskOutput { data })
    }),
    resource_req: ResourceReq::default(),
    inputs: vec![],
    outputs: vec![DataRef { name: "out".into(), format: DataFormat::RawBytes }],
    retries: 0,
});
let b = graph.add_task(TaskDef {
    name: "consume".into(),
    task_fn: Box::new(|ctx| {
        assert_eq!(ctx.inputs.get("out").unwrap(), b"hello");
        Ok(TaskOutput { data: std::collections::HashMap::new() })
    }),
    resource_req: ResourceReq::default(),
    inputs: vec![DataRef { name: "out".into(), format: DataFormat::RawBytes }],
    outputs: vec![],
    retries: 0,
});
graph.add_dependency(a, b).unwrap();
let plan = graph.build().unwrap();
assert_eq!(plan.waves().len(), 2);

Structs

DataRef

Reference to a named data artifact produced or consumed by a task.

ExecutionPlan

A validated execution plan with tasks grouped into waves.

ExecutionResult

Aggregate result of executing an entire ExecutionPlan.

Executor

Single-threaded task graph executor.

ResourceReq

Resource requirements for a task.

TaskContext

Execution context provided to a task function.

TaskDef

Definition of a task within the graph.

TaskGraph

Builder for constructing a task DAG.

TaskId

Unique identifier for a task within a TaskGraph.

TaskOutput

Output produced by a task function.

TaskResult

Result of executing a single task.

Enums

DataFormat

Format of data exchanged between tasks.

TaskStatus

Status of a completed task.