Image: Apache Airflow — the refine_estimate loop, on its third of up to ten iterations (official release blog screenshot)
📌 Key Highlights
- Apache Airflow 3.4.0 was released on 6 November 2026, with Task Loops (AIP-111), whole-Dag authoring beyond Python (AIP-85) and common.ai going GA.
- Task Loops: call
.loop()on a decorated task group with a requiredmax_iterationscap and an optionaluntilpredicate. - Whole Dags can now be written in TypeScript, Java or Go — all three SDKs are GA — plus an opt-in YAML importer.
- common.ai 1.1.0 is GA, turning LLM calls and agents into ordinary Airflow tasks with human approval.
- Upgrade with
pip install apache-airflow==3.4.0ordocker pull apache/airflow:3.4.0.
What happened?
Apache Airflow 3.4.0 landed on 6 November 2026. It follows Airflow 3.3.0 (6 July 2026), which gave tasks a memory and let task bodies be written in Java or Go. Where 3.3 was about what tasks remember, 3.4 is about workflow shape and who can write Dags.
Apache Airflow 3.4.0 new features explained
Task Loops (AIP-111): repeat until done, with a hard cap
Some workflows have no fixed shape: walking a tree of unknown depth, refining an estimate, or running an agentic generate → score → refine loop. Until now, Airflow made you hard-code a fixed depth or build Dags that trigger themselves.
Call .loop() on a decorated task group, with max_iterations as a required upper bound, so a loop can never run away. An optional until predicate decides when to stop early. It behaves like a do … while loop: the body runs before the check, so it always runs at least once, and a loop context object exposes loop.index, loop.previous and loop.result.
Iterations are created lazily, and a loop that hits its cap without converging is marked failed, so existing alerting catches it. The limits: no nesting, no loop inside a mapped task group, and a whole loop cannot be mapped.
Dags beyond Python: TypeScript, Java, Go and YAML
Airflow 3.3 let you write individual tasks in Java or Go while Python still owned the Dag. Airflow 3.4 takes the next step: you can define the entire Dag — schedule, tasks and dependencies — in TypeScript, Java, Go, with no Python file involved. All three SDKs are generally available, powered by AIP-85, the Dag Importer. An opt-in YAML importer on the same framework covers declarative Dags.
Each SDK builds one self-contained artifact: TypeScript to a .min.mjs file (Node.js 22+), Java to a fat JAR, Go to an executable needing no runtime. A Dag definition supports TaskFlow inputs, nested task groups, edge labels, if / else and switch / case branching, and triggering another Dag's run. Java teams will also find the runtime story improving elsewhere — GraalVM 25.5's native-image upgrades are worth a look alongside this release.
common.ai goes GA: LLM and agent tasks, with a human in the loop
The common.ai provider reaches general availability, shipping version 1.1.0 with Airflow 3.4.0. Built on pydantic-ai, it lets a failed call retry like any task. The simplest form is the @task.llm decorator; AgentOperator runs a full agent, with toolsets for SQL, hooks, MCP servers, object storage and sandboxes, plus cost limits. AI routing is a live cost question for agent pipelines, and Route Anything's AI model routing shows one way teams are cutting inference spend.
An agent can pause for a person to approve a tool call or the final answer, using the awaiting_input state from Airflow 3.3. The task parks without holding a worker slot and resumes when a reviewer responds.
What 3.4.0 changes vs 3.3
| Capability | In Airflow 3.3 | In Airflow 3.4.0 |
|---|---|---|
| Repeating work | Hard-coded depth or self-triggering Dags | Task Loops with a max_iterations cap |
| Dag language | Python owns the Dag | Whole Dag in TypeScript, Java, Go or YAML |
| AI tasks | common.ai 1.0 | common.ai 1.1.0 GA, with human approval |
| Dag organisation | Flat Dags list | Folders panel by bundle layout |
Image: Apache Airflow — a draft-and-review Task Loop at iteration 3 of 5, awaiting approval (official release blog screenshot)
Why it matters for Indian data teams
Airflow is a staple orchestrator for data and machine-learning teams in India, and this release lowers two barriers: first, the Python barrier — many Indian engineering teams are Java- or TypeScript-heavy, and until now joining the Airflow side of a data platform meant writing Python. Whole-Dag authoring lets them own pipelines in their own language. Second, iteration: Indian startups shipping LLM features need generate → score → refine loops and human review, and Task Loops plus common.ai's approval flow give them a supervised, bounded way to run those inside Airflow itself. Caveat: the new surface is young — pin the versions you test.
Should you care?
How to upgrade
Airflow is free, open-source software under the Apache licence — there is no price or India launch date to wait for. Upgrade with pip install apache-airflow==3.4.0 using the release constraints file, or pull docker pull apache/airflow:3.4.0. Before upgrading, review the Significant Changes section of the official release notes. PyCharm 2026.3 adds Python 3.15 lazy imports — see our PyCharm 2026.3 new features explained.
FAQs
What is new in Apache Airflow 3.4.0?
Released on 6 November 2026, it adds bounded Task Loops, whole-Dag authoring in TypeScript, Java, Go and YAML, common.ai 1.1.0 as generally available, and a Dag Folders panel.
How do Task Loops work in Apache Airflow 3.4.0?
Call .loop() on a decorated task group with a required max_iterations cap and an optional until predicate. It behaves like a do-while loop, and a loop that hits its cap without converging fails.
Can you write Airflow Dags in TypeScript, Java or Go?
Yes — in Airflow 3.4.0 all three language SDKs are generally available, so you can define the whole Dag with no Python file involved. Each SDK builds one artifact, and you register a coordinator per language.
What is the common.ai provider in Apache Airflow?
It turns LLM calls and agents into ordinary Airflow tasks, built on pydantic-ai. It offers @task.llm and AgentOperator, toolsets, cost limits, and human approval of tool calls or final answers. Version 1.1.0 ships with Airflow 3.4.0.
Takeaway
Airflow 3.4.0 is a release about reach: loops for work that repeats until it is done, Dags in your team's language, and AI agents that run as supervised, auditable tasks.
If you run Airflow, try the refine_estimate example on a staging Dag first.
Comments
Post a Comment