Airflow hacks for FC + Vortex — laptop demo, DAGs, white paper

[sethupathib]

Summary

Adds airflow-fc-vortex/: practical Airflow hacks to trigger Fusion Compiler → Vortex and guide workflow optimization — deliberately simpler than AutoFDO.

Why this exists

AutoFDO optimizes the CPU under the job. This project optimizes the assembly line: always run Vortex after FC, branch on severity, respect license seats, add corners as data.

Laptop-first (no Airflow install)

cd airflow-fc-vortex
pip install pyyaml
./examples/compare_all.sh
cat results/SUMMARY.txt

Ten hacks

Chain, severity branch, license pool, multi-corner map, short-circuit, dataset-on-log, retry/backoff, workflow JSON factory, SLA, UI params — see HACKS.md + drop-in dags/.

Killer measured insight

Naive grep on Vortex JSON false-positives all 4 corners (rule names always present). Fair overall routing correctly ARCHIVEs 2 clean corners.

FC→Vortex flow
Ten hacks
Grep trap
airflow_fc_vortex_flow.gif

Production

Copy dags/*.py → $AIRFLOW_HOME/dags, create pool fc_licenses, set FC_BIN / VORTEX_BIN / VORTEX_POLICY.

Test plan

• python3 tests/test_smoke.py
• ./examples/compare_all.sh — CLAIM_GATE yes_airflow_hacks_demo
• Figures, GIF, PDF built
• Mocks only — no proprietary binaries

_{To show artifacts inline, enable in settings.}