> why engineers are so keen on building their own workflow engines

Because all the existing ones suck.

(We built our own tiny one two. We need tight integration with systemd jobs and cgroups, and existing solutions don't do that.)