Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes.

slack CI CII Best Practices Twitter Follow

What is Argo Workflows?

Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).

  • Define workflows where each step in the workflow is a container.
  • Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG).
  • Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes.

Argo is a Cloud Native Computing Foundation (CNCF) hosted project.

Argo Workflows in 5 minutes

Use Cases

  • Machine Learning pipelines
  • Data and batch processing
  • ETL
  • Infrastructure automation
  • CI/CD

Why Argo Workflows?

  • Argo Workflows is the most popular workflow execution engine for Kubernetes.
  • It can run 1000s of workflows a day, each with 1000s of concurrent tasks.
  • Our users say it is lighter-weight, faster, more powerful, and easier to use
  • Designed from the ground up for containers without the overhead and limitations of legacy VM and server-based environments.
  • Cloud agnostic and can run on any Kubernetes cluster.

Read what people said in our latest survey

Try Argo Workflows

Access the demo environment (login using Github)

Screenshot

Ecosystem

Ecosystem

Argo Events | Argo Workflows Catalog | Couler | Katib | Kedro | Kubeflow Pipelines | Onepanel | Ploomber | Seldon | SQLFlow

Client Libraries

Check out our Java, Golang and Python clients.

Quickstart

kubectl create namespace argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/master/manifests/install.yaml

Who uses Argo Workflows?

Official Argo Workflows user list

Documentation

Features

  • UI to visualize and manage Workflows
  • Artifact support (S3, Artifactory, Alibaba Cloud OSS, HTTP, Git, GCS, raw)
  • Workflow templating to store commonly used Workflows in the cluster
  • Archiving Workflows after executing for later access
  • Scheduled workflows using cron
  • Server interface with REST API (HTTP and GRPC)
  • DAG or Steps based declaration of workflows
  • Step level input & outputs (artifacts/parameters)
  • Loops
  • Parameterization
  • Conditionals
  • Timeouts (step & workflow level)
  • Retry (step & workflow level)
  • Resubmit (memoized)
  • Suspend & Resume
  • Cancellation
  • K8s resource orchestration
  • Exit Hooks (notifications, cleanup)
  • Garbage collection of completed workflow
  • Scheduling (affinity/tolerations/node selectors)
  • Volumes (ephemeral/existing)
  • Parallelism limits
  • Daemoned steps
  • DinD (docker-in-docker)
  • Script steps
  • Event emission
  • Prometheus metrics
  • Multiple executors
  • Multiple pod and workflow garbage collection strategies
  • Automatically calculated resource usage per step
  • Java/Golang/Python SDKs
  • Pod Disruption Budget support
  • Single-sign on (OAuth2/OIDC)
  • Webhook triggering
  • CLI
  • Out-of-the box and custom Prometheus metrics
  • Windows container support
  • Embedded widgets
  • Multiplex log viewer

Community Meetings

We host monthly community meetings where we and the community showcase demos and discuss the current and future state of the project. Feel free to join us! For Community Meeting information, minutes and recordings please see here.

Participation in the Argo Workflows project is governed by the CNCF Code of Conduct

Community Blogs and Presentations

Project Resources

Similar Resources

An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers

An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers

Developer-oriented Continuous Delivery Product ⁣ English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use? How to

Oct 19, 2021

Argo Rollout visualization in Argo CD Web UI

Argo Rollout visualization in Argo CD Web UI

Rollout Extension The project introduces the Argo Rollout dashboard into the Argo CD Web UI. Quick Start Install Argo CD and Argo CD Extensions Contro

Dec 29, 2022

Hera is a Python framework for constructing and submitting Argo Workflows.

Hera is an Argo Workflows Python SDK. Hera aims to make workflow construction and submission easy and accessible to everyone! Hera abstracts away workflow setup details while still maintaining a consistent vocabulary with Argo Workflows.

Dec 31, 2022

Demo for my talk at ArgoCon '21 showing how to use Go to create and submit dynamic Argo Workflows.

Demo for my talk at ArgoCon '21 showing how to use Go to create and submit dynamic Argo Workflows.

argocon-21-demo Demo for my talk at ArgoCon '21 showing how to use Go to create and submit dynamic Argo Workflows. This repo implements a Go-based CLI

Oct 12, 2022

Argo Workflows URL Finder

Argo Workflows URL Finder When Argo Workflows archives a workflow, the URL changes, and the URL cannot be pre-determined. If you're using tools such a

Aug 25, 2022

Powerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resources. https://cyclone.dev

Powerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resources. https://cyclone.dev

Cyclone Cyclone is a powerful workflow engine and end-to-end pipeline solution implemented with native Kubernetes resources, with no extra dependencie

Dec 6, 2022

Devtron is an open source software delivery workflow for kubernetes written in go.

Devtron is an open source software delivery workflow for kubernetes written in go.

Devtron is an open source software delivery workflow for kubernetes written in go.

Jan 8, 2023

Github-workflow-dashboard - WEB and CLI dashboard for github action workflows

Github-workflow-dashboard - WEB and CLI dashboard for github action workflows

CLI capable of retrieving github action workflows stats Example usage Dashboard

Aug 30, 2022

Reward is a Swiss Army knife CLI utility for orchestrating Docker based development environments.

Reward is a Swiss Army knife CLI utility for orchestrating Docker based development environments.

Reward Reward is a Swiss Army knife CLI utility for orchestrating Docker based development environments. It makes possible to run multiple local envir

Dec 9, 2022

Ritchie CLI is an open-source tool that allows to create, store and share any kind of automation, executing them through command lines, to run operations or start workflows ⚙️ 🖥 💡

Ritchie CLI is an open-source tool that allows to create, store and share any kind of automation, executing them through command lines, to run operations or start workflows ⚙️ 🖥 💡

Table of contents 1. About 2. Getting Started i. Installation ii. Initialize rit locally iii. Add your first formulas repository iv. Run the Hello Wor

Dec 29, 2022

parallel: a Go Parallel Processing Library

parallel: a Go Parallel Processing Library Concurrency is hard. This library doesn't aim to make it easy, but it will hopefully make it a little less

May 9, 2022

M3u8-parallel-downloader - M3u8 parallel downloader with golang

m3u8-parallel-downloader Usage ./m3u8-parallel-downloader -input http://example.

Aug 12, 2022

Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines.

Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines.

Dataflow Summary Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines. Each pipeline is specified as a Kube

Jan 4, 2023

Automatic container image update for Argo CD

Argo CD Image Updater Introduction Argo CD Image Updater is a tool to automatically update the container images of Kubernetes workloads which are mana

Dec 25, 2022

Workflow engine for Kubernetes

Workflow engine for Kubernetes

What is Argo Workflows? Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflow

Jan 4, 2023

Hybridnet is an open source container networking solution, integrated with Kubernetes and used officially by following well-known PaaS platforms

Hybridnet What is Hybridnet? Hybridnet is an open source container networking solution, integrated with Kubernetes and used officially by following we

Jan 4, 2023

Serverless Container Workflows

Serverless Container Workflows

direktiv event-based serverless container workflows Check out our online demo: wf.direktiv.io What is Direktiv? Direktiv is a specification for a serv

Dec 20, 2022

Rook is an open source cloud-native storage orchestrator for Kubernetes

Rook is an open source cloud-native storage orchestrator for Kubernetes, providing the platform, framework, and support for a diverse set of storage solutions to natively integrate with cloud-native environments.

Oct 25, 2022
Related tags
Hera is a Python framework for constructing and submitting Argo Workflows.

Hera is an Argo Workflows Python SDK. Hera aims to make workflow construction and submission easy and accessible to everyone! Hera abstracts away workflow setup details while still maintaining a consistent vocabulary with Argo Workflows.

Dec 31, 2022
Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines.
Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines.

Dataflow Summary Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines. Each pipeline is specified as a Kube

Jan 4, 2023
Automatic container image update for Argo CD

Argo CD Image Updater Introduction Argo CD Image Updater is a tool to automatically update the container images of Kubernetes workloads which are mana

Dec 25, 2022
A Kubernetes operator for managing Argo CD clusters.

Argo CD Operator A Kubernetes operator for managing Argo CD clusters. Documentation See the documentation for installation and usage of the operator.

Dec 14, 2022
Notifications for Argo CD
Notifications for Argo CD

Argo CD Notifications Argo CD Notifications continuously monitors Argo CD applications and provides a flexible way to notify users about important cha

Nov 27, 2022
Argo CD ApplicationSet Controller

The ApplicationSet controller manages multiple Argo CD Applications as a single ApplicationSet unit, supporting deployments to large numbers of clusters, deployments of large monorepos, and enabling secure Application self-service.

Dec 14, 2022
Support for extending Argo CD

Argo CD Extensions To enable Extensions for your Argo CD cluster will require just a single kubectl apply. Here we provide a way to extend Argo CD suc

Dec 20, 2022
Argo-CD Autopilot
Argo-CD Autopilot

Introduction New users to GitOps and Argo CD are not often sure how they should structure their repos, add applications, promote apps across environme

Jan 6, 2023
A series of controllers for configuring namespaces to accomodate Argo

argo-controller A series of controllers for configuring namespaces to accomodate Argo. ArgoCD TBD Argo Workflows Make a service account in every names

Jan 4, 2022
Provider-generic-workflows - A generic provider which uses argo workflows to define the backend actions.

provider-generic-workflows provider-generic-workflows is a generic provider which uses argo workflows for managing the external resource. This will re

Jan 1, 2022