Workflow Orchestration Made Simple | Prefect

Workflow Orchestration Made Simple | Prefect

  • Product
  • Solutions
  • Resources

Pricing

Signup / Login

image

Resilient by design, from script to scale

Try Prefect Cloud

flow.py

1from prefect import flow, task
2
3
4@task(log_prints=True)5def say_hello(name: str):6    print(f"Hello {name}!")
7
8
9@flow10def hello_universe(names: list[str]):11    for name in names:
12        say_hello(name)
13
14
15if __name__ == "__main__":
16    # create your first deployment to automate your flow17    hello_universe.serve(name="your-first-deployment")
image

Modern workflow orchestration for data and ML engineers

Control Panel

Orchestrate your code and get full observability into your workflows

Pure PythonWrite code however you want, without boilerplate or strict DAGs

Recover Quickly

Immediately take the right action to restore your pipelines to healthy

Develop locally, deploy to productionBring your workflows to prod without jumping through hoops

Run your code where you wantChoose and configure the infrastructure where your code is executed

Complete visibility

Control panel for your workflows

Orchestrate your code with scheduling, automatic retries, and prioritized instant alerting, giving you full observability into your workflows.

Learn More

image
image

Write It Your way

Pure Python

You bring raw Python functions, Prefect handles the rest. No more boilerplate code.

  • if statements
  • for and while loops
  • native subflows

Learn More

flow.py

1from prefect import flow, task
2
3@task4def add_one(x: int):5    return x + 16
7@flow8def main():9    for x in [1, 2, 3]:
10        first = add_one(x)
11        second = add_one(first)

Don't dread failures

Recover quickly

With custom retry behavior, caching, and extensive automations, go from red to green in minutes instead of days.

Lightweight

Easy local development

Start a local test server with a single command,

and test your work without pain.

Learn More

hello.py

1from prefect import flow
2
3@flow4def say_hi():5    print("Hi, I'm a Prefect flow!")
6
7if __name__ == "__main__":
8    say_hi.serve(name="my-deployment")
Your flow 'say-hi' is being served and polling for scheduled runs!

To trigger a run for this flow, use the following command:

      $ prefect deployment run 'say-hi/my-deployment'

Convenient & Secure

Choose your own infrastructure

Configure your execution environment, all the way down to the individual @flow.

With work pools and work queues, you have granular control over your infrastructure.

Learn More

flow.py

1# flow.py2import os
3from prefect import flow
4
5@flow6def main():7    if os.cpu_count() <= 2:
8        print("💪")
9    else:
10        print("💪💪💪")
# Deploy to Kubernetesprefect deploy flow.py:main --pool kubernetes

# Deploy to staging serverprefect deploy flow.py:main --pool process-staging

Total Visibility

See Your Whole Stack

Look inside your pipelines with events from any third party tool.

Try Prefect Cloud

What developers are saying

Braun Reyes

Saving us days on DAG design vs. Airflow

Ben Doremus

Our critical bug rate has dropped by 65% since we’ve implemented Prefect

Lee Mendelowitz

Prefect Core provides a really nice, clean set of easy to use features for constructing data pipelines (Tasks, Flows, States, Results) and it’s easy to model the dependencies between tasks.

Chas DeVeas

We used to spend 20% of our time (and up to 80% at times) on errors. With Prefect, we're approaching 5%, a 75% reduction.

NateYesterday

Okay, Prefect is officially awesome. I just launched the server locally and realized i didn’t see the whole picture from the docs. Thanks for building such an amazing product.

Madison Schott

Our previous architecture took 24 hours to run the models. With Prefect, it's 3.

bargava

  • Airflow has a steeper learning curve. Team struggled.
  • @PrefectIO is more Pythonic and team picked it up quite easily
  • Cloud version has a generous free-tier. Lesser friction to get started.

Andreas

Underrated @PrefectIO use-case of the day: ... observability and retries! Prefect is not only a "scheduler". By adding a flow/task decorator, you can add observability to your python scripts.

sshawnkimm

Hello, Currently scoping out using Prefect for our orchestration. Thanks for the great project. :)

Andreas

Invest heavily in Automation early on. If you do things twice, automate it, or at least document it. @PrefectIO is free and easy to set up, it`ll be the reliable backbone of your business automations

Stay in the flow

Email Address

Product

Resources

Solutions

About

© Copyright 2024 Prefect Technologies, Inc.

All rights reserved.