Command-line interface¶
Note
This is an introductory tutorial to the command line interface; for a complete API description, see: Command line interface.
Entry points¶
By default, the CLI looks for an pipeline.yaml
file in certain standard
locations (Default locations). If your pipeline exists in a
non-standard location, pass the --entry-point
argument.
The pipeline.yaml
file is known as “entry point”. However, this is
not the only type of entry point (See this guide to learn
more: Spec API vs. Python API).
Basic commands¶
Build pipeline (skips up-to-date tasks):
ploomber build
Forced build (runs all tasks, regardless of status):
ploomber build --force
Generate pipeline plot:
ploomber plot
New in Ploomber 0.18.2: You can plot the pipeline without installing extra dependencies.
pygraphviz
is still supported but optional. To learn more, see this.
Interactive sessions¶
Interactive sessions allow you to access the structure of your pipeline to help you test and debug:
ploomber interact
The command above starts a Python session, parses your pipeline, and exposes
a dag
variable (an instance of the ploomber.DAG
class).
For example, to generate the plot:
dag.plot()
Get task names:
list(dag)
You can also interact with specific tasks:
task = dag['task_name']
Tip: If using IPython or Jupyter, press Tab
to get autocompletion when
typing the task name: dag['some_task']
Get task’s product:
dag['some_task'].product
If the product is a dictionary:
dag['some_task'].product['product_name']
You can use this to avoid hardcoding paths to load products:
import pandas as pd
df = pd.read_csv(dag['some_task'].product)
If you are working with Python tasks (functions, scripts, or notebooks), you can start a line by line debugging session:
dag['some_task'].debug()
Enter quit
to exit the debugging session. Refer to
The Python Debugger
documentation for details.
To print the source code of a given task:
dag['some_task'].source
To find the source code location of a given task:
dag['some_task'].source.loc
Get upstream dependencies:
dag['some_task'].upstream
Get downstream tasks:
dag.get_downstream('some_task')
Other commands¶
Some commands didn’t cover here:
examples
: Download examplesinstall
: Install dependenciesnb
(short for notebook): Manage notebooks and scriptsreport
: Generate a pipeline reportscaffold
: Create a new projectstatus
: Pipeline status summarytask
: Execute a single task
See the CLI API documentation Command line interface for a detailed overview of each command.
Enabling Completion¶
To configure autocompletion for the CLI, you need to configure your shell.
If using bash, add this to ~/bashrc
:
eval "$(_PLOOMBER_COMPLETE=zsh_source ploomber)"
If using zsh, add this to ~/.zshrc
:
eval "$(_PLOOMBER_COMPLETE=zsh_source ploomber)"
If using fish, add this to ~/.config/fish/completions/ploomber.fish
:
eval (env _PLOOMBER_COMPLETE=fish_source ploomber)