Roadmap

These are some of the features that we have in the pipeline, sorted by priority. Please help us prioritize this list; go to the related GitHub issue and comment on it.

  1. Improve code difference detection (i.e., detect when an imported function changes). (#111)

  2. Better feedback in Jupyter when failing to load the pipeline (we only display feedback to the Jupyter console). (#415)

  3. Better support for non-Jupyter editors like VSCode or PyCharm (the cell injection process only works on Jupyter). (#416)

  4. Give more visibility to the Python API: better documentation, examples, etc. (#417)

  5. Building online APIs from pipelines with scripts/notebooks (we only support exporting online APIs from pipelines with Python functions). (#418)

To send general feedback, open an issue or send us a message on Slack.

Ideas

These are some ideas we have that we haven’t prioritized yet.

  • A frontend Jupyter plug-in (e.g., to visualize execution status in real-time).

  • Autocompletion and linting in Jupyter when editing pipeline.yaml.

  • Automated pipeline testing.

  • Integration with data versioning tools such as LakeFS. (#414)

  • Expand integration with Google Cloud (we only support uploading to Cloud Storage).

  • Expand integration with AWS (we only support S3 and AWS Batch).

  • Integration with Azure Machine Learning services.

  • Support for Julia.

To send general feedback, open an issue or send us a message on Slack.