Exact Deduplication Ray-base KubeFlow Pipeline Transformation
Summary
This project allows execution of the noop Ray transform as a KubeFlow Pipeline
The detail pipeline is presented in the Simplest Transform pipeline tutorial
Compilation
In order to compile pipeline definitions run
from the directory. It creates a virtual environment (make workflow-venv) and after that compiles the pipeline definitions in the folder. The virtual environment is created once for all transformers.Note: the pipelines definitions can be compiled and executed on KFPv1 and KFPv2. Meantime, KFPv1 is our default. If you prefer KFPv2, please do the following:
The next steps are described in Deploying a pipeline and Executing pipeline and watching execution results