Integrating with Gradient

The Sync Python library provides developers with the fundamental tools to integrate the Sync continuous optimization in a number of ways.

Integration Guides

Pipeline integration is an essential step in ensuring you receive the best recommendations for each of your Spark jobs. Gradient uses the data from each pipeline execution to train models that unlock advanced optimizations only possible with current and historical metrics.

Integration process overview

Currently, Gradient supports integrations with Databricks Workflows and Apache Airflow. If you would like to integrate with a different orchestration tool, contact the Sync Customer Solutions team for assistance.