Discover Quickstart
Find your top jobs to optimize as well as discover new opportunities to improve your efficiency even more. This page is refreshed daily so you always get up-to date insights and historical tracking.
Last updated
Find your top jobs to optimize as well as discover new opportunities to improve your efficiency even more. This page is refreshed daily so you always get up-to date insights and historical tracking.
Last updated
After logging into Gradient, click on the "Discover" link on the left hand navigation. Click on the "Add Workspace" button to bring up the credentials prompt, as seen below. You will need to enter the following information:
Databricks Workspace ID - Can be found in the address bar of your web browser in your Databricks URL. It is the "o" parameter "o=9172567527460388", so in this case you would enter the number "9172567527460388".
Databricks Host - Can be found in the address bar of your web browser when at your Databricks workspace. It should look like this: https://dbc-6c213588-2400.cloud.databricks.com/
Databricks Token - You will need Databricks admin access to generate a personal access token for your Databricks workspace. Copy and paste your token value here. We recommend setting the "Lifetime" field blank so the token does not expire and interrupt any service down the line.
Compute Provider - Select which cloud your Databricks workspace is run on
That's it! You're done!
The discover page displays various pieces gathered from your workspace of information as explained below. Each "Finding" widget on the right is clickable and will filter the jobs list based on the parameters of the finding.
Top jobs to optimize with Gradient
Jobs with Photon enabled
Jobs with Autoscaling enabled
All purpose compute jobs
Jobs submitted from run-submit (meaning they could come from an external orchestrator like Airflow)
Timeline of total number of jobs
Timeline of total core hours (proportional to your costs)
Identify the jobs you want to import into Gradient to start automatically optimizing their clusters. If this is your first time setting up Gradient, please complete your workspace setup. If you've already onboarded a workspace, proceed to project setup to directly import your jobs.
