^{Originally posted by kostrse January 4, 2023}
Each Azure datasource in Grafana has a predefined list of supported Azure clouds: Public, China and US Government.

Currently, Its impossible to use the existing Azure datasources for Azure Stack or other hybrid cloud scenarios because it’s not possible to configure a custom cloud environment.

The proposal is to address this limitation by introducing cloud configuration via Grafana settings and Grafana Azure SDK.

This would allow to uniformly support custom clouds across all Azure datasources.

Grafana config

First step would be to introduce a new optional property to the [azure] section of Grafana config, cloud_config:

[azure]
cloud = AzureCloud
...
cloud_config = ./clouds.json

This property should point to a JSON file with definitions of all available clouds. If this property isn’t set, then the built-in list of clouds should be initialized (Public, China, US Gov), same as the existing behavior.

The JSON file would have the same structure as returned by Azure CLI:

az cloud list --output json > ./clouds.json

Example of such cloud config (returned by az cloud list):

[
  {
    "name": "AzureCloud",
    "isActive": true,
    "profile": "latest",
    "endpoints": {
      "activeDirectory", "https://login.microsoftonline.com",
      "portal", "https://portal.azure.com",
      "...": "..."
    },
    "suffixes": {
      "...": "..."
    },
  },
  "...": "..."
]

It has a name property which is a cloud identifier, same us currently used for built-in clouds (here) and two key/value bags endpoints and suffixes. Properties isActive and profile are not relevant for the current use case.

Path to the clouds file should be relative to the location of the grafana.ini config ($WORKING_DIR/conf).

New API in Grafana Azure SDK

Here are new structures and functions proposed for the Grafana Azure SDK:

type azcloud.AzureCloud struct {
	Name        string
	DisplayName string
}

type azcloud.AzureCloudDef struct {
	Name        string
	DisplayName string
	Endpoints   map[string]string
	Suffixes    map[string]string
}

func azcloud.InitDefaultClouds() error
func azcloud.InitFromConfig(configFile string) error

func azcloud.GetClouds() ([]AzureCloud, error)

func azcloud.GetCloud(azureCloud string) (*AzureCloudDef, error)

First, clouds configuration has to be initialized at startup using either InitDefaultClouds()  or InitFromConfig(configFile) . It needs to be called by Grafana host during initialization of settings (here) or by external plugin in its main function.

Once initialized, the list of clouds stored in memory as a singleton object and available via the following API:
GetClouds() returns a list of all available clouds and GetCloud(azureCloud) returns configuration of the given cloud.

These functions can be called by business logic of datasources to resolve cloud-specific resource endpoints and also by Grafana Azure SDK itself to resolve the AAD endpoint for authentication (e.g. here).

List of available clouds in frontend UI

Since actual authentication performed on backend, frontend only needs a list of cloud names to display in datasource configuration UI.

Grafana host can pass the list of clouds to frontend via the frontendsettings (here):

jsonObj := map[string]interface{}{
	...
	"azure": map[string]interface{}{
		"cloud":  hs.Cfg.Azure.Cloud,
		...
		"clouds": azcloud.GetCouds(),
	},
}

Frontend doesn’t need all cloud configuration properties, only cloud names.

Access to clouds configuration from external plugin backend

Grafana passes configuration from the host to external plugins via environment variables (here).
Clouds config is a large structure, it could be problematic to pass it via environment variables.
Currently, it’s not clear what is the best solution, but one would be to pass absolute path to the JSON file, so external plugin could read and parse it by itself.

Missing endpoints and extensibility of the clouds config JSON

The idea to reuse the existing format of clouds config from az cloud list is to avoid inventing own format and make it simpler to write deployment scripts which could reuse naming of properties etc.

It’s possible that the original cloud config from az cloud list doesn’t contain all needed properties, so it can be extended as needed, while reusing existing properties which already available.

Here are some properties which are missing in the original config and would need to be introduced: