A Concise Information to Utilizing A number of Azure Storage Accounts from a Single Spring Boot Utility

The summary implementation AzureStorageResource for Spring Cloud Azure primarily defines the format of the Azure storage useful resource protocol and accommodates the distinctive attributes of the Azure Storage Account service, e.g. the container identify and file identify. It is very important notice that AzureStorageResource is decoupled from the Azure Storage SDK.

The Spring Framework interface WritableResource represents the underlying API we construct upon to learn from and write to the Azure Storage useful resource.

summary class AzureStorageResource extends AbstractResource implements WritableResource {

    non-public boolean isAzureStorageResource(@NonNull String location) {
        ......
    }

    String getContainerName(String location) {
        ......
    }

    String getContentType(String location) {
        ......
    }

    String getFilename(String location) {
        ......
    }

    summary StorageType getStorageType();
}

The StorageBlobResource is Spring Cloud Azure Storage Blob’s implementation of the summary class AbstractResource. We will see StorageBlobResource makes use of the BlobServiceClient from the Azure Storage Blob SDK to implement all summary strategies, counting on the service consumer to work together with the Azure Storage Blob service.

public ultimate class StorageBlobResource extends AzureStorageResource {
    non-public ultimate BlobServiceClient blobServiceClient;
    non-public ultimate BlobContainerClient blobContainerClient;
    non-public ultimate BlockBlobClient blockBlobClient;

    public StorageBlobResource(BlobServiceClient blobServiceClient, String location, Boolean autoCreateFiles,
                               String snapshot, String versionId, String contentType) {
        ......
        this.blobContainerClient = blobServiceClient.getBlobContainerClient(getContainerName(location));
        BlobClient blobClient = blobContainerClient.getBlobClient(getFilename(location));
        this.blockBlobClient = blobClient.getBlockBlobClient();
    }

    @Override
    public OutputStream getOutputStream() throws IOException {
        strive {
            ......
            return this.blockBlobClient.getBlobOutputStream(choices);
        } catch (BlobStorageException e) {
            throw new IOException(MSG_FAIL_OPEN_OUTPUT, e);
        }
    }

    ......

    @Override
    StorageType getStorageType() {
        return StorageType.BLOB;
    }
}

Implementation of ResourcePatternResolver

Spring Cloud Azure gives an summary implementation AbstractAzureStorageProtocolResolver. This class incorporates basic processing of the Azure storage useful resource protocol, exposes particular capabilities of the Azure Storage Account service, and provides the requisite logic to the default useful resource loader. Like AzureStorageResource, the AbstractAzureStorageProtocolResolver can also be not coupled to the Azure Storage SDK.

public summary class AbstractAzureStorageProtocolResolver implements ProtocolResolver, ResourcePatternResolver,
    ResourceLoaderAware, BeanFactoryPostProcessor {

    protected ultimate AntPathMatcher matcher = new AntPathMatcher();

    protected summary StorageType getStorageType();

    protected summary Useful resource getStorageResource(String location, Boolean autoCreate);

    protected ConfigurableListableBeanFactory beanFactory;

    protected summary Stream<StorageContainerItem> listStorageContainers(String containerPrefix);

    protected summary StorageContainerClient getStorageContainerClient(String identify);

    @Override
    public void setResourceLoader(ResourceLoader resourceLoader) {
        if (resourceLoader instanceof DefaultResourceLoader) {
            ((DefaultResourceLoader) resourceLoader).addProtocolResolver(this);
        } else {
            LOGGER.warn("Customized Protocol utilizing azure-{}:// prefix is not going to be enabled.", getStorageType().getType());
        }
    }

    @Override
    public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
        this.beanFactory = beanFactory;
    }

    @Override
    public Useful resource resolve(String location, ResourceLoader resourceLoader) {
        if (AzureStorageUtils.isAzureStorageResource(location, getStorageType())) {
            return getResource(location);
        }
        return null;
    }

    @Override
    public Useful resource[] getResources(String sample) throws IOException {
        Useful resource[] assets = null;

        if (AzureStorageUtils.isAzureStorageResource(sample, getStorageType())) {
            if (matcher.isPattern(AzureStorageUtils.stripProtocol(sample, getStorageType()))) {
                String containerPattern = AzureStorageUtils.getContainerName(sample, getStorageType());
                String filePattern = AzureStorageUtils.getFilename(sample, getStorageType());
                assets = resolveResources(containerPattern, filePattern);
            } else {
                return new Useful resource[] { getResource(sample) };
            }
        }
        if (null == assets) {
            throw new IOException("Sources not discovered at " + sample);
        }
        return assets;
    }

    @Override
    public Useful resource getResource(String location) {
        Useful resource useful resource = null;

        if (AzureStorageUtils.isAzureStorageResource(location, getStorageType())) {
            useful resource = getStorageResource(location, true);
        }

        if (null == useful resource) {
            throw new IllegalArgumentException("Useful resource not discovered at " + location);
        }
        return useful resource;
    }

    /**
     * Storage container merchandise.
     */
    protected static class StorageContainerItem {
        non-public ultimate String identify;
        ......
    }

    protected static class StorageItem {

        non-public ultimate String container;
        non-public ultimate String identify;
        non-public ultimate StorageType storageType;
        ......
    }

    protected interface StorageContainerClient {

        ......
    }
}

The useful resource resolver AzureStorageBlobProtocolResolver is Spring Cloud Azure Storage Blob’s implementation of ResourcePatternResolver. It encapsulates assets in accordance with the situation or storage merchandise sample primarily based on BlobServiceClient and returns the related StorageBlobResource.

public ultimate class AzureStorageBlobProtocolResolver extends AbstractAzureStorageProtocolResolver {

    non-public BlobServiceClient blobServiceClient;
    @Override
    protected StorageType getStorageType() {
        return StorageType.BLOB;
    }

    @Override
    protected Useful resource getStorageResource(String location, Boolean autoCreate) {
        return new StorageBlobResource(getBlobServiceClient(), location, autoCreate);
    }

    non-public BlobServiceClient getBlobServiceClient() {
        if (blobServiceClient == null) {
            blobServiceClient = beanFactory.getBean(BlobServiceClient.class);
        }
        return blobServiceClient;
    }
}

Opinions

As talked about at first of this publish, the default capabilities fulfill the necessities admirably within the overwhelming majority of circumstances. However in accordance with the Spring ethos, Spring Cloud Azure Starter Storage Blob was designed to seamlessly deal with 80-90% of use instances “out of the field”, whereas nonetheless permitting for remaining (edge) instances with some additional effort.

As written, the storage blob useful resource helps a number of container operations utilizing the identical storage account. The salient level is that the blob paths below totally different containers may be correctly resolved into StorageBlobResource objects. Combining the sooner code for StorageBlobResource, the blob useful resource should maintain a blob service consumer, and if blobServiceClient.getBlobContainerClient(getContainerName(location)) efficiently returns a BlobServiceClient, the blob useful resource may be resolved and retrieved.

The BlobServiceClient bean represents an Azure Storage Account within the Azure Storage Blob SDK, that means that the present implementation doesn’t assist simultaneous availability utilizing a number of Azure Storage Accounts.

For these uncommon instances during which it is likely to be helpful to concurrently entry a number of Azure Storage accounts from the identical utility, there’s a technique to make that occur. To reveal this functionality, let’s create a brand new library referred to as spring-cloud-azure-starter-storage-blob-extend. The one exterior dependency for this new library is the prevailing spring-cloud-azure-starter-storage-blob.

Whereas the first purpose is to assist a number of storage accounts, a secondary design purpose is to make use of an identical construction to AzureStorageBlobProperties with a view to decrease the training curve and to retain Spring Cloud Azure 4.0’s out of the field authentication options.

public class ExtendAzureStorageBlobsProperties {

    public static ultimate String PREFIX = "spring.cloud.azure.storage.blobs";

    non-public boolean enabled = true;

    non-public ultimate Listing<AzureStorageBlobProperties> configurations = new ArrayList<>();

    public boolean isEnabled() {
        return enabled;
    }

    public void setEnabled(boolean enabled) {
        this.enabled = enabled;
    }

    public Listing<AzureStorageBlobProperties> getConfigurations() {
        return configurations;
    }
}

Dynamically register Storage Blob beans

Since there can be a number of Storage Account configurations, we should identify the beans corresponding to every storage account. The cleanest strategy is to easily use the account identify because the bean identify.

Now, let’s dynamically register these beans with the Spring context.

@Configuration(proxyBeanMethods = false)
@ConditionalOnProperty(worth = { "spring.cloud.azure.storage.blobs.enabled"}, havingValue = "true")
public class ExtendStorageBlobsAutoConfiguration implements BeanDefinitionRegistryPostProcessor, EnvironmentAware {

    non-public Surroundings atmosphere;

    public static ultimate String EXTEND_STORAGE_BLOB_PROPERTIES_BEAN_NAME = "extendAzureStorageBlobsProperties";

    @Override
    public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
        AzureGlobalProperties azureGlobalProperties =
            Binder.get(atmosphere)
                  .bind(AzureGlobalProperties.PREFIX, AzureGlobalProperties.class)
                  .orElse(new AzureGlobalProperties());
        ExtendAzureStorageBlobsProperties blobsProperties =
            Binder.get(atmosphere)
                  .bind(ExtendAzureStorageBlobsProperties.PREFIX, ExtendAzureStorageBlobsProperties.class)
                  .orElseThrow(() -> new IllegalArgumentException("Can't bind the azure storage blobs properties."));
        // merge properties
        for (AzureStorageBlobProperties azureStorageBlobProperties : blobsProperties.getConfigurations()) {
            AzureStorageBlobProperties transProperties = new AzureStorageBlobProperties();
            AzureGlobalPropertiesUtils.loadProperties(azureGlobalProperties, transProperties);
            copyAzureCommonPropertiesIgnoreTargetNull(transProperties, azureStorageBlobProperties);
        }

        DefaultListableBeanFactory manufacturing facility = (DefaultListableBeanFactory) beanFactory;
        registryBeanExtendAzureStorageBlobsProperties(manufacturing facility, blobsProperties);
        blobsProperties.getConfigurations().forEach(blobProperties -> registryBlobBeans(manufacturing facility, blobProperties));
    }

    non-public void registryBeanExtendAzureStorageBlobsProperties(DefaultListableBeanFactory beanFactory,
                                                               ExtendAzureStorageBlobsProperties blobsProperties) {
        BeanDefinitionBuilder beanDefinitionBuilder = BeanDefinitionBuilder.genericBeanDefinition(ExtendAzureStorageBlobsProperties.class,
            () -> blobsProperties);
        AbstractBeanDefinition rawBeanDefinition = beanDefinitionBuilder.getRawBeanDefinition();
        beanFactory.registerBeanDefinition(EXTEND_STORAGE_BLOB_PROPERTIES_BEAN_NAME, rawBeanDefinition);
    }

    non-public void registryBlobBeans(DefaultListableBeanFactory beanFactory, AzureStorageBlobProperties blobProperties) {
        String accountName = getStorageAccountName(blobProperties);
        Assert.hasText(accountName, "accountName can't be null or empty.");
        registryBeanStaticConnectionStringProvider(beanFactory, blobProperties, accountName);
        registryBeanBlobServiceClientBuilderFactory(beanFactory, blobProperties, accountName);
        registryBeanBlobServiceClientBuilder(beanFactory, accountName);
        registryBeanBlobServiceClient(beanFactory, accountName);
        registryBeanBlobContainerClient(beanFactory, blobProperties, accountName);
        registryBeanBlobClient(beanFactory, blobProperties, accountName);
    }

    non-public void registryBeanBlobServiceClientBuilder(DefaultListableBeanFactory beanFactory,
                                                      String accountName) {
        BeanDefinitionBuilder beanDefinitionBuilder = BeanDefinitionBuilder.genericBeanDefinition(BlobServiceClientBuilder.class,
            () -> {
                BlobServiceClientBuilderFactory builderFactory =
                    beanFactory.getBean(accountName + BlobServiceClientBuilderFactory.class.getSimpleName(),
                        BlobServiceClientBuilderFactory.class);
                return builderFactory.construct();
            });
        AbstractBeanDefinition rawBeanDefinition = beanDefinitionBuilder.getRawBeanDefinition();
        beanFactory.registerBeanDefinition(
            accountName + BlobServiceClientBuilder.class.getSimpleName(), rawBeanDefinition);
    }

    ......

    @Override
    public void setEnvironment(Surroundings atmosphere) {
        this.atmosphere = atmosphere;
    }
}

Prolong the AzureStorageBlobProtocolResolver

The subsequent activity is to make any container resolvable by the identical useful resource sample resolver. Specifying a storage blob useful resource location similar to azure-blob-accountname://containername/take a look at.txt, the resolver will use that to find the suitable BlobServiceClient bean by Azure Storage Account identify and return the storage useful resource.

public class ExtendAzureStorageBlobProtocolResolver extends ExtendAbstractAzureStorageProtocolResolver {
    non-public ultimate Map<String, BlobServiceClient> blobServiceClientMap = new HashMap<>();

    @Override
    protected Useful resource getStorageResource(String location, Boolean autoCreate) {
        return new ExtendStorageBlobResource(getBlobServiceClient(location), location, autoCreate);
    }

    non-public BlobServiceClient getBlobServiceClient(String locationPrefix) {
        String storageAccount = ExtendAzureStorageUtils.getStorageAccountName(locationPrefix, getStorageType());
        Assert.notNull(storageAccount, "storageAccount can't be null.");
        String accountKey = storageAccount.toLowerCase(Locale.ROOT);
        if (blobServiceClientMap.containsKey(accountKey)) {
            return blobServiceClientMap.get(accountKey);
        }

        BlobServiceClient blobServiceClient = beanFactory.getBean(
            accountKey + BlobServiceClient.class.getSimpleName(), BlobServiceClient.class);
        Assert.notNull(blobServiceClient, "blobServiceClient can't be null.");
        blobServiceClientMap.put(accountKey, blobServiceClient);
        return blobServiceClient;
    }
}

Once more, it’s essential to add the bean ExtendAzureStorageBlobProtocolResolver to the Spring context.