Spring initiatives usually are opinionated: 80-90% of use instances are dealt with “by default”, and code is usually rather more concise than could be required in any other case on account of Spring’s desire of conference over configuration. These and different “opinions” can lead to dramatically much less code to put in writing and keep and in consequence, extra centered impression.
Within the overwhelming majority of instances the place Azure Storage is used from an utility, there isn’t a compelling benefit to utilizing greater than a single Azure storage account. However there are edge instances, and being able to make use of a number of Azure Storage accounts from a single app – even when we would solely want that functionality round 10% of the time – may present an extremely helpful extension of our storage superpowers.
This text is the results of a collaboration with Shi Li Chen.
It’s all about assets
The Spring Framework defines the Useful resource
interface and gives a number of implementations constructed upon Useful resource
to facilitate developer entry to low-level assets. With the intention to deal with a specific type of useful resource, two issues are required:
- A
Useful resource
implementation - A
ResourcePatternResolver
implementation
A Spring utility evaluates assets in query utilizing a number of registered resolvers. When the kind of useful resource is recognized, the suitable Useful resource
implementation is used to entry and/or manipulate the underlying useful resource.
If the implementations constructed into Spring Framework don’t fulfill your use case, it’s pretty simple so as to add assist for added kinds of assets by defining your individual implementations of AbstractResource
and ResourcePatternResolver
interfaces.
This text will introduce the Spring Useful resource, assessment Spring Cloud Azure’s implementation of Spring’s Useful resource
(particularly with regard to Azure Storage Account concerns and limitations), and think about how you can increase stated implementation to deal with these edge instances during which it might be helpful to entry a number of Azure Storage Accounts from a single Spring Boot utility.
Getting resourceful
We’ve already talked about that the Spring Framework defines a number of helpful Useful resource implementations. As of this writing, the default sorts are:
UrlResource
ClassPathResource
FileSystemResource
PathResource
ServletContextResource
InputStreamResource
ByteArrayResource
As talked about earlier, every useful resource could have a corresponding useful resource resolver.
Enabling your Spring Boot utility to make use of a customized Useful resource
requires the next actions:
- Implement the
Useful resource
interface by extendingAbstractResource
- Implement the
ResourcePatternResolver
interface to resolve the customized useful resource kind - Register the implementation of
ResourcePatternResolver
as a bean
NOTE: Your resolver should be added to the default useful resource loader’s resolver set utilizing the org.springframework.core.io.DefaultResourceLoader#addProtocolResolver
methodology, however this code is current in AbstractAzureStorageProtocolResolver
; extending that class to create your implementation accomplishes this in your behalf except you select to override its setResourceLoader
methodology.
A ResourceLoader
makes an attempt to resolve every Useful resource
by evaluating its outlined location/format with all registered protocol sample resolvers till a non-null useful resource is returned. If no match is discovered, the Useful resource
can be evaluated in opposition to Spring’s built-in sample resolvers.
Spring assets in Spring Cloud Azure
Spring Cloud Azure gives two Spring useful resource and useful resource sample resolver implementations. On this article, we solely talk about the implementation of the Azure Storage Blob useful resource. You may study the supply code for Spring Cloud Azure Sources
at Spring Cloud Azure and associated documentation at Useful resource Dealing with.
NOTE: We use Spring Cloud Azure Starter Storage Blob model 4.2.0 for evaluation and experiments.
Implementation of AbstractResource
The summary implementation AzureStorageResource
for Spring Cloud Azure primarily defines the format of the Azure storage useful resource protocol and accommodates the distinctive attributes of the Azure Storage Account service, e.g. the container identify and file identify. It is very important notice that AzureStorageResource
is decoupled from the Azure Storage SDK.
The Spring Framework interface WritableResource
represents the underlying API we construct upon to learn from and write to the Azure Storage useful resource.
summary class AzureStorageResource extends AbstractResource implements WritableResource { non-public boolean isAzureStorageResource(@NonNull String location) { ...... } String getContainerName(String location) { ...... } String getContentType(String location) { ...... } String getFilename(String location) { ...... } summary StorageType getStorageType(); }
The StorageBlobResource
is Spring Cloud Azure Storage Blob’s implementation of the summary class AbstractResource
. We will see StorageBlobResource
makes use of the BlobServiceClient
from the Azure Storage Blob SDK to implement all summary strategies, counting on the service consumer to work together with the Azure Storage Blob service.
public ultimate class StorageBlobResource extends AzureStorageResource { non-public ultimate BlobServiceClient blobServiceClient; non-public ultimate BlobContainerClient blobContainerClient; non-public ultimate BlockBlobClient blockBlobClient; public StorageBlobResource(BlobServiceClient blobServiceClient, String location, Boolean autoCreateFiles, String snapshot, String versionId, String contentType) { ...... this.blobContainerClient = blobServiceClient.getBlobContainerClient(getContainerName(location)); BlobClient blobClient = blobContainerClient.getBlobClient(getFilename(location)); this.blockBlobClient = blobClient.getBlockBlobClient(); } @Override public OutputStream getOutputStream() throws IOException { strive { ...... return this.blockBlobClient.getBlobOutputStream(choices); } catch (BlobStorageException e) { throw new IOException(MSG_FAIL_OPEN_OUTPUT, e); } } ...... @Override StorageType getStorageType() { return StorageType.BLOB; } }
Implementation of ResourcePatternResolver
Spring Cloud Azure gives an summary implementation AbstractAzureStorageProtocolResolver
. This class incorporates basic processing of the Azure storage useful resource protocol, exposes particular capabilities of the Azure Storage Account service, and provides the requisite logic to the default useful resource loader. Like AzureStorageResource
, the AbstractAzureStorageProtocolResolver
can also be not coupled to the Azure Storage SDK.
public summary class AbstractAzureStorageProtocolResolver implements ProtocolResolver, ResourcePatternResolver, ResourceLoaderAware, BeanFactoryPostProcessor { protected ultimate AntPathMatcher matcher = new AntPathMatcher(); protected summary StorageType getStorageType(); protected summary Useful resource getStorageResource(String location, Boolean autoCreate); protected ConfigurableListableBeanFactory beanFactory; protected summary Stream<StorageContainerItem> listStorageContainers(String containerPrefix); protected summary StorageContainerClient getStorageContainerClient(String identify); @Override public void setResourceLoader(ResourceLoader resourceLoader) { if (resourceLoader instanceof DefaultResourceLoader) { ((DefaultResourceLoader) resourceLoader).addProtocolResolver(this); } else { LOGGER.warn("Customized Protocol utilizing azure-{}:// prefix is not going to be enabled.", getStorageType().getType()); } } @Override public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException { this.beanFactory = beanFactory; } @Override public Useful resource resolve(String location, ResourceLoader resourceLoader) { if (AzureStorageUtils.isAzureStorageResource(location, getStorageType())) { return getResource(location); } return null; } @Override public Useful resource[] getResources(String sample) throws IOException { Useful resource[] assets = null; if (AzureStorageUtils.isAzureStorageResource(sample, getStorageType())) { if (matcher.isPattern(AzureStorageUtils.stripProtocol(sample, getStorageType()))) { String containerPattern = AzureStorageUtils.getContainerName(sample, getStorageType()); String filePattern = AzureStorageUtils.getFilename(sample, getStorageType()); assets = resolveResources(containerPattern, filePattern); } else { return new Useful resource[] { getResource(sample) }; } } if (null == assets) { throw new IOException("Sources not discovered at " + sample); } return assets; } @Override public Useful resource getResource(String location) { Useful resource useful resource = null; if (AzureStorageUtils.isAzureStorageResource(location, getStorageType())) { useful resource = getStorageResource(location, true); } if (null == useful resource) { throw new IllegalArgumentException("Useful resource not discovered at " + location); } return useful resource; } /** * Storage container merchandise. */ protected static class StorageContainerItem { non-public ultimate String identify; ...... } protected static class StorageItem { non-public ultimate String container; non-public ultimate String identify; non-public ultimate StorageType storageType; ...... } protected interface StorageContainerClient { ...... } }
The useful resource resolver AzureStorageBlobProtocolResolver
is Spring Cloud Azure Storage Blob’s implementation of ResourcePatternResolver
. It encapsulates assets in accordance with the situation or storage merchandise sample primarily based on BlobServiceClient
and returns the related StorageBlobResource
.
public ultimate class AzureStorageBlobProtocolResolver extends AbstractAzureStorageProtocolResolver { non-public BlobServiceClient blobServiceClient; @Override protected StorageType getStorageType() { return StorageType.BLOB; } @Override protected Useful resource getStorageResource(String location, Boolean autoCreate) { return new StorageBlobResource(getBlobServiceClient(), location, autoCreate); } non-public BlobServiceClient getBlobServiceClient() { if (blobServiceClient == null) { blobServiceClient = beanFactory.getBean(BlobServiceClient.class); } return blobServiceClient; } }
Opinions
As talked about at first of this publish, the default capabilities fulfill the necessities admirably within the overwhelming majority of circumstances. However in accordance with the Spring ethos, Spring Cloud Azure Starter Storage Blob was designed to seamlessly deal with 80-90% of use instances “out of the field”, whereas nonetheless permitting for remaining (edge) instances with some additional effort.
As written, the storage blob useful resource helps a number of container operations utilizing the identical storage account. The salient level is that the blob paths below totally different containers may be correctly resolved into StorageBlobResource
objects. Combining the sooner code for StorageBlobResource
, the blob useful resource should maintain a blob service consumer, and if blobServiceClient.getBlobContainerClient(getContainerName(location))
efficiently returns a BlobServiceClient
, the blob useful resource may be resolved and retrieved.
The BlobServiceClient
bean represents an Azure Storage Account within the Azure Storage Blob SDK, that means that the present implementation doesn’t assist simultaneous availability utilizing a number of Azure Storage Accounts.
Creating an prolonged model of Spring Cloud Azure Starter Storage Blob
For these uncommon instances during which it is likely to be helpful to concurrently entry a number of Azure Storage accounts from the identical utility, there’s a technique to make that occur. To reveal this functionality, let’s create a brand new library referred to as spring-cloud-azure-starter-storage-blob-extend
. The one exterior dependency for this new library is the prevailing spring-cloud-azure-starter-storage-blob
.
Prolong the Storage Blob properties
Whereas the first purpose is to assist a number of storage accounts, a secondary design purpose is to make use of an identical construction to AzureStorageBlobProperties
with a view to decrease the training curve and to retain Spring Cloud Azure 4.0’s out of the field authentication options.
public class ExtendAzureStorageBlobsProperties { public static ultimate String PREFIX = "spring.cloud.azure.storage.blobs"; non-public boolean enabled = true; non-public ultimate Listing<AzureStorageBlobProperties> configurations = new ArrayList<>(); public boolean isEnabled() { return enabled; } public void setEnabled(boolean enabled) { this.enabled = enabled; } public Listing<AzureStorageBlobProperties> getConfigurations() { return configurations; } }
Dynamically register Storage Blob beans
Since there can be a number of Storage Account configurations, we should identify the beans corresponding to every storage account. The cleanest strategy is to easily use the account identify because the bean identify.
Now, let’s dynamically register these beans with the Spring context.
@Configuration(proxyBeanMethods = false) @ConditionalOnProperty(worth = { "spring.cloud.azure.storage.blobs.enabled"}, havingValue = "true") public class ExtendStorageBlobsAutoConfiguration implements BeanDefinitionRegistryPostProcessor, EnvironmentAware { non-public Surroundings atmosphere; public static ultimate String EXTEND_STORAGE_BLOB_PROPERTIES_BEAN_NAME = "extendAzureStorageBlobsProperties"; @Override public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException { AzureGlobalProperties azureGlobalProperties = Binder.get(atmosphere) .bind(AzureGlobalProperties.PREFIX, AzureGlobalProperties.class) .orElse(new AzureGlobalProperties()); ExtendAzureStorageBlobsProperties blobsProperties = Binder.get(atmosphere) .bind(ExtendAzureStorageBlobsProperties.PREFIX, ExtendAzureStorageBlobsProperties.class) .orElseThrow(() -> new IllegalArgumentException("Can't bind the azure storage blobs properties.")); // merge properties for (AzureStorageBlobProperties azureStorageBlobProperties : blobsProperties.getConfigurations()) { AzureStorageBlobProperties transProperties = new AzureStorageBlobProperties(); AzureGlobalPropertiesUtils.loadProperties(azureGlobalProperties, transProperties); copyAzureCommonPropertiesIgnoreTargetNull(transProperties, azureStorageBlobProperties); } DefaultListableBeanFactory manufacturing facility = (DefaultListableBeanFactory) beanFactory; registryBeanExtendAzureStorageBlobsProperties(manufacturing facility, blobsProperties); blobsProperties.getConfigurations().forEach(blobProperties -> registryBlobBeans(manufacturing facility, blobProperties)); } non-public void registryBeanExtendAzureStorageBlobsProperties(DefaultListableBeanFactory beanFactory, ExtendAzureStorageBlobsProperties blobsProperties) { BeanDefinitionBuilder beanDefinitionBuilder = BeanDefinitionBuilder.genericBeanDefinition(ExtendAzureStorageBlobsProperties.class, () -> blobsProperties); AbstractBeanDefinition rawBeanDefinition = beanDefinitionBuilder.getRawBeanDefinition(); beanFactory.registerBeanDefinition(EXTEND_STORAGE_BLOB_PROPERTIES_BEAN_NAME, rawBeanDefinition); } non-public void registryBlobBeans(DefaultListableBeanFactory beanFactory, AzureStorageBlobProperties blobProperties) { String accountName = getStorageAccountName(blobProperties); Assert.hasText(accountName, "accountName can't be null or empty."); registryBeanStaticConnectionStringProvider(beanFactory, blobProperties, accountName); registryBeanBlobServiceClientBuilderFactory(beanFactory, blobProperties, accountName); registryBeanBlobServiceClientBuilder(beanFactory, accountName); registryBeanBlobServiceClient(beanFactory, accountName); registryBeanBlobContainerClient(beanFactory, blobProperties, accountName); registryBeanBlobClient(beanFactory, blobProperties, accountName); } non-public void registryBeanBlobServiceClientBuilder(DefaultListableBeanFactory beanFactory, String accountName) { BeanDefinitionBuilder beanDefinitionBuilder = BeanDefinitionBuilder.genericBeanDefinition(BlobServiceClientBuilder.class, () -> { BlobServiceClientBuilderFactory builderFactory = beanFactory.getBean(accountName + BlobServiceClientBuilderFactory.class.getSimpleName(), BlobServiceClientBuilderFactory.class); return builderFactory.construct(); }); AbstractBeanDefinition rawBeanDefinition = beanDefinitionBuilder.getRawBeanDefinition(); beanFactory.registerBeanDefinition( accountName + BlobServiceClientBuilder.class.getSimpleName(), rawBeanDefinition); } ...... @Override public void setEnvironment(Surroundings atmosphere) { this.atmosphere = atmosphere; } }
Prolong the AzureStorageBlobProtocolResolver
The subsequent activity is to make any container resolvable by the identical useful resource sample resolver. Specifying a storage blob useful resource location similar to azure-blob-accountname://containername/take a look at.txt, the resolver will use that to find the suitable BlobServiceClient
bean by Azure Storage Account identify and return the storage useful resource.
public class ExtendAzureStorageBlobProtocolResolver extends ExtendAbstractAzureStorageProtocolResolver { non-public ultimate Map<String, BlobServiceClient> blobServiceClientMap = new HashMap<>(); @Override protected Useful resource getStorageResource(String location, Boolean autoCreate) { return new ExtendStorageBlobResource(getBlobServiceClient(location), location, autoCreate); } non-public BlobServiceClient getBlobServiceClient(String locationPrefix) { String storageAccount = ExtendAzureStorageUtils.getStorageAccountName(locationPrefix, getStorageType()); Assert.notNull(storageAccount, "storageAccount can't be null."); String accountKey = storageAccount.toLowerCase(Locale.ROOT); if (blobServiceClientMap.containsKey(accountKey)) { return blobServiceClientMap.get(accountKey); } BlobServiceClient blobServiceClient = beanFactory.getBean( accountKey + BlobServiceClient.class.getSimpleName(), BlobServiceClient.class); Assert.notNull(blobServiceClient, "blobServiceClient can't be null."); blobServiceClientMap.put(accountKey, blobServiceClient); return blobServiceClient; } }
Once more, it’s essential to add the bean ExtendAzureStorageBlobProtocolResolver
to the Spring context.
Testing the Spring Cloud Azure Starter Storage Blob Prolong
You should use begin.spring.io to generate a Spring Boot 2.6.7 or higher challenge with Azure Storage assist (or construct on this storage blob pattern in case you choose).
Add the extending starter dependency to the pom.xml file:
<dependency> <groupId>com.azure.spring.prolong</groupId> <artifactId>spring-cloud-azure-starter-storage-blob-extend</artifactId> <model>1.0-SNAPSHOT</model> </dependency>
Delete the src/most important/assets/utility.properties file or add the next configuration file application-extend.yml, which allows a number of storage account utilization:
application-extend.yml
spring: cloud: azure: storage: blob: enabled: false blobs: enabled: true configurations: - account-name: ${FIRST_ACCOUNT} container-name: ${FIRST_CONTAINER} account-key: ${ACCOUNT_KEY_OF_FIRST_ACCOUNT} - account-name: ${SECOND_ACCOUNT} container-name: ${SECOND_CONTAINER} account-key: ${ACCOUNT_KEY_OF_SECOND_ACCOUNT}
NOTE: You could present values for the atmosphere variables above (listed in all capital letters) with energetic Azure Storage Account useful resource data.
Add class com.azure.spring.prolong.pattern.storage.useful resource.prolong.SampleDataInitializer
with the next physique:
@Profile("prolong") @Element public class SampleDataInitializer implements CommandLineRunner { ultimate static Logger logger = LoggerFactory.getLogger(SampleDataInitializer.class); non-public ultimate ConfigurableEnvironment env; non-public ultimate ExtendAzureStorageBlobProtocolResolver resolver; non-public ultimate ExtendAzureStorageBlobsProperties properties; public SampleDataInitializer(ConfigurableEnvironment env, ExtendAzureStorageBlobProtocolResolver resolver, ExtendAzureStorageBlobsProperties properties) { this.env = env; this.resolver = resolver; this.properties = properties; } /** * That is used to initialize some information for every Azure Storage Account Blob container. */ @Override public void run(String... args) { properties.getConfigurations().forEach(this::writeDataByStorageAccount); } non-public void writeDataByStorageAccount(AzureStorageBlobProperties blobProperties) { String containerName = blobProperties.getContainerName(); if (!StringUtils.hasText(containerName) || blobProperties.getAccountName() == null) { return; } String accountName = getStorageAccountName(blobProperties); logger.data("Start to initialize the {} container of the {} account", containerName, accountName); lengthy currentTimeMillis = System.currentTimeMillis(); String fileName = "fileName-" + currentTimeMillis; String information = "information" + currentTimeMillis; Useful resource storageBlobResource = resolver.getResource("azure-blob-" + accountName + "://" + containerName +"/" + fileName + ".txt"); strive (OutputStream os = ((WritableResource) storageBlobResource).getOutputStream()) { os.write(information.getBytes()); logger.data("Write information to container={}, fileName={}.txt", containerName, fileName); } catch (IOException e) { logger.error("Write information exception", e); } logger.data("Finish to initialize the {} container of the {} account", containerName, accountName); } }
Run the pattern with following Maven command:
mvn clear spring-boot:run -Dspring-boot.run.profiles=prolong
Lastly, confirm the anticipated final result. Your console ought to show the next output:
c.a.s.e.s.s.r.e.SampleDataInitializer : Start to initialize the container first of the account firstaccount. c.a.s.e.s.s.r.e.SampleDataInitializer : Write information to container=first, fileName=fileName-1656641340271.txt c.a.s.e.s.s.r.e.SampleDataInitializer : Finish to initialize the container first of the account firstaccount. c.a.s.e.s.s.r.e.SampleDataInitializer : Start to initialize the container second of the account secondaccount. c.a.s.e.s.s.r.e.SampleDataInitializer : Write information to container=second, fileName=fileName-1656641343572.txt c.a.s.e.s.s.r.e.SampleDataInitializer : Finish to initialize the container second of the account secondaccount.
All pattern challenge code is revealed on the repository spring-cloud-azure-starter-storage-blob-extend-sample.
Inside this prolonged utility, it’s nonetheless potential to revert to the unique, single storage account utilization of Spring Cloud Azure Starter Storage Blob by including the next configuration file application-current.yml:
spring: cloud: azure: storage: blob: account-name: ${FIRST_ACCOUNT} container-name: ${FIRST_CONTAINER} account-key: ${ACCOUNT_KEY_OF_FIRST_ACCOUNT} present: second-container: ${SECOND_CONTAINER}
NOTE: You could set or change the listed atmosphere variable assigned values with energetic Azure Storage Account useful resource data.
Run the pattern with following Maven command:
mvn clear spring-boot:run -Dspring-boot.run.profiles=present
To confirm right operation utilizing a single storage account, examine terminal output with that listed right here:
c.a.s.e.s.s.r.c.SampleDataInitializer : StorageApplication information initialization of 'first-container' start ... c.a.s.e.s.s.r.c.SampleDataInitializer : Write information to container=first-container, fileName=fileName1656641162614.txt c.a.s.e.s.s.r.c.SampleDataInitializer : StorageApplication information initialization of 'first-container' finish ... c.a.s.e.s.s.r.c.SampleDataInitializer : StorageApplication information initialization of 'second-container' start ... c.a.s.e.s.s.r.c.SampleDataInitializer : Write information to container=second-container, fileName=fileName1656641165411.txt c.a.s.e.s.s.r.c.SampleDataInitializer : StorageApplication information initialization of 'second-container' finish ...
Conclusion
Implementing a selected useful resource kind and corresponding sample resolver is comparatively easy, largely due to clear documentation, the various built-in implementations, frequent utilization inside the Spring know-how stack.
One level that warrants consideration is the protocol definition for the useful resource, e.g. the Azure Storage Blob Useful resource. We should notice whether or not we’re utilizing azure-blob:// or azure-blob-[account-name]:// and plan app capabilities accordingly. Moreover, because the identifier of a community useful resource should be uniquely identifiable, the latter location format might end in a for much longer identify and in addition exposes the identify of the storage account. These tradeoffs should be evaluated in gentle of necessities and threat profile.