I launched Cloud-Native Surroundings Company DEM information!
On the FOSS4G UK 2026 convention in October, I gave a chat saying that I’d re-released the Surroundings Company DEM information for England in cloud-native codecs. This was masking some work I did for my shopper Rebalance Earth and that we’ve determined to launch brazenly.
If you happen to already know what I’m speaking about then you’ll find the information on Supply Cooperative as COGs with STAC indexes. The README on that web page will clarify the way to entry the information, and you may entry an interactive webmap of the information too. If that didn’t make full sense to you, then learn on…
I’m not going to enter full particulars right here, as someday quickly the video recording of my speak on the convention ought to be launched, and I can hyperlink to that right here. Within the meantime, my slides can be found and should assist with understanding a few of what I clarify beneath.
So, mainly, the Surroundings Company (EA) gather LiDAR information throughout England and course of it to provide Digital Elevation Mannequin (DEMs). DEMs are rasters the place each pixel comprises a top worth, and the EA produce two sorts: a Digital Terrain Mannequin which is simply the peak of the terrain, and a Digital Floor Mannequin which incorporates the peak of every thing on high of the bottom – like buildings, bushes and so forth.
The EA present entry to this information beneath the Open Authorities License, however their internet interface for getting maintain of the information is a little bit of a ache – you’ll be able to’t obtain a lot information without delay. You’ll be able to ask for bulk entry to the information, however even then you must work out precisely what OS 5x5km grid squares you need, and the entire thing is a ache.
As the information is launched beneath the OGL, persons are allowed to re-release processed variations of it beneath the identical license. So, I transformed the recordsdata to fashionable cloud-native codecs, particularly Cloud Optimized GeoTIFF with a Spatio-Temporal Asset Catalog. These codecs retailer the information effectively, permitting small components of the picture to be simply downloaded, in addition to ‘zoomed-out’ lower-resolution variations of the information.
The upshot of all of that is that you should utilize a number of traces of Python or a GDAL command-line to obtain and mosaic the DEM over any space you select, and you may as well show the information simply on an online map. The latter is sort of spectacular – you’ll be able to see an instance webmap right here, which does dwell rescaling within the browser, together with hillshading and color mixing, producing a far nicer output than the EA’s viewer – the instance beneath is the DSM over Bristol (the situation of subsequent 12 months’s FOSS4G worldwide convention!)
I’ll go away the detailed documentation on the way to use this to the README file within the information repository, however a number of fast examples will present how simple it’s to make use of.
You should use a command like this to obtain and mosaic all the DSM information in a specific bounding field:
gdal raster clip
--bbox 438000,108000,446000,116000
GTI:/vsicurl/https://information.supply.coop/rebalance-earth/ea-dem/ea-lidar-fz-dsm-1m-items.parquet output.tifYou can even do equally in Python utilizing the lazycogs library:
import lazycogs
da = lazycogs.open(
“objects.parquet”,
bbox=lonlat_to_bng_bbox(bbox_lonlat),
crs="EPSG:27700",
decision=1.0,
chunks=chunks)
da = da.squeeze(["band", "time"], drop=True)which provides you a pleasant xarray DataArray to play with.
The speak went down very nicely, and was described by some individuals because the “spotlight of the convention” (together with Alastair Graham on the Geomob Podcast). The information has additionally been downloaded quite a bit for the reason that convention – there have been over 1 million particular person downloads, with over 4TB of knowledge transferred. Given that every tile of DEM information is simply round 80MB in measurement then that’s the equal of half 1,000,000 full tiles (the entire of England is simply round 5,500 tiles).
I’m fairly satisfied with this – it’s one thing I’ve needed to do for ages however by no means fairly received spherical to, till we discovered that my shopper Rebalance Earth wanted one thing like this, and determined we might launch it brazenly. I wish to thank Rebalance Earth for paying me to work on this (alongside numerous different stuff) and masking my prices of attending the convention to current it.
If you happen to discovered this publish helpful, please contemplate shopping for me a espresso.
This publish initially appeared on Robin’s Weblog.
Categorised as: Educational, GIS, Programming, Python, Distant Sensing
