Releases: AD4GD/pilot-2-preprocessing
Data4Land v1.1.1, stable Notebook mode
Overview
This release marks the Notebook version of the Data4Land tool, which provides main functionalities to enrich land-use/land-cover (LULC) datasets with other vector data, for example, OpenStreetMap database and World Database on Protected Areas.
Functionality
This tool includes four Jupyter Notebooks, which can be optional depending on purposes. These Notebooks are mainly developed to apply to the AD4GD project (biodiversity pilot) to enhance the calculation process of habitat connectivity. Interactive execution of these Jupyter Notebooks can be done fon user machines with stable network connection as its uses REST APIs to fetch new datasets.
The key changes compared to the v1.0.0 are:
- documentation supplied with the user guide and requirements for running each Notebook in the README file
- fixed bug with skipped requests to OpenStreetMap database in the 2nd Notebook
- added filters on bridges and tunnels to query OpenStreetMap features (roads, railways, waterways) as they do not act as ecological barriers for habitat connectivity (2nd Notebook)
- fixed bug with ignored manual edits of the configuration impedance file, which were not updated in the 4th Notebook
- added an example of configured parameters of decay and lambda/k-value for the case study of Northern England here
- supplied with output data which are produced by Data4Land with the sample input data - see Zenodo
Documentation
README file includes brief descriptions of each Jupyter Notebook, tool's diagram and installation instructions. A user guide has been added to the documentation, but all explicit descriptions of input/output data and its requirements are also given in the corresponding Notebooks:
- Access to historical data from the World Database on Protected Areas (WDPA) and harmonization
- Access and harmonisation of historical vector data on land-use/land-cover (LULC) - Open Street Map (OSM) data
- Enrichment of land-use/land-cover (LULC) data
- Impedance calculation ('edge effect' of biodiversity stressors)
Known Issues
All issues are currently being gathered from separate blocks of this tool, but the most notorious are listed in the [Issues section] of the parent repo within the AD4GD project (https://github.com/AD4GD/pilot-2/issues).
Installation
There is no specific installation required, but user is expected to run Jupyter Notebooks through a Docker image (see instructions in README). This prevents potential issues with applying some of the libraries used by this tool, especially GDAL.
License
See the detailed description here.
Contributors
Thanks everyone who contributed to this release:
Full Changelog available here: v1.0.0...v1.1.1
Contributors
Assets 2
Data4Land v2.0.0, Command-Line Mode
Overview
This release provides the Data4Land tool with the command-line mode, which marks the significant changes in functionality and allows users to enrich land-use/land-cover (LULC) datasets in a more robust way.
Functionality
This tool includes four commands (or technical components) described in the README file. 1st and 4th commands (process-wdpa and recalc-impedance) are mainly developed to be applied to the AD4GD project (biodiversity pilot) to enhance the calculation of habitat connectivity. The process-osm and enrich-lulc commands can be used to enrich land-use/land-cover datasets for other purposes, for example, in spatial planning or verifying data on heat islands.
The key changes are:
- command-line mode
- implementation of the ohsome API along with Overpass Turbo API from version v1.0.0 to fetch OpenStreetMap data
- curated Overpass Turbo API queries (for example, excluding bridges and tunnels) and added query to fetch vineyards
- ability to run commands for series of input datasets covering multiple years
- increased scalability with two sample datasets of different spatial extent and support of multiple case studies and sub case studies
- fixed bugs with concatenation of data from the World Database on Protected Areas
- compression of output datasets with Cloud Optimised Geotiff (COG) option
Warning: users generally should have a stable network connection to run the 1st and 2nd commands as they use REST APIs to fetch data. However, if users do not need to fetch new data (for example, it is stored from previous sessions), they should specify --skip-fetch parameter, which would skip this step.
Warning: as in the previous version, the functionality of process-wdpa component is limited by the Protected Planet API authentication. Only registered users are free to use this command by pasting their token in the token key of the configuration file.
Documentation
README: includes brief descriptions of each command, parameter, possible values, usage examples, and the diagram illustrating the data flow within Data4Land.
Known issues
All known issues are gathered in the Issues section, because this software is part of the Pilot 2 technical components as a subrepository.
Installation
There is no specific installation required, but the user is expected to run main.py in command line through a Docker image (see instructions in README. This prevents potential issues with applying some of the libraries used by this tool, especially GDAL. Otherwise, certain libraries used by Data4Land may not function properly.
Multiple ways to configure and install Docker are available:
Windows, Linux, and macOS.
Once Docker is installed, the user needs to:
- download the software repository
- in the command line, run 'cd /path/data4land' (navigate to the working directory on local machine, replace the directory name if required)
- then run 'docker-compose up' (this will run all Docker commands and build a container)
Licence
See the detailed description here.
Acknowledgement
This software is the part of the AD4GD project, biodiversity pilot. The AD4GD project is co-funded by the European Union, Switzerland and the United Kingdom (UK Research and Innovation).
Thanks everyone who contributed to this release:
Full Changelog available here: v1.0.0...v2.0.0
Contributors
Assets 2
v1.0.0: Updated installation instruction
Overview
This release marks the initial version of the tool, which provides main functionalities to enrich land-use/land-cover (LULC) datasets with other vector data.
Functionality
This tool includes four Jupyter Notebooks, which can be optional depending on purposes. These Notebooks are mainly developed to apply to the AD4GD project (biodiversity pilot) to rectify the calculation process of habitat connectivity. Interactive execution of these Jupyter Notebooks can be done from user machines with stable network connection as its uses REST APIs to fetch new datasets.
Documentation
README: Readme files includes brief descriptions of each Jupyter Notebook, tool's diagram and installation instructions. All explicit descriptions of input/output data and its requirements are given in the corresponding Notebooks.
Known Issues
All issues are currently being gathered from separate blocks of this tool, but the most notorious are listed in the [Issues section] of the parent repo within the AD4GD project (https://github.com/AD4GD/pilot-2/issues).
Installation
There is no specific installation required, but user is expected to run Jupyter Notebooks through a Docker image (see instructions in README). This prevents potential issues with applying some of the libraries used by this tool, especially GDAL.
License
See the detailed description here.
Contributors
Thanks everyone who contributed to this release: