The Analysis Hub includes several sets of ready to use tools for use in the Analysis Hub. These tools are provided in Python and come complete with source code and can be run in the Jupyter Hub environment or as a standalone command line tool.
| Tool Set | Description |
|---|---|
| Data Store | Tools for accessing the STAC Catalog in the data store. Creating, updating, or removing items in the Catalog. Searching for items using metadata, spatial, or time parameters. |
| LiDAR Processing | Tools for processing point cloud data (e.g LAS/LAZ files) including filters, PDAL integration, tiling, visualization, DEM extraction, and other common tools |
| Raster Processing | Tools for working with raster data such as imagery, elevation data, GDAL integration, creating hillshades, applying transforms, change detection, raster to vector, etc. |
| Parallel Processing | Example tools for processing in parallel using scalable compute resources based on DASK or batch processing |
| Custom Tools | Custom tools and examples depending on customer applications |
The Analysis Hub allows a set of python libraries to be accessible for the user and can be configured based on the application. The following table includes some of the most commonly used and included libraries in the environment:
| Python Libraries | Description | Reference Documentation |
|---|---|---|
|
NumPy |
Scientific computing in Python. |
|
|
matplotlib/seaborn /GeoPlot/cartopy |
For creating plots & diagrams. |
|
|
argparse |
Command line parsing module. |
|
|
logging |
Flexible event logging. |
|
|
datetime/time |
Manipulate dates and times. |
|
|
os/sys |
Portable way of using operating system dependent functionality. |
| Python Libraries | Description | Reference Documentation |
|---|---|---|
|
boto3 |
Working with AWS S3 storage. |
|
|
s3fs |
Allows file-system-type operations within s3. |
|
|
fsspec |
fsspec provides a common interface for various file-system backends (s3fs, gcsfs). |
|
|
AWSCLi |
AWSCLi provides a command-line interface for AWS. |
|
|
kerchunk |
Efficient file access to cloud data |
|
|
zarr |
File storage format for chunk, compressed, N-dimensional arrays |
|
|
Netcdf |
File for formats for scientific multidimensional arrays. |
|
|
Azure |
Accessing azure resources from python code |
|
|
requests |
Web service requests and 3rd party integration |
| Python Libraries | Description | Reference Documentation |
|---|---|---|
|
PDAL |
Point Data Abstraction Library for LiDAR processing. |
|
|
GDAL/osgeo |
Numerous tools in the GDAL library. |
|
|
laspy/laszip |
Working with point cloud data (e.g LAS/LAZ files). |
|
|
rasterio |
Working with raster data (e.g GeoTIFF files). |
|
|
Icecream |
Inspect variables, expressions, and program execution without using print(). |
|
|
UUID |
Create immutable UUID (Universally Unique Identifier) objects. |
|
|
proj |
Handling coordinate reference systems (CRSs) and projection |
|
|
pillow/PIL/exif |
Working with image files |
|
|
shapely/geopandas/OGR/Fiona |
Working with vector file formats. Shapely does set-theoretic analysis and manipulation of planar features. Geopandas makes working with geospatial data in python easier. Fiona streams simple feature data to and from GIS formats like .json and .shp files. |
| Python Libraries | Description | Reference Documentation |
|---|---|---|
|
ipyleaflet |
Ipyleaflet is a Jupyter widget for Leaflet.js, enabling interactive maps in the Jupyter notebook. |
|
|
cv2 |
Automated CI toolchain to produce precompiled opencv-python packages. |
|
|
SciPy/Scikit-learn/Scikit-image |
Algorithmic optimization and predictive data analysis. Typically used in change detection tools. |
|
|
Collections |
This module implements specialized container datatypes providing alternatives to Python’s general purpose built-in containers. |
|
|
tifffile |
Store NumPy arrays in TIFF (Tagged Image File Format) files, and read image and metadata from TIFF-like files. |
| Python Libraries | Description | Reference Documentation |
|---|---|---|
|
tqdm notebook |
IPython/Jupyter Notebook progressbar decorator for iterators. |
|
|
Xarray |
Working with multidimensional labelled datasets and arrays |
|
|
Dask/dask_gateway/dask_distributed |
Efficient parallelization for data analytics in python. |
|
|
Collections |
This module implements specialized container datatypes providing alternatives to Python’s general purpose built-in containers. |
|
|
tifffile |
Store NumPy arrays in TIFF (Tagged Image File Format) files, and read image and metadata from TIFF-like files. |