Warning

CCMORPH2 (version 2.0) is no longer supported. The new official fork of the software is available here: CMORPH.

The Coastal Cliffs Morphology Analysis Toolbox v2.0 (CCMORPH2)

The toolbox is series of Python scripts accompanied with JavaScript tool that can be utilized to analyse dynamics of coastal cliffs based on DEM from different sources such as GPS-RTK, UAVs, aerial or terrestrial LiDAR scanners. Python scripts are client-based (run locally), while Java Script tool can be run as a server-based application (in a web-browser environment).

The repository includes Python tools for automatic analysis of input data (generator-py, finder-py and analyser-py), a web tool that allows manual correction of selected coastal characteristic points (base and top of the profile), and sample data and configurations demonstrating how the tools work.

The repository also includes Jupyter notebook scripts for automatic comparison of output data (csv_compared-ipynb, csv_compared_2-ipynb, dem_compared-ipynb, figures-ipynb and figures_compared-ipynb)

Tools

generator-py

The program is used to generate elevation profiles based on a given coastal DEM file (geotiff), a specified shoreline (SHP line) and a trimming area (SHP polygon). Basic parameters such as input and output data paths or transect parameters (lengths and spacing) can be set in the config.json file. Note that for a shoreline and a trimming area, in addition to *.shp files, *.prj files should be prepared, containing information about coordinate reference systems. In the configuration (shapes/coastline/dst_crs and shapes/buffer/dst_crs), you have to specify a system they are being transformed to for further calculations.

Both final results and all intermediate results are stored in separate files that can be used, for example, by finder-py, analyzer-py or click-the-coast-js

What does the program do?

1. validates input data
2. creates or cleans directories for output and intermediate data
3. initializes the database
4. generates transects based on shoreline (data/input/coast)
5. creates buffers around transects and for each buffer creates a DEM based on the overall image (data/input/dem)
6. generates elevation profiles from transects and trimmed DEM models
7. trims profiles to the specified area (data/input/crop) and saves them (data/output/profiles/cropped)
8. saves data for profile visualization in click-the-coast-js web application (data/output/web/geotif, data/output/web/geojdon, data/output/web/names)

configuration

The default configuration of the program can be changed by editing the generator-py/config.json file. Among other things, we can specify:

• directory paths for input data,
• directory paths for output data,
• the path to the database,
• parameters of generated transects,
• parameters of layers in the database (names, crs, ...), etc.

finder-py

The program allows you to automatically determine the base and the top of a cliff on each profile by approximating the distance between shoreline and the cliff top. A new feature has been implemented in the methodology of the finder-py script which is called “beyond_top_buffer”. It refers to the distance between the maximum height point (top point of the cliff) to the last point determined by the algorithm. It informs how far beyond the highest point of the profile the profile itself ends (as long as we do not go beyond its maximum range).

What does the program do?

1. reads the trimmed profiles generated by generator-py (data/output/profiles/cropped)
2. finds the base and the top on the profiles (choose one of the methods described in the article)
3. saves finder.csv results to two directories (data/output/results and data/output/web/results)
4. optionally saves calculation details for each profile for method no. 1 (data/output/debug)

configuration

• paths to the input data,
• paths to the output data,
• the selected method of determining points (1 or 2),
• optional profile smoothing parameters,
• optional debug parameters, etc.

click-the-coast-js

A web application that visualizes elevation profiles for which the base and the top could not be automatically determined. It allows you to manually set both parameters of the profile. The profile is displayed as a line on the map (top view) and an intersection (side view). Basic parameters, such as paths, can be set in scripts/config.js file).

What does the program do?

1. reads profile names generated by generator-py (data/output/web/names/names.json)
2. reads trimmed profiles (output/web/geojson) and BBOXes (bounding boxes) (data/output/web/geotiff)
3. reads information about profiles for which base and top have already been automatically determined by finder-py (data/output/results/finder.csv)
4. visualizes profiles without designated bases and tops and allows you to mark them manually
5. saves the results to a csv file (data/output/web/results/manual.csv)

NOTE: Click-the-coast-js is a web-based application. Practically all the application logic is executed on the client side (JavaScript). Only the saving to a file of points marked on the map is performed on the server side and requires the PHP interpreter. The application can therefore be run using any web server supporting PHP. Since the application can only refer to the directory in which it is placed, so the necessary data must be copied to and from it. The easiest way is to move a piece of the data/output/web directory hierarchy to the application directory. Remember to manually copy the manual.csv file created by the application to the original results directory so that analyzer-py can use it!

configuration

The default configuration can be changed by editing the click-the-coast-js/scripts/config.js file. Among other things, you can set:

• path to the directory with profiles,
• path to the directory with bbox files,
• path to the file with profile names,
• map parameters, etc.

analyzer-py

The program collects information about the bases and tops of profiles (generated automatically or marked manually) and calculates their properties such as distance, slope and volume. Parameters should be set in the config.json file.

What does the program do?

1. reads the trimmed profiles generated by generator-py (data/output/profiles/cropped)
2. reads information about profiles whose base and top have already been automatically determined by finder-py (data/output/web/results/finder.csv)
3. reads the results of click-the-coast-js, i.e. bases and tops marked manually (data/output/web/results/manual.csv - remember to copy this file after using click-the-coast-js from the web application's data directory)
4. determines profile parameters (distance, slope, volume) for designated bases and tops
5. saves the results in a csv file (data/output/finall)
6. saves lists of tops and bases separately to shp files (data/output/finall)

configuration

• paths to the input data,
• paths to the output data,
• format of csv file with results from finder-py, etc.

Notebook Scripts

csv_compared

The script takes the generated CSV file from analyzer-py for comparison, the user can input multiple CSV files for comparison.

What does the program do?

1. reads the morphological properties inside the CSV (data/output/analyzer)
2. Compares the properties between inputted CSV files
3. Saves the results of multiple CSVs in a single csv file (OUTPUT_CSV_FOLDER, f'{field}_diff.csv')

csv_compared_2

The script takes the generated CSV file from analyzer-py for comparison, the user make comparison between two different CSV files.

What does the program do?

1. reads the morphological properties inside the CSV (data/output/analyzer)
2. Compares the properties between both CSV files.
3. Saves the results of CSVs in a single csv file (OUTPUT_CSV_FOLDER, f'{field}_diff.csv')

figures

Figures script is specifically implemented to generate graphical representations and comparisons of important cliff features such as cliff zero, top, and bottom points in multiple CSV files. The figures subcomponent marks these critical cliff features on automatically drawn lines according to the profile id in the corresponding CSV

What does the program do?

1. Reads the important cliff features (cliff top, bottom and zero) according to the profile_id in the given CSV.
2. Generates graphical images representing the cliff features
3. Saves the images in either SVG/PNG/JPG ((OUTPUT_FOLDER, f'{title}.cut{".zoom" if zoom else ""}.jpg'))

figures_compared

Figures script is specifically implemented to generate graphical representations and comparisons of important cliff features such as cliff zero, top, and bottom points between two different CSV files. The figures subcomponent marks these critical cliff features on automatically drawn lines according to the profile id in the corresponding CSV

What does the program do?

1. Reads the important cliff features (cliff top, bottom and zero) according to the profile_id in the given CSV.
2. Generates graphical images representing the cliff features in (SVG/PNG) format
3. Saves the images in either SVG/PNG/JPG ((OUTPUT_FOLDER, f'{title}.cut{".zoom" if zoom else ""}.jpg'))

dem_compared

A similar approach to representing cliff features is employed by this script, which generates a set of SVG/PNG/JPG images for a specified list of input cropped DEM files. The subcomponent marks the top, bottom and zero points of the cliff on the DEM raster.

What does the program do?

1. Reads the important cliff features (cliff top, bottom and zero) according to the profile_id in the given CSV.
2. Generates graphical images with marked cliff top, bottom and zero points on a DEM raster
3. Saves the images in either SVG/PNG/JPG ((OUTPUT_FOLDER, f'{title}.cut{".zoom" if zoom else ""}.jpg'))

Instalação e inicialização

Opção 1 — Docker (recomendado, único pré-requisito)

O jeito mais simples de rodar o toolbox: o único pré-requisito é ter o Docker (e o plugin docker compose) instalados e em execução. Não é necessário instalar Python, GDAL, Rtree ou qualquer outra dependência no host.

git clone <url-do-repositorio>
cd CCMORPH
docker compose build toolbox

Isso constrói uma imagem Ubuntu 22.04 com Python 3.10, GDAL, Rtree e todos os pacotes de requirements.txt (incluindo o richdem, compilado com a correção de compatibilidade necessária).

Para rodar todos os samples de demonstração de uma vez (equivalente a ./run_demo.sh):

docker compose run --rm toolbox

Para rodar apenas um sample específico:

docker compose run --rm toolbox ./run_sample.sh sample_1
docker compose run --rm toolbox ./run_sample.sh meu_dado

O diretório do projeto é montado como volume dentro do container (/app), então os resultados aparecem normalmente em demo/<sample>/output/ no host, e qualquer alteração de configuração feita localmente já é refletida no container sem precisar reconstruir a imagem.

Para abrir um shell interativo dentro do container (útil para depuração):

docker compose run --rm toolbox bash

Opção 2 — Instalação nativa (Linux / WSL2)

Alternativa sem Docker, testada em Debian/Ubuntu via WSL2 no Windows 11 Pro, com Python 3.10. O script init.sh automatiza toda a configuração do ambiente.

Pré-requisitos

• Linux (Ubuntu/Debian) ou WSL2 no Windows
• git instalado (sudo apt install git)
• Conexão com a internet (para baixar pyenv e pacotes)

Instalação automática

Execute na raiz do projeto:

./init.sh

O script realiza automaticamente:

1. Instala o pyenv (se necessário) e o Python 3.10
2. Instala as dependências do sistema: GDAL, Rtree, OpenBLAS
3. Cria e ativa o ambiente virtual Python (env/)
4. Detecta a versão do GDAL instalada e atualiza o requirements.txt
5. Compila e instala o richdem com correção de compatibilidade
6. Instala todos os pacotes Python
7. Monta os arquivos TIF de demo (divididos em partes no repositório)

Ativação do ambiente (sessões futuras)

source env/bin/activate

Exemplos

Há cinco conjuntos de dados de teste em demo/sample_1 até demo/sample_5. Para rodar todos de uma vez (após ./init.sh, ou usando docker compose run --rm toolbox como na Opção 1):

./run_demo.sh

Rodando um sample específico

Use o script run_sample.sh passando o nome do diretório dentro de demo/ (prefixe com docker compose run --rm toolbox se estiver usando Docker):

./run_sample.sh sample_1
./run_sample.sh meu_dado

Adicionando seus próprios dados

Crie um diretório em demo/ com a seguinte estrutura:

demo/meu_dado/
├── input/
│   ├── coast/     # shapefile da linha de costa (.shp + .prj + .dbf + .shx)
│   ├── dem/       # arquivo GeoTIFF do DEM
│   └── crop/      # shapefile do polígono de recorte (.shp + .prj + .dbf + .shx)
├── generator_config.json
├── finder_config.json
└── analyzer_config.json

Os arquivos de configuração seguem o modelo dos samples existentes. O campo "base" deve apontar para ../../demo/meu_dado e o CRS deve corresponder ao dos seus dados (verifique com gdalinfo ou inspecionando o .prj).

Modificando parâmetros

Para alterar o espaçamento entre transectos, edite o generator_config.json:

"transect": {
    "distance": 10,

Click-the-Coast (correção manual no browser)

Requer Docker instalado e em execução (independente de estar usando a Opção 1 ou 2 acima):

# Sobe a aplicação web com os dados de um sample
./ctc-start.sh demo/sample_5/output/web
# Acesse: http://localhost:8082

# Baixa os resultados corrigidos manualmente
./ctc-download.sh demo/sample_5/output/web/results

# Remove o container quando terminar
./ctc-finish.sh

Após baixar os resultados, rode o analyzer-py novamente para recalcular com os dados corrigidos:

./run_sample.sh sample_5
# ou, via Docker:
docker compose run --rm toolbox ./run_sample.sh sample_5

Notas

Problemas mais comuns:

• Docker (Opção 1): garanta que o Docker Desktop/daemon esteja em execução antes de docker compose build/run. No Windows, use o backend WSL2.
• Instalação nativa (Opção 2): Rtree ou GDAL não instalados no sistema — use ./init.sh para instalar automaticamente
• Conflito de versão entre o GDAL do sistema e o pacote Python — tanto o init.sh quanto o Dockerfile detectam a versão instalada e ajustam requirements.txt automaticamente
• Arquivos .shp sem o .prj correspondente — o CRS é obrigatório
• Dado sem .prj ou com datum incorreto — verifique com ogrinfo -al -so seu_arquivo.shp