Idea: Allow the user to set the path for the gpbackman history_db file using an environment variable

[xanadu-dev]

In a Cloudberry installation with more than one database there could be a use case where the rules for data retention and backup deletion would be different for the different databases.

In this case it would be desirable to have different history files for different databases. This would avoid the situation where one history file is used for several databases and hence files for all database backups are cleaned up instead of just one database.

This could be achieved if there could be more than one history database with the path for each history db specified by an environment variable "GPBACKMAN_HISTORY_DIR" or similar.

Also it would also be useful if there was a specific gpbackman command to initialise the database.

[woblerr]

Hi,

gpbackup already initializes the history database automatically. If gpbackup_history.db does not exist, it creates the file and initializes the required schema when writing the first backup entry. Therefore, a separate gpbackman initialization command does not seem necessary.

gpbackman currently supports several ways to select the history database:

1. Specify the full path explicitly:

   gpbackman --history-db /path/to/gpbackup_history.db ...

2. Resolve it from $COORDINATOR_DATA_DIRECTORY:

   gpbackman --auto-load-history-db ...

   This resolves to:

   $COORDINATOR_DATA_DIRECTORY/gpbackup_history.db
   

3. If neither option is specified, gpbackman looks for gpbackup_history.db in the current working directory.

An environment variable can also already be used together with the existing flag:

gpbackman \
  --history-db "$GPBACKMAN_HISTORY_DIR/gpbackup_history.db" \
  ...

Considering these options, adding another built-in environment variable for selecting the same file looks redundant.

gpbackup itself always writes history to the canonical cluster path:

<primary coordinator data directory>/gpbackup_history.db

The history database is cluster-wide, and other functionality, including incremental backup discovery, restore metadata lookup, and standby history synchronization, relies on this canonical location.

The actual issue in the described use case is that backup-clean and history-clean do not support filtering by database. I think it would be better to keep one cluster history database and add a database filter, for example:

gpbackman backup-clean \
  --database db1 \
  --older-than-days 7

gpbackman history-clean \
  --database db1 \
  --older-than-days 30

Alternatively, we could add symmetric --db-include and --db-exclude options.

The database_name field is already stored in the backups table.

Would database filtering for backup-clean and history-clean solve your use case?

[xanadu-dev]

Thanks for the quick response

I 100% agree with your suggestion and would prefer to maintain one cluster wide history database and have the ability to filter by database

In our use case we would trigger a database clean only after a a successful backup is taken so a --database filter would be sufficient and --db-include and --db-exclude options would not be needed,

The actual issue in the described use case is that backup-clean and history-clean do not support filtering by database. I think it would be better to keep one cluster history database and add a database filter, for example:

gpbackman backup-clean
--database db1
--older-than-days 7

gpbackman history-clean
--database db1
--older-than-days 30

Alternatively, we could add symmetric --db-include and --db-exclude options.

The database_name field is already stored in the backups table.

Would database filtering for backup-clean and history-clean solve your use case?