Stairway to backup - another stairway maintenance

Maintenance release

Third maintenance release for v3.0.0 series.
It mainly adds emailing support, better prometheus metrics, and a smoother GUI experience.

Disruptive changes

• An automatic config file migration tool has been added which will migrate config files from 3.0.0 to 3.0.3
• The configuration file format has changed for retention policies
  • 'tags' has become 'keep_tags' which is more meaningful
  • 'apply_tags' has been added to filter retention policies only on specific snapshots containing defined tags
• Existing files will be in-place upgraded

Features

• New finer grained concurrency control
• New optional random delay before backup operations
• All prometheus metrics now have repo_name label, thanks to @GuitarBilly
• All prometheus metrics now include os, arch and audience labels
• Metrics are now send for non executed backups because of concurrency checks, #161, thanks to @GuitarBilly
• Implement operations notifications per email, #139
• CLI: Add --list-selected-repos command
• Retention policies can now be filtered by tags
• GUI: Restore operation now has throughput statistics and a better overall presentation

Fixes

• GUI: Fix possible result mismatch with concurrency checks
• GUI: Hide background window while restoration is running
• Fix npbackup_exec_state inherited previous run states in the same session (ie GUI or group operations)

Misc

• GUI: Finally a worthy fix for hiding the console window in Windows 11 (they changed conhost.exe to Windows Terminal)
• GUI: Workaround to make the config window close faster by threading it into background
• Update restic metrics tests
• Updated Nuitka compiler to 2.7.8
• KVM Qemu script now has VM exclusion lists
• KVM Qemu script does not try to create snapshots for VMs that aren't running
• KVM Qemu can now add specific tags to VMs defined in a list
• KVM Qemu now issues thaw command after a quiescing snapshot in order to workaround some FreeBSD corner cases

Stairway to backup - stairway maintenance 2

Maintenance release

Second maintenance release for v3.0.0 series.

Enhancements

• Make NTP offset configurable at compile time
• Implement new restic --exclude-cloud-files, replacing previous implementation

Fixes

• Fix rclone debug output preventing proper execution, #132, thanks @GuitarBilly
• Don't throw error on missing metrics size
• GUI: Fix empty strings in numeric conversions, #156, thanks @GuitarBilly
• GUI: Fix missing inheritance on prune settings
• GUI: Fix showing irregular / missing files in GUI did not show the right icons, #157, thanks @GuitarBilly
• GUI: Don't lock viewer, #155, thanks @jmigual for reporting
• Fix potential AES key read problem
• Don't require lock for snapshot viewing, #160, thanks @GuitarBilly

Misc

• Bump restic version to 0.18.0
• Make alternative legacy builds for linux and windows
• Reduce debug output
• Don't run snapshot command on every run, fixes #159, thanks @GuitarBilly
• Various log message improvements
• Set default backup interval to 23h55, #162, thanks @GuitarBilly

Stairway to backup - stairway maintenance

Maintenance release

So this is shall become the first maintenance release for v3.0.0 series.
At this point we shouldn't add any major features, but still add minor enhancements.
The major enhancement here comes from random housekeeping / auto-upgrade run chances, which will allow many workstation instances to spread these actions over a larger time range, in order to avoid network congestion.

Enhancements

• Enable prune tuning parameters
• Add housekeeping after backup run chance parameter
• Add auto upgrade run chance parameter
• Configuration file checks are now subversion aware
• Allow partial configuration files (missing default repo / group)

Fixes

• Missing specified backup paths won't trigger errors (backend behavior), see #151
• Updated Grafana Dashboard: fixed variable gathering from wrong metric name)
• CLI: Fix --prune-max (deprecated)
• CLI: Add --prune max (replaces --prune-max)
• CLI: Fix --policy not allowed in group operations
• GUI: Fix CTRL+C triggers and error, #125
• GUI: Fix showing inheritance icon on entries wiith units, #148
• GUI: Permission set button should be red if no permission set
• GUI: Default units should be IEC bytes, like restic backend, fixes #143
• Fix for textcollector & push gateway data being overwritten in group operations mode, #150
• Dissociate minimum backup size error from absent backup error, #141

Misc

• GUI general UX and cosmetic changes

Thanks to @GuitarBilly for his help reporting bugs ;)

Stairway to backup - stairway maintenance

Maintenance release

So this is shall become the first maintenance release for v3.0.0 series.
At this point we shouldn't add any major features, but still add minor enhancements.
The major enhancement here comes from random housekeeping / auto-upgrade run chances, which will allow many workstation instances to spread these actions over a larger time range, in order to avoid network congestion.

Enhancements

• Enable prune tuning parameters
• Add housekeeping after backup run chance parameter
• Add auto upgrade run chance parameter (deprecates interval runs)

Fixes

• Missing specified backup paths won't trigger errors (backend behavior), see #151
• Updated Grafana Dashboard: fixed variable gathering from wrong metric name)
• CLI: Fix --prune-max
• CLI: Fix --policy not allowed in group operations
• GUI: Fix CTRL+C triggers and error, #125
• GUI: Fix showing inheritance icon on entries wiith units, #148
• Fix for textcollector & push gateway data being overwritten in group operations mode, #150
• Dissociate minimum backup size error from absent backup error, #141

Misc

• GUI general UX and cosmetic changes

Stairway to Backup

Official v3.0.0 release

It's been a long wait for this release. Almost all parts of NPBackup have been rewritten or have been updated.
A very long release candidate period has been observed in order to iron out all possible bugs and nipticks.
So here we are, proud to build some stairways to h... euhh easier but reliable backups.

This release is way too big to enumerate all the new functions. But here's a list of the major features:

• Multi-repository support
• Easier repo setups which can inherit from group settings
• New viewer application for viewing/restoring data from any restic repositories
• CLI and GUI are now two separate applications (still GUI can act as CLI)
• CLI can now fully operate in API mode via --json parameter
• New operation center which allows to run operations on multiple repositories and / or groups
• New housekeeping operation, which checks, implements retention policy and prunes backups
• Prometheus monitoring now supports all operations
• Upgrade server can now filter on client / group / platforms
• Memory and speed optimization
• Automatic tests

For the full changelog, please see CHANGELOG.md file.

Special thanks to @GuitarBilly for his help in finding bugs / asking questions / general help.

6-strings

Almost half of the bug fixes for this release could happen thanks to @GuitarBilly who made alot of nice feedback. Thank you.

Features

• NPBackup now allows not having default repo and default_group group
• Allow group operations to be cancelled

Fixes

• GUI
  • Make sure we don't allow dots in repo and group names
  • Fix update of repo type on repo change
  • Fix double full permission introduced in rc14, thanks to @GuitarBilly
• Preserve pre/post script execution order, thanks to @GuitarBilly
• Fix writing metrics twice in group operation mode
• Fix repo name not populated in prometheus metrics when metrics destination not set
• Fix forget operation bug introduced in earlier rc
• Fix --repo-name trying to show corresponding groups
• Fix scheduled task creation bug introduced in rc13, thanks to @GuitarBilly

Misc

• viewer now creates it's own log file, thanks to @GuitarBilly
• GUI: cosmetic hack for linux

Made with love... and python

Today's valentine day special... A new NPBackup v3 release candidate.
This one has been long in the making.

I'd like to thank @GuitarBilly for his really good issues / bug tracking, which highly improves the quality of this RC release.

Features

• Upgrade server/client now can filter and provide different binaries / upgrade scripts for specific machines / groups
• Upgrade rollback has been improved on Windows
• Upgrade client can identify if the initial scheduled program launch needs to run
• Upgrade server now has a basic multi-user and permission system
• Upgrade client now tests if new binaries can properly read current config file
• GUI:
  • Minor UX improvements
  • GUI can now remove a manager password
  • Improve upgrade process messages
  • Add new config file santizer check
• New restore_only permission

Fixes

• Upgrade client now retains config file if it was in the same directory as the binaries
• Rollback behavior is more resilient under Windows
• GUI:
  • Don't polluate logs with snapshot content
  • GUI now accepts multiple --run-as-cli arguments when it's relaunched with prior arguments
  • Adding multiple files at once to backup paths now expand to list
  • Don't try to kill current process when exiting, only backup backend
  • Fix deletion of just created objects
  • Fix permissions & manager password reset when saving configuration twice
• Fix files-from source not recognized
• Windows x86 legacy builds now include the adequate older backend binary

Misc

• Unlocking actions is now permitted for all permissions
• Reword some metrics in Grafana dashboard
• Added variable and user expansion for additional_parameters and additional_backup_only_parameters
• Added a new environment variable with path to current backend binary for script usage
• Newer windows builds now run with Python 3.13
• Switched to Nuitka commercial v2.6.4 compiler

2 Minutes to Release

Features

• GUI: Better UX for cloud provider setup

Fixes

• GUI: Fix show snapshot content for legacy builds using Python 3.9 or 3.10 (Using non existing StrEnum)

Misc

• Include buildtype in version string and prometheus metrics
• CLI: Improve --show-config help
• By default, follow mountpoints in backup (prefer backup too much than not enough philosophy)
• Remove /mnt from default linux exclusions (again, same philosophy)
• Linux now has legacy (Python 3.9) and recent (Python 3.12) builds, which run depending on your glibc version

Caught Somewhere in Time(stamp)

Features

• New restic binaries are now downloaded on the fly when compiling
• Added windows/linux functional tests for all operations

Fixes

• GUI: Avoid complaining about missing snapshots*
• CLI: Fix upgrade process for linux, also allow .tar.gz archives
• CLI: Improve upgrade process logs
• CLI: Make upgrader resume initial command after succesfull upgrade
• CLI: Fix --dry-run wasn't honored for commands other than backup
• CLI: Fix --json only output when logfile isn't writable (we printed a warning to stdout, which should only be logged)
• CLI: Make sure --dump only outputs raw data without logs

Misc

• GUI: Added default values to task creation
• CLI: Allow combining --json and --debug
• Increased default timeouts for init operation and tests
• Don't set process priority when getting backend version (operation isn't time consuming enough)

Interim Build

This is a quick interim build with restic 0.17.3 to address the VSS issues that came with 0.17.2.