After some further development of the rapidtar project, I've released version 0.2. The headlining feature is volume spanning, but other facilities have also been added or improved. The code for this project is available on GitHub and can be installed via cargo. Downloadable binaries are not yet available.
- rapidtar's library has been split off into the separate librapidarchive crate, whose API is subject to change and shouldn't be relied upon yet. The binaries remain in rapidtar and that package is what you should install if you want them.
- A new command-line utility called rapidmt was added, which adds an mt reimplementation that runs on Windows.
- rapidmt does not support all mt operations yet. The biggest omission is software eject.
- The new read and write subcommands allow copying files from or storing files on tape. This facility is block size agnostic; you can dump tapes regardless of how they were recorded.
- There is currently no way to dump files from, or write files to, a tape while preserving variable record lengths. rapidmt treats your tape files like, well, files. The notion of variable record size is very specific to tape and a special image format will need to be designed to archive such data.
- I have successfully tested this facility by dumping tapes people forgot to wipe when selling them to me off eBay. For obvious ethical reasons, that data isn't going to be used, but it was stored in Backup Exec format. This means that I now have legally acquired test vectors for a proprietary archive format that I could reverse-engineer. rapidbex, anyone?
- Written archives can now span multiple volumes.
- When the target volume runs out of space, rapidtar will prompt you to insert a new one.
- You can also force a volume change after a specific number of bytes written.
- Despite the fact that we only support it on "POSIX" format archives, file splitting is actually an under-documented GNU extension with some weird quirks. Like, the offset index goes in a global header, rather than the resumed file's header.
- Currently, we don't make it easy to reassemble split files if encountered by a non-GNU, but PAX-compliant unarchiver. In a future version of rapidtar, we will either move file splitting into a separate format flag (perhaps PAX_GNU) or change our archive headers to make manual reassembly easier.
- rapidtar now has tape device support on UNIX
- Caveat: Seeking definitely only works with Linux SCSI tape devices, since the IOCTLs are specific to that driver.
- You can now specify a --format of USTAR or POSIX archive format. The latter is the default and supports far more features.
- You can now specify a --label. Multi-volume archives will be labeled in sequence.
- Accumulated write totals and speeds are written at every volume change if requested.
- Ownership metadata (user/group) is now correctly archived.
- Instead of UIDs and GIDs, Windows uses SIDs, which are variable-length equivalents of UIDs and GIDs and cannot be faithfully represented in tar headers. Instead, we fill the UID/GID slot with 65534, which at least on Linux references nobody/nogroup.
- The name of the owning user and principal group of a Windows file or directory is faithfully represented in rapidtar's output.
- We also support the PAX extended uname/gname field if the username or group name won't fit in the USTAR header.
- Whoever designed this terrible, terrible, non-thread-safe, poorly documented API needs to be flogged. Would it kill you to call malloc()?!
- For that matter, who in their right mind thought it would be a good idea to allow getting the name of a SID to fail due to missing permissions!?
- Further improvements to overall system stability and other minor adjustments have been made to enhance the user experience.
- Fixed a bug where tape devices on Windows wouldn't open 60% of the time because I forgot a null terminator.
- Filemarks are now actually written between each tape file.
rapidtar 0.3 roadmap
So, I recently bought and installed a used PowerVault TL2000 to automate my tape backups. This is great, but I can't actually remote control it from the Linux command line. Media commands issued by mtx are failed by the drives. Instead of debugging firmware versions or what have you, I instead plan to write a new utility called rapidmtx which will allow issuing remote control commands to the device's web management UI. It probably won't be an exact replacement for mtx, but I want to retain enough functionality that I can script volume changes from within a tar operation. (This also means rapidtar needs to learn how to host subshells and scripts like GNU tar does.)