Newer
Older
git-annex (8.20210804) UNRELEASED; urgency=medium
* Deal with clock skew, both forwards and backwards, when logging
information to the git-annex branch.
* GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1)
rather than needing to be advanced each time a new change is made.
* Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex.
* add: When adding a dotfile, avoid treating its name as an extension.
Joey Hess
committed
* rsync special remote: Stop displaying rsync progress, and use
git-annex's own progress display.
* Many special remotes now checksum content while it is being retrieved,
instead of in a separate pass at the end. This is supported for all
special remotes on Linux (except for bittorrent), and for many
on other OS's (except for adb, external, gcrypt, hook, and rsync).
Joey Hess
committed
* unused: Skip the refs/annex/last-index ref that git-annex recently
started creating.
* New --batch-keys option added to these commands:
get, drop, move, copy, whereis
* Added annex.youtube-dl-command config. This can be used to run some
forks of youtube-dl.
-- Joey Hess <id@joeyh.name> Tue, 03 Aug 2021 12:22:45 -0400
git-annex (8.20210803) upstream; urgency=medium
* whereused: New command, finds what files use a key, or where a key
was used historically.
* Fix a bug that prevented getting content from a repository that
started out as a bare repository, or had annex.crippledfilesystem
set, and was converted to a non-bare repository.
* Fix retrieval of content from borg repos accessed over ssh.
* sync: When --quiet is used, run git commit, push, and pull without
their ususual output.
* merge: When --quiet is used, run git merge without its usual output.
* sync, merge, post-receive: Avoid merging unrelated histories,
which used to be allowed only to support direct mode repositories.
(However, sync does still merge unrelated histories when importing
trees from special remotes, and the assistant still merges unrelated
histories.)
* sync, merge: Added --allow-unrelated-histories option, which
is the same as the git merge option.
* Fix bug that caused some transfers to incorrectly fail with
"content changed while it was being sent", when the content was not
changed.
* Fix bug that could prevent pointer files from being populated,
in a repository that was upgraded from v7.
* fsck: Detect and correct stale or missing inode caches.
* Fix a rounding bug in display of data sizes.
* git-annex get when run as the first git-annex command in a new repo
did not populate unlocked files.
(Reversion in version 8.20210621)
-- Joey Hess <id@joeyh.name> Tue, 03 Aug 2021 12:20:09 -0400
git-annex (8.20210714) upstream; urgency=medium
* assistant: Avoid unncessary git repository repair in a situation where
git fsck gets confused about a commit that is made while it's running.
* addurl: Avoid crashing when used on beegfs.
* --debug output goes to stderr again, not stdout.
(Reversion in version 8.20210428)
* init: Fix misbehavior when core.sharedRepository = group that
caused it to enter an adjusted branch and set annex.crippledfilesystem
(Reversion in version 8.20210630)
* assistant: When adding non-large files to git, honor annex.delayadd
configuration. Also, don't add non-large files to git when they
are still being written to.
-- Joey Hess <id@joeyh.name> Wed, 14 Jul 2021 12:22:55 -0400
git-annex (8.20210630) upstream; urgency=medium
* Fixed bug that interrupting git-annex repair (or assistant) while
it was fixing repository corruption would lose objects that were
contained in pack files. Also the assistant might sometimes have
crashed with the same result.
* repair: Fix reversion in version 8.20200522 that prevented fetching
missing objects from remotes.
* sync: Partly work around github behavior that first branch to be pushed
to a new repository is assumed to be the head branch, by not pushing
synced/git-annex first.
* Added annex.freezecontent-command and annex.thawcontent-command
configs.
* Improve display of errors when transfers fail.
* Dropping an unused object with drop --unused or dropunused will
mark it as dead, preventing fsck --all from complaining about it
after it's been dropped from all repositories.
* addurl, importfeed: Added --no-raw option that forces download
with youtube-dl or a special remote. In particular this can avoid
falling back to raw download when youtube-dl is blocked for some
reason.
-- Joey Hess <id@joeyh.name> Wed, 30 Jun 2021 11:48:16 -0400
git-annex (8.20210621) upstream; urgency=medium
* New matching options --excludesamecontent and --includesamecontent
* When two files have the same content, and a required content expression
matches one but not the other, dropping the latter file will fail as it
would also remove the content of the required file.
* drop, move, mirror: When two files have the same content, and
different numcopies or requiredcopies values, use the higher value.
* drop --auto: When two files have the same content, and a preferred content
expression matches one but not the other, do not drop the content.
* sync --content, assistant: When two unlocked files have the same
content, and a preferred content expression matches one but not the
other, do not drop the content. (This was already the case for locked
files.)
* sync --content, assistant: Fix an edge case where a file that is not
preferred content did not get dropped.
* filter-branch: New command, useful to produce a filtered version of the
git-annex branch, eg when splitting a repository.
* fromkey: Create an unlocked file when used in an adjusted branch
where the file should be unlocked, or when configured by annex.addunlocked.
* Fix behavior of several commands, including reinject, addurl, and rmurl
when given an absolute path to an unlocked file, or a relative path
that leaves and re-enters the repository.
* smudge: Fix a case where an unlocked annexed file that annex.largefiles
does not match could get its unchanged content checked into git,
due to git running the smudge filter unecessarily.
* reinject: Error out when run on a file that is not annexed, rather
than silently skipping it.
* assistant: Fix a crash on startup by avoiding using forkProcess.
* init: When annex.commitmessage is set, use that message for the commit
that creates the git-annex branch.
* Added annex.adviceNoSshCaching config.
* Fix an exponential slowdown when large numbers of duplicate files are
being added in unlocked form.
-- Joey Hess <id@joeyh.name> Mon, 21 Jun 2021 12:17:24 -0400
git-annex (8.20210428) upstream; urgency=medium
* New annex.private and remote.name.annex-private configs that can
prevent storing information about a repository and remotes in the
git-annex branch.
* initremote: Added --private option to set up a private special remote.
* Added --debugfilter (and annex.debugfilter)
* forget: Preserve currently exported trees, avoiding problems with
exporttree remotes in some unusual circumstances.
* fsck: When downloading content from a remote, if the content is able
to be verified during the transfer, skip checksumming it a second time.
* Avoid excess commits to the git-annex branch when stall detection is
enabled.
* git-annex-config: Allow setting annex.securehashesonly, which has
otherwise been supported since 2019, but was missing from the list of
allowed repo-global configs.
* init: Fix a crash when the repo's was cloned from a repo that had an
adjusted branch checked out, and the origin remote is not named "origin".
* Fix some bugs that made git-annex not see recently recorded status
information when configured with annex.alwayscommit=false.
* When mincopies is set to a larger value than numcopies, make sure that
mincopies is satisfied. Before, it assumed a sane configuration would
have numcopies larger or equal to mincopies. It's still a good idea
not to configure git-annex this way.
* Avoid more than 1 gpg password prompt at the same time, which
could happen occasionally before when concurrency is enabled.
-- Joey Hess <id@joeyh.name> Wed, 28 Apr 2021 12:13:24 -0400
git-annex (8.20210330) upstream; urgency=medium
* Behavior change: When autoenabling special remotes of type S3, weddav,
or glacier, do not take login credentials from environment variables,
as the user may not be expecting the autoenable to happen, or may
have those set for other purposes and not intend git-annex to use them.
* New annex.supportunlocked config that can be set to false to avoid
some expensive things needed to support unlocked files, if you do not
use them.
* Fix bug importing from a special remote into a subdirectory more than
one level deep, which generated unusual git trees that could confuse
git merge.
* borg: Fix a bug that prevented importing keys of type URL and WORM.
* borg: Support importing files that are hard linked in the borg backup.
* export: When a submodule is in the tree to be exported, skip it.
* import: When the previously exported tree contained a submodule,
preserve it in the imported tree so it does not get deleted.
* export --json: Fill in the file field.
* rmurl: When youtube-dl was used for an url, it no longer needs to be
prefixed with "yt:" in order to be removed.
* rmurl: If an url is both used by the web and also claimed by another
special remote, fix a bug that caused the url to to not be removed.
Joey Hess
committed
* unregisterurl: Fix a bug that caused an url to not be unregistered
when it is claimed by a special remote other than the web.
Joey Hess
committed
* whereis: Don't include yt: prefix when showing url to content
retrieved with youtube-dl.
* webdav: Work around some buggy webdav server behavior involving
renaming files.
* Make --debug also enable debugging in child git-annex processes.
* fsck: When --from is used in combination with --all or similar options,
do not verify required content, which can't be checked properly when
operating on keys.
* Sped up git-annex init in a clone of an existing repository.
* Improved display of errors when accessing a git http remote fails.
* Fix build with attoparsec-0.14.
-- Joey Hess <id@joeyh.name> Tue, 30 Mar 2021 13:01:17 -0400
git-annex (8.20210310) upstream; urgency=medium
* When non-annexed files in a tree are exported to a special remote,
importing from the special remote keeps the files non-annexed,
as long as their content has not changed, rather than converting
them to annexed files.
(Such a conversion will still happen when importing from a remote
an old git-annex exported such a tree to before; export the tree
with the new git-annex before importing to avoid that.)
* Added support for git-remote-gcrypt's rsync URIs, which access a remote
using rsync over ssh, and which git pushes to much more efficiently
than ssh urls.
* unregisterurl: New command.
* registerurl: Allow it to be used in a bare repository.
* Prevent combinations of options such as --all with --include.
* Fixed handling of --mimetype or --mimeencoding combined with
options like --all or --unused.
* Fix handling of --branch combined with --unlocked or --locked.
* Fix support for local gcrypt repositories with a space in their URI.
* uninit: Fix a small bug that left a lock file in .git/annex
* Windows: Correct the path to the html help file for 64 bit build.
* OSX dmg: Updated bundled git to 2.30.2 which fixes CVE-2021-21300.
-- Joey Hess <id@joeyh.name> Wed, 10 Mar 2021 13:58:15 -0400
git-annex (8.20210223) upstream; urgency=medium
* annex.stalldetection can now be set to "true" to make git-annex
do automatic stall detection when it detects a remote is updating its
* When annex.stalldetection is not enabled and a likely stall is
detected, display a suggestion to enable it.
* Commands can be added to git-annex, by installing a program in PATH
with a name starting with "git-annex-"
* Fix a reversion that made import of a tree from a special remote
result in a merge that deleted files that were not preferred content
of that special remote.
* Bugfix: fsck --from a ssh remote did not actually check that the
content on the remote is not corrupted.
* unannex, uninit: When an annexed file is modified, don't overwrite
the modified version with an older version from the annex.
Joey Hess
committed
* When adding files to an adjusted branch set up by --unlock-present,
add them unlocked, not locked.
* Fix an oddity in matching options and preferred content expressions
such as "foo (bar or baz)", which was incorrectly handled as if
it were "(foo or bar) and baz)" rather than the intended
"foo and (bar or baz)"
* Checksum as content is received from a remote git-annex repository,
rather than doing it in a second pass.
* Tahoe: Avoid verifying hash after download, since tahoe does sufficient
verification itself.
* unannex, uninit: Don't run git rm once per annexed file,
for a large speedup.
* When a git remote is configured with an absolute path, use that
path, rather than making it relative.
* get: Improve output when failing to get a file fails.
* Fix build on openbsd.
Thanks, James Cook for the patch.
* Include libkqueue.h file needed to build the assistant on BSDs.
-- Joey Hess <id@joeyh.name> Tue, 23 Feb 2021 14:40:14 -0400
git-annex (8.20210127) upstream; urgency=medium
enforced even more strictly. While numcopies can be violated in
concurrent drop situations involving special remotes that do not
support locking, mincopies cannot be. The default value has always
* Behavior change: When numcopies is set to 0, git-annex used to drop
content without requiring any copies. Now to get that (highly unsafe)
behavior, mincopies also needs to be set to 0.
* Behavior change: git-annex trust now needs --force, since unconsidered
use of trusted repositories can lead to data loss.
* Behavior change: --trust and --trust-glacier options no longer overrides
trust, since that can lead to data loss, which should never be enabled
by an option other than --force.
* add: Significantly speed up adding lots of non-large files to git,
by disabling the annex smudge filter when running git add.
* add --force-small: Run git add rather than updating the index itself,
so any other smudge filters than the annex one that may be enabled will
be used.
* Fix --time-limit, which got broken in several ways by some optimisations
in version 8.20201007.
* When syncing changes back from an adjusted branch to the basis branch,
include deletions of submodules.
Thanks, Kyle Meyer for the patch.
* Bug fix: export with -J could fail when two files had the same content.
Joey Hess
committed
* Bug fix: Fix tilde expansion in ssh urls when the tilde is the last
character in the url.
Thanks, Grond for the patch.
* Avoid crashing when there are remotes using unparseable urls.
Including the non-standard URI form that git-remote-gcrypt uses for rsync.
* Directory special remotes with importtree=yes now avoid unncessary
hashing when inodes of files have changed, as happens whenever a FAT
filesystem gets remounted.
* Fix a bug that prevented git-annex init from working in a submodule.
* Fix a bug in view filename generation when a metadata value ended with
"/" (or ":" or "\" on Windows)
* adjust: Fix some bad behavior when unlocked files use URL keys.
* smudge: Fix some bad behavior when git add is run on an unlocked
file that used an URL key.
* Added GETGITREMOTENAME to external special remote protocol.
* Windows: Work around win32 length limits when dealing with lock files.
-- Joey Hess <id@joeyh.name> Wed, 27 Jan 2021 11:09:25 -0400
git-annex (8.20201129) upstream; urgency=medium
* New borg special remote. This is a new kind of remote, that examines
borg backups of git-annex repositories, learns what files have been
backed up, and can restore files from the backup and so on. As well
as backups, it can be useful for archival storage, since borg can
efficiently store many related versions of files.
* New config annex.stalldetection, remote.name.annex-stalldetection,
which can be used to deal with remotes that stall during transfers,
or are sometimes too slow to want to use.
* Support special remotes that are configured with importtree=yes but
without exporttree=yes.
* Fix bug that made the next download after an empty file from a ssh
or tor remote fail.
* Avoid spurious "verification of content failed" message when downloading
content from a ssh or tor remote fails due to the remote no longer
having a copy of the content.
* Fix bug that matched include= and exclude= in preferred/required content
expressions relative to the current directory, rather than the path
from the top of the repository.
(Reversion introduced in version 8.20201116.)
* Fix hang on shutdown of external special remote using ASYNC protocol
extension.
(Reversion introduced in version 8.20201007.)
* Guard against running in a repo where annex.uuid is set but
* Avoid autoinit when a repo does not have annex.version or annex.uuid
set, but has a git-annex objects directory, suggesting it was used
Joey Hess
committed
* importfeed: Avoid using youtube-dl when a feed does not contain an
enclosure, but only a link to an url which youtube-dl does not support.
* initremote: Prevent enabling encryption with exporttree=yes or
importtree=yes.
Joey Hess
committed
* Windows: include= and exclude= containing '/' will also match filenames
that are written using '\'. (And vice-versa, but it's better to use '/'
for portability.)
* Fix a bug that could prevent getting files from an importtree=yes
remote, because the imported tree was allowed to be garbage collected.
* stack.yaml: Updated to lts-16.27.
-- Joey Hess <id@joeyh.name> Tue, 29 Dec 2020 12:52:58 -0400
git-annex (8.20201127) upstream; urgency=medium
* adjust: New --unlock-present mode which locks files whose content is not
present (so the broken symlink is visible), while unlocking files whose
content is present.
* Added annex.adjustedbranchrefresh git config to update adjusted
branches set up by git-annex adjust --unlock-present/--hide-missing.
* Fix hang when an external special remote program exited but
the stderr pipe to it was left open, due to a daemon having inherited
the file descriptor.
* Fix a bug that could make resuming a download from the web fail
when the entire content of the file is actually already present
locally.
* examinekey: Added a "file" format variable for consistency with find,
and for easier scripting.
* init: When writing hook scripts, set all execute bits, not only
the user execute bit.
* upgrade: Support an edge case upgrading a v5 direct mode repo
where nothing had ever been committed to the head branch.
* Made the test suite significantly less noisy, only displaying command
output when something failed.
* Fix building without the torrent library.
Thanks, Kyle Meyer.
* Fix build on Windows.
* Prevent windows assistant from trying (and failing) to upgrade
itself, which has never been supported on windows.
-- Joey Hess <id@joeyh.name> Fri, 27 Nov 2020 12:54:11 -0400
git-annex (8.20201116) upstream; urgency=medium
Joey Hess
committed
* move: Fix a regression in the last release that made move --to not
honor numcopies settings.
* sync --content: Fix a bug where a file that was not preferred content
could be transferred to a remote. This happened when the file got deleted
after the sync started running.
* Warn when adding a annex symlink or pointer file that uses a key that
is not known to the repository, to prevent confusion if the user has
copied it from some other repository.
* examinekey: Added two new format variables: objectpath and objectpointer
* examinekey: Added --migrate-to-backend
-- Joey Hess <id@joeyh.name> Mon, 16 Nov 2020 09:36:35 -0400
git-annex (8.20201103) upstream; urgency=medium
Joey Hess
committed
* move: Improve resuming a move that was interrupted after the object
was transferred, in cases where numcopies checks prevented the resumed
move from dropping the object from the source repository.
* When a special remote has chunking enabled, but no chunk sizes are
recorded (or the recorded ones are not found), speculatively try
chunks using the configured chunk size.
* Fixed some problems that prevented this command from working:
git submodule foreach git annex init
* Improve shutdown process for external special remotes and external
backends. Make sure to relay any remaining stderr from the process,
and potentially avoid the process getting a SIGPIPE if it writes to
stderr too late.
* Fix a bug that prevented linux standalone bundle from working on a fresh
install.
Joey Hess
committed
* Windows build changed to one done by the datalad-extensions project
using Github actions.
* Windows build now includes libmagic, so mimetype and mimeencoding
will work.
Thanks to John Thorvald Wodder II and Yaroslav Halchenko for their work
on this.
* view: Avoid using ':' from metadata when generating a view, because
it's a special character on Windows ("c:")
* Fix a memory leak introduced in the last release.
* add, import: Fix a reversion in 7.20191009 that broke handling
of --largerthan and --smallerthan.
* view: Fix a reversion in 8.20200522 that broke entering or changing views.
* Fix build on Windows with network-3.
* testremote: Display exceptions when tests fail, to aid debugging.
-- Joey Hess <id@joeyh.name> Tue, 03 Nov 2020 11:40:56 -0400
git-annex (8.20201007) upstream; urgency=medium
Joey Hess
committed
* --json output now includes a new field "input" which is the input
value (filename, url, etc) that caused a json object to be output.
* --batch combined with -J now runs batch requests concurrently for many
commands. Before, the combination was accepted, but did not enable
concurrency. Since the output of batch requests can be in any order,
--json with the new "input" field is recommended to be used,
to determine which batch request each response corresponds to.
Joey Hess
committed
* aws-0.22 improved its support for setting etags, which improves
support for versioned S3 buckets.
* Serialize use of C magic library, which is not thread safe.
This fixes failures uploading to S3 when using -J.
* add, addurl, importfeed, import: Added --no-check-gitignore option
for finer grained control than using --force.
* import: Check gitignores when importing trees from special remotes.
* addunused: Don't check .gitignores when adding files.
* Improve the "Try making some of these repositories available"
message, with some hints for the user for what to do.
* Improve --debug output to show pid of processes that are started and
stopped.
* sync --all: Sped up seeking to around twice as fast, by avoiding a
pass over the worktree files when preferred content expressions of the
local repo and remotes don't use include=/exclude=.
* Sped up seeking for files to operate on, when using options like
--copies or --in, by around 20%
* import --no-content: Check annex.largefiles, and import small
files into git, the same as is done when importing with content.
If the largefiles expression needs the file content available
(due to mimetype or mimeencoding being used), the import will fail.
* sync: When run without --content, import without copying from
importtree=yes directory special remotes.
(Other special remotes may support this later as well.)
* addurl: Avoid a redundant git ignores check for speed.
* upgrade: Avoid an upgrade failure of a bare repo in unusual circumstances.
Joey Hess
committed
* httpalso: Support being used with special remotes that do not have
encryption= in their config.
* Parse youtube-dl progress output, which lets progress be displayed
when doing concurrent downloads.
* Enable building with git-annex benchmark by default, only turning it
off when the criterion library is not installed.
* runshell: Fix a edge case where rm errors were sent to stdout, which
could confuse things parsing git-annex output.
* runshell: Update files atomically when preparing to run git-annex.
Joey Hess
committed
-- Joey Hess <id@joeyh.name> Wed, 07 Oct 2020 12:19:05 -0400
Joey Hess
committed
git-annex (8.20200908) upstream; urgency=medium
* Added httpalso special remote, which is useful for accessing
content stored on other remotes that is published by http.
* The external special remote protocol got an ASYNC extension.
This can be used by an external special remote to let a single process
perform concurrent actions, rather than multiple processes being
started, when that is more efficient.
* Retry transfers to exporttree=yes remotes same as for other remotes.
* import: Retry downloads that fail, same as is done for downloads generally.
* Limit retrying of failed transfers when forward progress is being made
to 5, to avoid some unusual edge cases where too much retrying could
result in far more data transfer than makes sense.
* Exposed annex.forward-retry git config, to configure the forward retry
behavior that git-annex has had for a long time.
* sync, assistant, merge: When merge.directoryRenames is not set,
default it it to "false", which works better with automatic merge
conflict resolution than git's ususual default of "conflict".
(This is not done when automatic merge conflict resolution is disabled.)
* resolvemerge: Improve cleanup of cruft left in the working tree
by a conflicted merge.
* Support git remotes where .git is a file, not a directory,
eg when --separate-git-dir was used.
* Fixed several cases where files were created without file mode bits
that the umask would usually set. This included exports to the
directory special remote, torrent files used by the bittorrent special
remote, hooks written by git-annex init, and some log files in .git/annex/
* Fix reversion in 7.20190322 that made addurl --file not be honored
when youtube-dl was used to download media.
Joey Hess
committed
* Fix reversion in 8.20200617 that made annex.pidlock being enabled
result in some commands stalling, particularly those needing to
autoinit.
* Display warning when external special remote does not start up
properly, or is not usable.
Joey Hess
committed
* Display a message when git-annex has to wait for a pid lock file
held by another process.
* test: Stop gpg-agent daemons that are started for the test framework's
gpg key.
* Removed the S3 and WebDAV build flags so these special remotes are
always supported.
-- Joey Hess <id@joeyh.name> Tue, 08 Sep 2020 14:20:11 -0400
* Added support for external backend programs. So if you want a hash
that git-annex doesn't support, or something stranger, you can write a
small program to implement it.
* Fix a bug in find --branch in the previous version.
* importfeed: Fix reversion that caused some '.' in filenames to be
replaced with '_'
Joey Hess
committed
* Fix a lock file descriptor leak that could occur when running commands
like git-annex add with -J. Bug was introduced as part of a different FD
leak fix in version 6.20160318.
* Fix a hang when using git-annex with an old openssh 7.2p2, which had
some weird inheriting of ssh FDs by sshd. Bug was introduced in
git-annex version 7.20200202.7.
* move, copy --to: Sped up seeking files by 2x.
* drop: Sped up seeking files to drop by 2x, and also some performance
improvements to checking numcopies.
Joey Hess
committed
* Deal with unusual IFS settings in the shell scripts for linux
standalone and OSX app.
Thanks, Yaroslav Halchenko
* Avoid complaining that a file with "is beyond a symbolic link"
when the filepath is absolute and the symlink in question is not
actually inside the git repository.
* Slightly sped up the linux standalone bundle.
* Support building with dlist-1.0
git-annex (8.20200720.1) upstream; urgency=medium
* Fix a bug in find --batch in the previous version.
-- Joey Hess <id@joeyh.name> Mon, 20 Jul 2020 19:39:11 -0400
git-annex (8.20200720) upstream; urgency=medium
Joey Hess
committed
* import: Added --no-content option, which avoids downloading files
from a special remote. Currently only supported by the directory
special remote.
* Honor annex.largefiles when importing a tree from a special remote.
Joey Hess
committed
(Except for when --no-content is used.)
* Fix a deadlock that could occur after git-annex got an unlocked
file, causing the command to hang indefinitely. Known to happen on
vfat filesystems, possibly others.
* Build with the http-client-restricted and git-lfs libraries when
available, otherwise use the vendored copy as before.
* testremote: Fix over-allocation of resources and bad caching,
including starting up a large number of external special remote processes.
(Regression introduced in version 8.20200501)
* test: Fix some test cases that assumed git's default branch name.
* importfeed: Added some additional --template variables:
itempubyear, itempubmonth, itempubday, itempubhour,
itempubminute, itempubsecond.
* Made several special remotes support locking content on them,
which allows dropping from other special remotes in some situations
Joey Hess
committed
where it was not possible before. Supported special remotes:
* Fix reversion that broke passing annex.* and remote.*.annex-*
git configs with -c. (Since version 8.20200330.)
* Bring back git-annex branch read cache. This speeds up some operations,
eg git-annex sync --content --all gets 20% faster.
* Fix a recently introduced bug that could cause a "fork: resource exhausted"
after getting several thousand files.
* Sped up the --all option by 2x to 16x by using git cat-file --buffer.
Thanks to Lukey for finding this optimisation.
* Sped up seeking for annexed files to operate on by a factor of nearly 2x.
* Sped up sync --content by 2x and other commands like fsck --fast and
whereis by around 50%, by using git cat-file --buffer.
* importfeed: Made checking known urls step around 15% faster.
* fsck: Detect if WORM keys contain a carriage return, and recommend
upgrading the key. (git-annex could have maybe created such keys back
in 2013).
* When on an adjust --hide-missing branch, fix handling of files that
have been deleted but the deletion is not yet staged.
-- Joey Hess <id@joeyh.name> Mon, 20 Jul 2020 14:40:51 -0400
git-annex (8.20200617) upstream; urgency=medium
* Added annex.skipunknown git config, that can be set to false to change
the behavior of commands like `git annex get foo*`, to not skip
over files/dirs that are not checked into git and are explicitly listed in
the command line.
* annex.skipunknown is planned to change to default to false in a
git-annex release in early 2022. If you prefer the current behavior,
you can explicitly set it to true.
* Try to enable special remotes configured with autoenable=yes
when git-annex auto-initialization happens in a new clone of an
existing repo. Previously, git-annex init had to be explicitly run to
enable them. Special remotes cannot display anything when autoenabled
this way, to avoid interfering with the output of git-annex query
commands.
* export: Added options for json output.
Joey Hess
committed
* addurl: Make --preserve-filename also apply when eg a torrent contains
multiple files.
* Fix a crash or potentially not all files being exported when
sync -J --content is used with an export remote.
* export: Let concurrent transfers be done with -J or annex.jobs.
* move --to, copy --to, mirror --to: When concurrency is enabled, run
cleanup actions in separate job pool from uploads.
* init: If lock probing stalls for a long time (eg a broken NFS server),
display a message to let the user know what's taking so long.
* init: When annex.pidlock is set, skip lock probing.
* Fix file descriptor leak when importing from a directory special remote
that is configured with exporttree=yes.
* Note that external special remote programs should not block SIGINT or
SIGTERM.
* Avoid creating the keys database during init when there are no unlocked
files, to prevent init failing when sqlite does not work in the
filesystem.
* import: Avoid using some strange names for temporary keys,
which broke importing from a directory special remote onto a vfat
filesystem.
* S3: The REDUCED_REDUNDANCY storage class is no longer cheaper so
stop documenting it, and stop offering it as a choice in the assistant.
* Improve display of problems auto-initializing or upgrading local git
remotes.
* When a local git remote cannot be initialized because it has no
git-annex branch or a .noannex file, avoid displaying a message about it.
* checkpresentkey: When no remote is specified, try all remotes, not
only ones that the location log says contain the key. This is what
the documentation has always said it did.
* Fix regression in external special remote handling: GETCONFIG did not
return a value that was set with SETCONFIG immediately before.
(Regression introduced in version 7.20200202.7)
* Fix bug that made initremote of extrnal special remotes with
embedcreds=yes or gpg encryption not store the creds in the git-annex
branch. git-annex-remote-googledrive one was special remote affected by
this bug.
(Regression introduced in version 7.20200202.7)
* Fix bug that made creds not be stored in git when a special remote
was initialized with gpg encryption, but without an explicit
embedcreds=yes.
(Regression introduced in version 7.20200202.7)
* Fix a annex.pidlock issue that made eg git-annex get of an unlocked
file hang until the annex.pidlocktimeout and then fail.
-- Joey Hess <id@joeyh.name> Wed, 17 Jun 2020 15:58:59 -0400
git-annex (8.20200522) upstream; urgency=medium
* Fix bug that made enableremote of S3 and webdav remotes, that
have embedcreds=yes, fail to set up the embedded creds, so accessing
the remotes failed. (Regression introduced in version 7.20200202.7)
* addurl, importfeed: Avoid adding filenames with leading '.', instead
it will be replaced with '_'.
Joey Hess
committed
* addurl, importfeed: Allow '-' in filenames, as long as it's not the
first character.
* addurl --preserve-filename: New option, uses server-provided filename
Joey Hess
committed
without any sanitization, but will fail if the filename has an obvious
security problem like using an escape sequence or trying to escape
the current directory.
* whereis: Added --format option.
* S3: Support signature=v4, to use S3 Signature Version 4.
Some S3 services seem to require v4, while others may only
support v2, which remains the default.
* upgrade: When upgrade fails due to an exception, display it.
* repair: Improve fetching from a remote with an url in host:path format.
* git-lfs repos that encrypt the annexed content but not the git repo
only need --force passed to initremote, allow enableremote and
autoenable of such remotes without forcing again.
* When accessing a remote fails, always display a reason why.
* Display a warning message when a remote uses a protocol, such as
git://, that git-annex does not support. Silently skipping such a
remote was confusing behavior.
* Also display a warning message when a remote, without a known uuid,
is located in a directory that does not currently exist, to avoid
silently skipping such a remote.
* sync: Avoid an ugly error message when nothing has been committed to
master yet and there is a synced master branch to merge from.
* Display a warning message when asked to operate on a file inside a
directory that's a symbolic link to elsewhere.
-- Joey Hess <id@joeyh.name> Fri, 22 May 2020 12:05:01 -0400
git-annex (8.20200501) upstream; urgency=medium
* Improve git-annex's ability to find the path to its program,
especially when it needs to run itself in another repo to upgrade it.
* adb: Better messages when the adb command is not installed.
* Sped up query commands that read the git-annex branch by around 9%.
* Various speed improvements gained by using ByteStrings for git refs and
shas.
* Fix a potential failure to parse git config.
Joey Hess
committed
* Support boolean git configs that are represented by the name of the
setting with no value, eg "core.bare" is the same as "core.bare = true".
* When parsing git configs, support all the documented ways to write
true and false, including "yes", "on", "1", etc.
* Fix --batch commands (and git-annex info) to accept absolute filenames
for unlocked files, which already worked for locked files.
* Avoid repeatedly opening keys db when accessing a local git remote
and -J is used.
* Avoid running a large number of git cat-file child processes when run
with a large -J value.
* Avoid running with more git check-attr and check-ignore processes than
there are CPU cores when run with a large -J value.
* get --from, move --from: When used with a local git remote, these used
to silently skip files that the location log thought were present on the
remote, when the remote actually no longer contained them. Since that
behavior could be surprising, now instead display a warning.
* external special remotes: remote.name.annex-readonly=true no longer
disables running the external special remote program. Instead, it just
makes the remote operate in a readonly mode, same as any remote.
To disable running the external special remote program, now need to set
remote.name.annex-externaltype=readonly. That is done when
git-annex enableremote is passed readonly=true.
* Stop storing readonly=true in remote.log of external special remotes;
it is a local setting only.
* sync: When some remotes to sync with are specified, and --fast is too,
pick the lowest cost of the specified remotes, do not sync with a
faster remote that was not specified.
* addurl: When run with --fast on an url that
annex.security.allowed-ip-addresses prevents accessing, display
a more useful message.
* When the required content is set to "groupwanted", use whatever
expression has been set in groupwanted as the required content of the
repo, similar to how setting required content to "standard" already
worked.
* Avoid a test suite failure when the environment does not let gpg be
tested due to eg, too long a path to the agent socket.
-- Joey Hess <id@joeyh.name> Fri, 01 May 2020 13:09:24 -0400
git-annex (8.20200330) upstream; urgency=medium
Joey Hess
committed
* fsck: Fix reversion in 8.20200226 that made it incorrectly warn
that hashed keys with an extension should be upgraded.
* add --force-small: Fix a bug that, when adding a symbolic link,
checked in the content of the file the symlink pointed to.
* add --force-small: Fix failure when passed a modified submodule.
Thanks, Kyle Meyer for the patch.
* When syncing changes back from an adjusted branch to the basis branch,
include changes to submodules.
Thanks, Kyle Meyer for the patch.
* webdav: Made exporttree remotes faster by caching connection to the
server.
* Fix a minor bug that caused options provided with -c to be passed
multiple times to git.
Joey Hess
committed
-- Joey Hess <id@joeyh.name> Mon, 30 Mar 2020 12:14:50 -0400
Joey Hess
committed
git-annex (8.20200309) upstream; urgency=medium
Joey Hess
committed
* Fix bug that caused unlocked annexed dotfiles to be added to git by the
smudge filter when annex.dotfiles was not set.
* Upgrade other repos than the current one by running git-annex upgrade
inside them, which avoids problems with upgrade code making assumptions
that the cwd will be inside the repo being upgraded. In particular,
this fixes a problem where upgrading a v7 repo to v8 caused an ugly
git error message.
* Fix upgrade failure when a file has been deleted from the working tree.
* Fix regression 1 month ago that prevented external special remotes from
using GETCONFIG to query values like "name".
* Improve behavior when a directory git-annex is writing to gets
unmounted. Previously it could in some cases re-create the mount point
and directory tree, and even write object contents to the wrong disk.
* Don't ignore --debug when it is followed by -c.
Joey Hess
committed
* whereis: If a remote fails to report on urls where a key
is located, display a warning, rather than giving up and not displaying
any information.
* When external special remotes fail but neglect to provide an error
message, say what request failed, which is better than displaying an
empty error message to the user.
* git-annex config: Only allow configs be set that are ones git-annex
actually supports reading from repo-global config, to avoid confusion.
* Avoid converting .git file in a worktree or submodule to a symlink
when the repository is not a git-annex repository.
* Linux standalone: Use md5sum to shorten paths in .cache/git-annex/locales
* Fix build with ghc 8.8 (MonadFail)
Thanks, Peter Simons
-- Joey Hess <id@joeyh.name> Mon, 09 Mar 2020 17:04:08 -0400
git-annex (8.20200226) upstream; urgency=medium
* v7 upgrades automatically to v8. The upgrade deletes old sqlite
databases, which may cause git-annex to need to do extra work to
regenerate the databases or due to not having the information from the
old databases available. Two notable cases are interrupted incremental
fscks and interrupted exports, both of which will restart from the
beginning.
* Improved serialization of filenames and keys to the sqlite databases,
avoiding encoding problems and speeding up operations on them.
* Add some missing indexes to sqlite databases. This will speed up
some things involving export and import remotes, and git-annex smudge.
Microbenchmarks show around 10-25% speedup of sqlite database operations.
* add: When adding a whole directory, any dotfiles found in it will
not be skipped, but will be added to git by default. This behavior
can be configured with annex.dotfiles.
* add: Removed special case for explicitly passing dotfiles,
that no longer adds them to the annex, but to git. This behavior
can be configured with annex.dotfiles.
* initremote, enableremote: Set remote.name.skipFetchAll when
the remote cannot be fetched from by git, so git fetch --all
will not try to use it.
* Fix some cases where handling of keys with extensions varied depending
on the locale.
* annex.maxextensionlength used to be the number of characters, not
bytes, when in a utf-8 locale. It's now always the number of bytes.
* Extended annex.security.allowed-ip-addresses to let specific ports
of an IP address to be used, while denying use of other ports.
* init --version: When the version given is one that automatically
upgrades to a newer version, use the newer version instead.
* Auto upgrades from older repo versions, like v5, now jump right to v8.
* Makefile: Support newer versions of cabal that use the new-build system.
-- Joey Hess <id@joeyh.name> Wed, 26 Feb 2020 18:49:58 -0400
git-annex (7.20200226) upstream; urgency=high
* Fix serious regression in gcrypt and encrypted git-lfs remotes.
Since version 7.20200202.7, git-annex incorrectly stored content
on those remotes without encrypting it.
If your remotes are affected, you will want to make sure to delete
any content that git-annex has stored on them that is not encrypted!
* initremote: Fix regression in parsing the exporttree= parameter.
* info: Fix display of the encryption value.
(Some debugging junk had crept in.)
* Bugfix to getting content from an export remote with -J, when the
export database was not yet populated.
* Bugfix: export --tracking (a deprecated option) set
annex-annex-tracking-branch, instead of annex-tracking-branch.
-- Joey Hess <id@joeyh.name> Wed, 26 Feb 2020 17:18:16 -0400
git-annex (7.20200219) upstream; urgency=medium
* Added sync --only-annex, which syncs the git-annex branch and annexed
content but leaves managing the other git branches up to you.
* Added annex.synconlyannex git config setting, which can also be set with
git-annex config to configure sync in all clones of the repo.
* fsck --from remote: Fix a concurrency bug that could make it incorrectly
detect that content in the remote is corrupt, and remove it, resulting in
data loss.
* When git-annex is built with a ssh that does not support ssh connection
caching, default annex.sshcaching to false, but let the user override it.
* Improve warning messages further when ssh connection caching cannot
be used, to clearly state why.
Joey Hess
committed
* Avoid throwing fatal errors when asked to write to a readonly
git remote on http.
* Fix support for repositories tuned with annex.tune.branchhash1=true,
including --all not working and git-annex log not displaying anything
for annexed files.
-- Joey Hess <id@joeyh.name> Wed, 19 Feb 2020 12:44:43 -0400
git-annex (7.20200204) upstream; urgency=medium
* Fix build with persistent-template 2.8.0.
Joey Hess
committed
* Makefile: Really move the fish completion to the
vendor_completions.d directory.
-- Joey Hess <id@joeyh.name> Tue, 04 Feb 2020 14:30:55 -0400
git-annex (7.20200202.7) upstream; urgency=medium
* add: --force-large/--force-small options make it easier to override
annex.largefiles configuration (and potentially safer as it avoids
bugs like the smudge bug fixed in the last release).
* reinject --known: Fix bug that prevented it from working in a bare repo.
* Support being used in a git repository that uses sha256 rather than sha1.
* initremote, enableremote: Be stricter about rejecting invalid
configuration parameters for remotes, particularly things like foo=true
when foo=yes is expected.
* initremote, enableremote: Reject unknown configuration parameters
provided to these commands.
* initremote: Added --whatelse option, to show additional
configuration parameters you might want to set. Eg:
git annex initremote type=directory encryption=none --whatelse
* Added LISTCONFIGS to external special remote protocol. Special remote
programs that use GETCONFIG/SETCONFIG are recommended to implement it.
* init: Avoid an ugly error message when http remote has no git-annex
uuid configured.
* Support git remotes that need http basic auth to be accessed,
using git credential to get the password.
* Display a warning when concurrency is enabled but ssh connection caching
is not enabled or won't work due to a crippled filesystem.
* Makefile: Move the fish completion to the vendor_completions.d directory.
* Fixed a test suite failure when run in the C locale.
-- Joey Hess <id@joeyh.name> Sun, 02 Feb 2020 00:00:00 -0400
git-annex (7.20191230) upstream; urgency=medium
* Optimised processing of many files, especially by commands like find
and whereis that only report on the state of the repository. Commands
like get also sped up in cases where they have to check a lot of
files but only transfer a few files. Speedups range from 30-100%.
* Added build dependency on the filepath-bytestring library.
* Fixed an oversight that had always prevented annex.resolvemerge
from being honored, when it was configured by git-annex config.
* annex.largefiles can be configured by git-annex config,
to more easily set a default that will also be used by clones,
without needing to shoehorn the expression into the gitattributes file.
The git config and gitattributes override that.
* annex.addunlocked can be set to an expression with the same format used by
annex.largefiles, when you want to default to unlocking some files but
not others.
* annex.addunlocked can be configured by git-annex config.
* git-annex-config --set/--unset: No longer change the local git config
setting, except for in the special case of annex.securehashesonly.
Joey Hess
committed
* Improve file ordering behavior when one parameter is "." and other
parameters are other directories.
* smudge bugfix: When annex.largefiles=anything, files that were already
stored in git, and have not been modified could sometimes be converted
to being stored in the annex. Changes in 7.20191024 made this more
of a problem. This case is now detected and prevented.
-- Joey Hess <id@joeyh.name> Mon, 30 Dec 2019 12:43:30 -0400
* git-lfs: The url provided to initremote/enableremote will now be
stored in the git-annex branch, allowing enableremote to be used without
an url. initremote --sameas can be used to add additional urls.
* git-lfs: When there's a git remote with an url that's known to be
used for git-lfs, automatically enable the special remote.
* sync, assistant: Pull and push from git-lfs remotes.
* Fix bug that made bare repos be treated as non-bare when --git-dir
was used.
* inprogress: Support --key.
* Sped up many git-annex commands that operate on many files, by
avoiding reserialization of keys.
find is 7% faster; whereis is 3% faster; and git-annex get when
all files are already present is 5% faster
* Stop displaying rsync progress, and use git-annex's own progress display
for local-to-local repo transfers.
* benchmark: Changed --databases to take a parameter specifiying the size
of the database to benchmark.
* benchmark --databases: Display size of the populated database.
* benchmark --databases: Improve the "addAssociatedFile (new)"
benchmark to really add new values, not overwriting old values.
* Windows: Fix handling of changes to time zone. (Used to work but was
broken in version 7.20181031.)
-- Joey Hess <id@joeyh.name> Wed, 18 Dec 2019 13:53:51 -0400
git-annex (7.20191114) upstream; urgency=medium
* Make --json-error-messages capture more errors,
particularly url download errors.
Joey Hess
committed
* Fix a crash (STM deadlock) when -J is used with multiple files
that point to the same key.
* linuxstandalone: Fix a regression that broke git-remote-https.
* OSX git-annex.app: Fix a problem that prevented using the bundled
git-remote-https, git-remote-http, and git-shell.
-- Joey Hess <id@joeyh.name> Thu, 14 Nov 2019 21:57:59 -0400
* init: Fix bug that lost modifications to unlocked files when init is
re-ran in an already initialized repo.
* benchmark: Add --databases to benchmark sqlite databases.
git-annex (7.20191024) upstream; urgency=medium
* Changed git add/git commit -a default behavior back to what it was
before v7; they add file contents to git, not to the annex.
(However, if a file was annexed before, they will still add it to
the annex, to avoid footgun.)
* Configuring annex.largefiles overrides that; once git-annex has
been told which files are large git add/git commit -a will annex them.
* Added annex.gitaddtoannex configuration. Setting it to false prevents
git add from adding files to the annex even when annex.largefiles
is configured. (Unless the file was annexed before.)