Fix#4333, as `is_ignored_file` and `is_ignored_dir` and `__call__`
share the same call `DvcIgnorePatterns.matches`, we only need to focus
on the out of repo situation.
1. Add tests for the `is_ignored_file` and `is_ignored_dir` for the out
of repo condition.
2. Now `is_ignored_file` and `is_ignored_dir` returns false for those
files outside the repo.
* Allow ignoring subrepos in dvcignore
I implemented this inside dvcignore, and it
should not require "internal" APIs hack as we discussed before.
Right now, `ignore_subrepos` will only work for the paths of subrepo,
not anything inside it. And the API user is more or less only
RepoTree, so should be safe (we switch DvcTree and repo.tree based
on path prefixes, so this is only required during `walk`, so as it
shows the repo that were previously dvcignored and so that we could
switch the trees later during walk.
So, if dir is `dir -> repo`
So, `tree.walk("dir")` will return `[]`,
whereas `tree.walk("dir", ignore_subrepos=False)` will return `["repo"]`
But, successive walk will just return `("repo", [], [])`, and could
just be ignored as a side-effect.
* Fix tests
* Update some tests first
fix#3736
* first edition
* solve failure in Windows
* For Windows ci
* Some help ducuments issue.
* Update dvc/ignore.py
abspath
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Refactor with OutOfWorkSpaceError
* Solve a bug
* Update dvc/command/check_ignore.py
Co-authored-by: Jorge Orpinel <[email protected]>
* Update dvc/command/check_ignore.py
Co-authored-by: Jorge Orpinel <[email protected]>
* Update dvc/command/check_ignore.py
* Revert "Refactor with OutOfWorkSpaceError"
This reverts commit 27eec49ed67346c89157edea998b6b5fa81d88f5.
* Two change request
1. Argument `targets`'s description.
2. Error handling of `_get_normalize_path`
* Update dvc/main.py
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* `check_ignore` now only accept one path a time
1. Add a new test for the out side repo cases
2. check ignore now only check one file not file lists
Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: karajan1001 <[email protected]>
Co-authored-by: Jorge Orpinel <[email protected]>
* Update
1. unit test update
2. two might cause bugs
* dvc ignore
* Some refactoring
* clean
* Config tree use
* Solve sub dir repo problem. Add more tests
* typo error
* Change request
Co-authored-by: karajan1001 <[email protected]>
* dvc: get rid of is_walking_tree
No longer needed. Some applications of `isinstance()` will be refactored
in the future, as trees will just provide the needed functionality.
* dvc: move GitTree to dvc/tree/
* dvc: rename *RemoteTree to *Tree
* dvcignore: move file/dir/parent checks into DvcIgnoreFilter
* dvc: get rid of CleanTree
CleanTree is a very awkward wrapper that spreads the context across
the codebase and results in unexpected behaviour when using it.
This PR starts moving dvcignore-related logic into the trees themselves
(it makes a lot of sense, kinda like `state`) so they could deal with
it however they like.
There are at least two temporary ugly parts about this PR:
1) dvcignore is used by individual trees and not packed into
tree/base.py yet;
2) `dvcignore_root` argument. This one is caused by the dvcignore trying
to collect everything topdown starting from the certain root dir. What
it should do instead is for a certain path that is being checked look up
the tree through the parents until it finds repo root (.dvc dir) and
then stop. That would handle subrepos as well. At the same time we need
to leverage existing dvcignore trie structure to cache those results.
Working tree is really just a regular local tree and should be
used by outputs when trying to compute a hash for themselves.
We didn't use it previously because local tree was embeded into
the local remote class.
Related to #4050
* Remove Duplicate ignore Match
* Continue Optimize Dvcignore
fix#3869
* Add a new test with multi ignore files
* Solve merging two dvc files
* Solve Code Climate
* For Windows
* Complete addition of patterns. Add one test
* Systematic test
* Change request
* Change request
* Seperate path sepcification math
* Rename and add comment
* rename change_dirname to private
* Update dvc/pathspec_math.py
list comprehension
Co-authored-by: Alexander Schepanovski <[email protected]>
* Change request
* Update dvc/ignore.py
Co-authored-by: karajan1001 <[email protected]>
Co-authored-by: Alexander Schepanovski <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Add more tests according to .gitignore
* Test update
* Update test
* Solve question of path
* update one test
* Update 2 tests
* Add one tests
* comment those failed tests caused by other package
* parametrize some tests
Co-authored-by: karajan1001 <[email protected]>
* dvcignore: ignore paths outside of CleanTree root
* remote: local remote/cache should always use its own working tree
* repo tree (CleanTree) instance should not be relevant to local cache
operations. When the dvcignore CleanTree is relevant, repo.tree should
be used. For local remote/cache operations, the local remote work_tree
should be used.
* state: tie state to local cache working tree
* tests: update tests
* state: use os.path.exists instead of WorkingTree.exists in some cases
* BaseRemote: start moving tree functions into BaseRemoteTree
* remote: add LocalRemoteTree
* remote: add cloud remote trees
* remote: move remove(), makedirs() into tree
* remote: move move(), copy(), symlink(), hardlink(), reflink() into tree
* fix attributes for moved remote methods
* tests: update unit tests for moved remote tree methods
* tests: mv unit/remote/test_remote_dir.py unit/remote/test_remote_tree.py
* fix attributes for moved remote tree functions
* tests: update func tests for moved remote tree methods
* test fixes
* fix DS warnings
* ignore: dvcignore root is distinct from tree_root
* for subrepos, dvcignore root should be the subrepo's .dvcignore, not
host repo's .dvcignore
* tests: add test for subrepo dvcignore behavior
* drop python 3.5 support
* dvc: remove fspath() use inside os.path calls
* dvc: replace dvc.compat.fspath with os.fspath
* tests: replace dvc.compat.fspath with os.fspath
* utils: relpath() should use fspath/str
* utils: os.readlink doesn't support path-like obj on Windows until 3.8
* state: use ignore file handler when accessing metadata
* fs: don't add dir size to overall size for dir
* ignore: use posix-like path on windows
* ignore: remove tree from dvc_walk arguments
* ignore: handle DvcIgnoreFilter creation outside of DVC repo
* ignore: throw on dvcignore file under output dir, but not dependency dir
* ignore: move ignore initialization to Repo init
* ignore: work in progress
* ignore: work in progress 2
* ignore: throw on .dvcignore upon collecting dir
* ignore: refactor
* ignore: loading upper level ignores in stages()
* Update dvc/repo/__init__.py
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* dvcignore review refactor
Saugat Pachhai
Paweł Redzyński
karajan1001
AnatolyandRuslan Kuprieiev
Peter Rowlands (변기호)
Casper da Costa-Luis
nik123
Mr. Outis