We used `Popen` just to provide a general way of launching our daemons
on both Windows and *nix, but it is causing us issues when dvc is
launched from some exotic places (like nix package) or through `main()`
helper from a script that is not dvc.
This patch makes us just use `main()` directly in our double-forked
process.
Windows implementation has to use the `Popen` mechanism, because this is
the only way to daemonize a process on Windows. This means that dvc on
Windows is still susceptible to this problem, but there is nothing we
can do about it right now :(
Fixes#4206Fixes#4165
* Update
1. unit test update
2. two might cause bugs
* dvc ignore
* Some refactoring
* clean
* Config tree use
* Solve sub dir repo problem. Add more tests
* typo error
* Change request
Co-authored-by: karajan1001 <[email protected]>
* erepo: use shallow clones when rev is branch or tag name
* tests: test erepo shallow clone behavior
* erepo: fix detached head after shallow cloning a tag
* move unshallow logic into its own function
* erepo: default rev should be origin HEAD not local HEAD
* use functools.partial
* review fixes
* comment shallow clone behavior in erepo
* don't duplicate unnecessary fetch/pull call
* dvc: get rid of is_walking_tree
No longer needed. Some applications of `isinstance()` will be refactored
in the future, as trees will just provide the needed functionality.
* dvc: move GitTree to dvc/tree/
* dvc: rename *RemoteTree to *Tree
* key api
* Restyled by black
* test
* fixes
* test
* Restyled by black
* Trigger CI
* default val
* remove redundant default
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Update tests/unit/remote/test_s3.py
* rename fields
* update test
* key_id->secret_key_id
Co-authored-by: Restyled.io <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* experiments: add initial experiments.show
* only includes metrics + params (no code/data information)
* experiments: basic `dvc experiments show` functionality
* experiments: clone and run repro inside clone workspace
* repro: add -e/--experiment option
* experiments: hash experiments to identify duplicates
* experiments: update show
* experiments: checkout experiment after running repro
* experiments: add `dvc experiments checkout`
* experiments: add simple `dvc experiments diff` command
* just shows combined output from `metrics diff` and `plots diff` for
now
* add very simple tests
* add some useful output messages
* experiments: suppress help messages while feature is in development
* experiment related commands will be accessible but hidden from the
default command help messages
* add simple experiments command tests
* experiments: disable feature by default
* experiments only enabled in test environment (DVC_TEST) or when
core.experiments config option is true
* use fetch on checkout missing revs
* dvcignore: move file/dir/parent checks into DvcIgnoreFilter
* dvc: get rid of CleanTree
CleanTree is a very awkward wrapper that spreads the context across
the codebase and results in unexpected behaviour when using it.
This PR starts moving dvcignore-related logic into the trees themselves
(it makes a lot of sense, kinda like `state`) so they could deal with
it however they like.
There are at least two temporary ugly parts about this PR:
1) dvcignore is used by individual trees and not packed into
tree/base.py yet;
2) `dvcignore_root` argument. This one is caused by the dvcignore trying
to collect everything topdown starting from the certain root dir. What
it should do instead is for a certain path that is being checked look up
the tree through the parents until it finds repo root (.dvc dir) and
then stop. That would handle subrepos as well. At the same time we need
to leverage existing dvcignore trie structure to cache those results.
Working tree is really just a regular local tree and should be
used by outputs when trying to compute a hash for themselves.
We didn't use it previously because local tree was embeded into
the local remote class.
Related to #4050
Outputs and dependencies don't really care about the remotes, as they
don't do any pushing or pulling. What they really need are the trees,
so let's use them directly.
Part of #4050
* Remove Duplicate ignore Match
* Continue Optimize Dvcignore
fix#3869
* Add a new test with multi ignore files
* Solve merging two dvc files
* Solve Code Climate
* For Windows
* Complete addition of patterns. Add one test
* Systematic test
* Change request
* Change request
* Seperate path sepcification math
* Rename and add comment
* rename change_dirname to private
* Update dvc/pathspec_math.py
list comprehension
Co-authored-by: Alexander Schepanovski <[email protected]>
* Change request
* Update dvc/ignore.py
Co-authored-by: karajan1001 <[email protected]>
Co-authored-by: Alexander Schepanovski <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* remove: remove dvc.yaml, dvc.lock if they are empty
* tests: remove: simplified dvc.yaml in test_remove_stage_removes_dvcfiles_if_no_stages_left
* tests: remove: minor fixes
* tests: remove: simplified test_remove_stage_removes_dvcfiles_if_no_stages_left
* remove: independent removal of dvc.lock and dvc.yaml
The falsy values were being skipped when merging params values
from dvc.yaml and dvc.lock, showing the values as always changed.
These values were never added to `info`, but were found in
`self.params` making them appear as changed.
Fixes#4184
* Add more tests according to .gitignore
* Test update
* Update test
* Solve question of path
* update one test
* Update 2 tests
* Add one tests
* comment those failed tests caused by other package
* parametrize some tests
Co-authored-by: karajan1001 <[email protected]>
Usually we need it to parametrize tests with, so there is no
reason to duplicate the code. I've left `local_remote` fixture
just because it is so convenient and often used in a lot of tests.
Ruslan Kuprieiev
karajan1001andkarajan1001
Peter Rowlands (변기호)
Casper da Costa-Luis
Paweł Redzyński
faucct
nik123
Dmitry PetrovandRuslan Kuprieiev
Saugat Pachhai