* tree: make open method use mode and encoding
* api: try to open using self.tree if output not found
* repo: rename OutputMissingException
* test: api: test adding not cached output
* api: open: exists test instead of try
* repo: open: try opening from tree first
* repo: open: handle missing and not cached files
* repo: open: separate find out logic from opening
* checkout: raise on non existing dir cache
* checkout: don't throw on failed load_dir_cache
* checkout: raise on failed checkouts, only after all checkouts are perfromed
* remote: base: raise on failed dir cache loading
* rename exception for failed checkout
* wip - pull is working
* use fetch instead of reproduce
* test pull and fetch
* style changes
* fetch: catch config error on fetch instead of get_remote
* output: remove unused guard clause
* fetch: define a more specific error instead of ConfigError
* fetch: display number of errors correctly
* exceptions: remove "Failed" prefix to download/upload
* fetch: catch other exceptions that might halt the download
* fetch: log catched exceptions
* fetch: reformat exception
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* fix formatting
* get: show error message when path is a DVC-file
Fix#2468
* exceptions: rename output for data artifact
* get: use method to indetify if file is a DVC-file
* exceptions: change GetDVCFileError message
* state: use ignore file handler when accessing metadata
* fs: don't add dir size to overall size for dir
* ignore: use posix-like path on windows
* ignore: remove tree from dvc_walk arguments
* ignore: handle DvcIgnoreFilter creation outside of DVC repo
* ignore: throw on dvcignore file under output dir, but not dependency dir
* ignore: move ignore initialization to Repo init
* ignore: work in progress
* ignore: work in progress 2
* ignore: throw on .dvcignore upon collecting dir
* ignore: refactor
* ignore: loading upper level ignores in stages()
* Update dvc/repo/__init__.py
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* dvcignore review refactor
Replaced Path* dataclasses with PathInfo/URLInfo/CloudURLInfo, which
know how parse, serialize, stringify and join themselves. They also
provide useful accessors like part of an url or pathlib.Path like
.parent or .name.
Along the way two families of bugs where fixed and should never occur in
the future:
- paths not matched as strings anymore, e.g. using str.startswith()
- Output.path is eliminated as ambiguous
Both families might had caused undefined surprising behavior when local
and remote paths clash.
* test: remove never used copy of load_stage_file()
* dvc: save comments and custom formatting in stage files
* Add a couple of TODOs
* test: test repro doesn't clear comments and formatting in .dvc
As part of this I started writing pytest-like tests and added commonly
used fixtures.
* test: simplify and prettify dvc formatting retained test
Also test that line comments after md5 fields are retained.
* test: fix tests
* dvc: fix stage and metrics opens not closed
* dvc: always use utf-8 in stage files
It used to encode in system default encoding. Not sure if PyYAML was
handling this silently or we were just lucky all the way.
* test: unit test apply_diff()
* test: remove TODOs and ASK
* dvc: polish dvc-formatting PR
* dvc: move stage load/dump utils to a new stage utils file
* dvc: clean up apply_diff()
* test: test stage md5 ignores comments
* test: remove pytests starting marker
* s3: fixed wrong etag when copying multipart objects
The etag of multipart objects depends of the number of parts, when copying to the cache we should do so in the same number of parts that the original object was moved/uploaded in.
Fixes part of #1410
* s3: added check on copy for equal etag
* s3: added specific exception for ETag mismatch
* s3: use multipart copy to preserve etags
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* test: add tests for etag preservation on s3
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* test: requirements: use dev version moto
Specifically because of this https://github.com/spulec/moto/issues/1941
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* test: requirements: install dev moto without -e
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* test: stop using moto
Turned out to be quite buggy.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
1) split Project's methods into separate files;
2) rename Project into Repo(similar to gitpython's Repo)
Signed-off-by: Ruslan Kuprieiev <[email protected]>
flake8 provides some additional features like unused vars check, etc, so
it makes total sense to use it as well.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Paweł Redzyński
Ruslan Kuprieiev
Jorge Orpinel
Casper da Costa-Luis
Yossi Farjoun
Mr. OutisandRuslan Kuprieiev
Alexander Schepanovski
Jorge Orpinel
Paulo Olveira
Ivan Shcheklein