Commit Graph
35 Commits
Author SHA1 Message Date
Aman Sharma 74850fe728 Move walk_files to fs.py 2020-01-09 14:55:49 +05:30
Mr. Outis 5611ccb43a exceptions: remove cause 2020-01-02 20:43:24 -06:00
Mr. Outis fd82402d34 py3: use new magic from PEP 3135 2020-01-02 16:24:31 -06:00
Mr. Outis 886afdb203 py3: get rid of is_py checks 2019-12-30 18:53:36 -06:00
pawel 691d771d56 remove CleanTree from GitTree completely 2019-12-30 16:50:43 +01:00
pawel d8c0ae6909 remove most occurences of dvc_walk 2019-12-30 14:19:43 +01:00
Paweł Redzyński 956fec02ce repo: move dvcignore from repo to tree 2019-12-30 14:19:43 +01:00
Ruslan Kuprieiev c81795c047 dvc: support granularity for fetch/pull/push/status/checkout
Using output path or a subdir/subfile path within an output now works
with `dvc push/pull/fetch/status -c` commands. Other commands don't
support the same logic for now, as there are some questions about what
should commands like `dvc remove` do when given a specific output path.

Example:
    dvc add data
    dvc pull data/subdir # will only pull files within data/subdir

Related to #2458
2019-12-29 01:31:08 +02:00
Ruslan Kuprieiev da49badbfe Merge branch 'master' into relink 2019-12-13 20:35:02 +02:00
pawel 26ef77cf96 checkout: test: refactor to dir helpers 2019-12-11 13:54:44 +01:00
Alexander Schepanovski 2f79e362f4 dvc: add checkout --relink 2019-12-09 13:25:32 +03:00
Alexander Schepanovski 4ab4e43606 dvc: return to using .changed() shortcut in .checkout() 2019-12-02 14:38:39 +01:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Ruslan Kuprieiev 59e62a7227 checkout: accept multiple targets (#2640)
Make `Repo.checkout` accept a list of targets, instead of having to feed
it targets one-by-one. This allows us to properly handle checkout
errors.

Fixes #2580
2019-10-21 17:06:21 +03:00
Ruslan Kuprieiev 381601f4ee dvc: cache graph/stages/pipelines (#2529)
* dvc: cache graph/stages/pipelines

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* repo: check_dag: don't reset the cache

https://github.com/iterative/dvc/pull/2529#discussion_r327711928

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* repo: rename check_dag(stages) -> check_modified_graph(new_stages)

https://github.com/iterative/dvc/pull/2529#discussion_r327708439

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* repo: refactor "active" logic and remove it from Repo class

* repo: use tree.setter to automate cache reset

As noted by @Suor's , this is much less prone to errors.

* repo: move graph-related staticmethods into graph.py

As per @Suor's request.

* dvc: remove unused from_directory

https://github.com/iterative/dvc/pull/2529#discussion_r328079258

* repo: use _tree instead of __tree

https://github.com/iterative/dvc/pull/2529#discussion_r328071115

* repo: add a note about cache invalidation on repo.lock release

https://github.com/iterative/dvc/pull/2529#discussion_r328071635

* repo: rename reset() -> _reset()

https://github.com/iterative/dvc/pull/2529#discussion_r328079899

* pipeline: use graph helpers

https://github.com/iterative/dvc/pull/2529#discussion_r327502293

* repo: use list comprehension for collect()

https://github.com/iterative/dvc/pull/2529#discussion_r328076651

* repo: graph: only use first cycle

https://github.com/iterative/dvc/pull/2529#discussion_r328081346

* dvc: don't use G.nodes()

https://github.com/iterative/dvc/pull/2529#discussion_r328089671
https://github.com/iterative/dvc/pull/2529#discussion_r328089878
2019-09-26 07:42:29 +03:00
Paweł Redzyński be012e70cf checkout: raise on non existing dir cache (#2514)
* checkout: raise on non existing dir cache

* checkout: don't throw on failed load_dir_cache

* checkout: raise on failed checkouts, only after all checkouts are perfromed

* remote: base: raise on failed dir cache loading

* rename exception for failed checkout
2019-09-24 16:31:36 +03:00
pawel d4c9c71a2f remote: checkout: reuse _checkout_file functionality 2019-09-11 11:26:30 +02:00
Casper da Costa-Luis 093ad20a84 flake8 unused imports 2019-08-17 16:15:15 +01:00
Casper da Costa-Luis 534d6fdb0c remove defunct test, extraneous comments
- https://github.com/iterative/dvc/pull/2333#discussion_r314927966
2019-08-17 15:43:54 +01:00
Casper da Costa-Luis adf05e6ff0 minor rename 2019-08-16 22:32:47 +01:00
Casper da Costa-Luis b018933b5a fix py2 MagicMock(sys.stderr).encoding in tests 2019-08-16 22:26:19 +01:00
Casper da Costa-Luis f88c2fe5fc remove old test 2019-08-14 00:19:44 +01:00
Casper da Costa-Luis 120dfff05a naive test update 2019-08-11 01:21:50 +01:00
Casper da Costa-Luis ee4248df92 fix and update more tests 2019-08-11 01:17:42 +01:00
Paweł RedzyńskiandRuslan Kuprieiev 393d926b74 state: use dvcignore when accessing metadata (#2161)
* state: use ignore file handler when accessing metadata

* fs: don't add dir size to overall size for dir

* ignore: use posix-like path on windows

* ignore: remove tree from dvc_walk arguments

* ignore: handle DvcIgnoreFilter creation outside of DVC repo

* ignore: throw on dvcignore file under output dir, but not dependency dir

* ignore: move ignore initialization to Repo init

* ignore: work in progress

* ignore: work in progress 2

* ignore: throw on .dvcignore upon collecting dir

* ignore: refactor

* ignore: loading upper level ignores in stages()

* Update dvc/repo/__init__.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* dvcignore review refactor
2019-07-12 11:50:22 +03:00
Ruslan Kuprieiev 2013e21aaf checkout: safe_remove output if no checksum is found
Fixes #2146

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-05 02:59:57 +03:00
Ruslan Kuprieiev 918ef646dc appveyor: test py2 and py3
Related to #2166

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-23 13:33:47 +03:00
prd 0615ff8730 pathinfo: extract relative path method to standalone method 2019-06-18 12:08:19 +02:00
pawel ac5ca4656a remote: progress callback for unpacked dir creation 2019-06-10 20:19:47 +02:00
David Příhoda e11c3cbbc5 Add repro -R, support dirs & stage files when running pull/push/repro -R. 2019-05-27 12:48:46 +02:00
Tadas Audinis ee6a0b2875 tests: resolve deprecation warnings (#2028)
* tests: resolve deprecation warnings

Fixes #2021

* tests: resolve deprecation warnings

Fixes #2021
2019-05-24 03:12:36 +03:00
Alexander Schepanovski 7478e7d4b0 test: clean up test dirs (#1904)
* test: clean up test dirs

Closes #1770. Partially does #1888.

* dvc: close anything opened

* test: finalize git repo explicitly to remove test dir later

This was previously an issue on Windows.
2019-04-25 14:55:39 +03:00
Ruslan Kuprieiev 00cfb04fea remote: generalize file/dir handling
Fixes #1654
Fixes #1655

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 05:09:48 +03:00
Alexander Schepanovski 0c867399bb dvc: retain formatting and comments in .dvc files (#1885)
* test: remove never used copy of load_stage_file()

* dvc: save comments and custom formatting in stage files

* Add a couple of TODOs

* test: test repro doesn't clear comments and formatting in .dvc

As part of this I started writing pytest-like tests and added commonly
used fixtures.

* test: simplify and prettify dvc formatting retained test

Also test that line comments after md5 fields are retained.

* test: fix tests

* dvc: fix stage and metrics opens not closed

* dvc: always use utf-8 in stage files

It used to encode in system default encoding. Not sure if PyYAML was
handling this silently or we were just lucky all the way.

* test: unit test apply_diff()

* test: remove TODOs and ASK

* dvc: polish dvc-formatting PR

* dvc: move stage load/dump utils to a new stage utils file

* dvc: clean up apply_diff()

* test: test stage md5 ignores comments

* test: remove pytests starting marker
2019-04-20 01:12:53 +03:00
Ruslan Kuprieiev 6f11f1f51f test: move func tests to tests/func
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-16 20:19:22 +03:00