Commit Graph
25 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 803ffaec57 remote: local: don't auto-create directory in __init__
Imagine the scenario of shared cache. To set it up, user would do:

```
dvc config cache.dir /path
dvc config cache.shared group
```

Now, when he runs any dvc operation that involves the Repo instance,
RemoteLOCAL will also be initialized and it will currently auto-create
the directory with a regular `os.mkdir`, which is not using group
permissions. To solve that, we can't simply use `self.makdirs` instead,
as it will cause same issues if you run any dvc command before
`dvc config cache.shared group`. So what we should do is create
directories on demand, instead of creating it in advance, same as we do
in other remotes.

The change itself is pretty trivial and involved removing that `mkdir`
line, adjusting tests to not rely on cache dir existence and also
adjusting `dvc version` to not rely on cache directory existance.

Fixes #2563
2019-10-14 15:50:49 +03:00
Mr. Outis 9496f34f45 tests: Windows specific filename for gitignore 2019-09-30 11:28:38 -05:00
Mr. Outis c9c36c2049 git: escape filename before ignoring it
Fix #2043
2019-09-29 23:16:46 -05:00
Ruslan Kuprieiev 2c003fdc04 Merge pull request #2426 from mroutis/close-1652
ssh: support linking
2019-09-09 20:48:22 +03:00
Paweł Redzyński 0a214ae15a test: add: remove TestDvcGit (#2467) 2019-09-06 23:15:26 +03:00
Mr. Outis 6eb26de856 fix tests 2019-09-06 11:32:31 -05:00
5ec1d274ae remote: base: don't checkout existing file (#2358)
* remote: base: don't checkout existing file

* remote: _checkout_file: split to smaller methods

* state: local: verify if file needs checkout considering link type and access rights

* tests: checkout: test avoiding unnecessary copying

* efiop review refactor

* remote: local: refactor _needs_checkout NOTE

* remote: local: remove bool cast for `protected`, wrap linking in try

* test: checkout: test relinking upon existing cache

* stage: remove obsolete unprotecing from Stage.create

* test: checkout: refactor setting link type

* test: checkout: parametrize relinking test

* remote: local: don't remove cache test file on cache_copy test

* remote: base: remove force from _needs_checkout arguments

* Update dvc/remote/local/__init__.py

Co-Authored-By: Alexander Schepanovski <[email protected]>

* suor review refactor

* remote: base: re-add note why do we unprotect on _checkout_file

* test: checkout: tests for relinking and resetting target protection

* remove checkout behavior change

* remove unused imports

* remote: base: reuse __checkout_file in __checkout_dir

* initial dir

* checkout: test: should relink on repeated add

* remote: cleanup commented lines

* remote: base/local: add save_link flag to _checkout_file

* remote: local: add allow_copy flag to unprotect method

* Update dvc/remote/local/__init__.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* remote: local: remove unnecessary removal warning on _checkout_file

* remote: local: proper cache link type detection

* test: checkout: move repeated add tests

* Update dvc/remote/local/__init__.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* remote: base: remove _needs_checkout

* Update dvc/remote/base.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* remote: base: remove wrappers from _checkout_file
2019-09-06 17:20:32 +03:00
pawel 651d925578 stage: don't unprotect outs on stage create 2019-09-04 12:54:32 +02:00
Alexander Schepanovski 8984e9fed4 dvc: use wdir == "." on add
Fixes #2404
2019-08-24 10:23:41 +07:00
Jorge Orpinel 91bd0faa87 dvc: feedback to #2112 round 1 2019-06-26 16:25:56 -05:00
Jorge Orpinel f004a84e7c tests: fix failing test_add (due to extra 's' in word)
To close #2112
2019-06-26 14:05:18 -05:00
Jorge Orpinel fba7d85cbd dvc: Use "DVC-file" term as much as possible
Keep "stage file" in the context of pipelines

No issue
2019-06-26 14:05:18 -05:00
Ivan Shcheklein 478d6aec2f test: ls/dir depends on platform, dir is not supported by Mac 2019-06-23 14:09:22 -07:00
Ruslan Kuprieiev 918ef646dc appveyor: test py2 and py3
Related to #2166

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-23 13:33:47 +03:00
prd 0615ff8730 pathinfo: extract relative path method to standalone method 2019-06-18 12:08:19 +02:00
Andrew Grigorev 5fce9a01e4 tests: fix for pytest>=4.6.0 caplog 2019-06-02 15:41:21 +03:00
pawel fae55c2ba2 remote: local: roll back cache dir metadata storing 2019-05-28 13:39:48 -07:00
pawel 01c2f6c174 tests: rename dvc repo fixture 2019-05-21 15:04:43 -07:00
prd 1cb4e5233e path: local: str return full path on value error 2019-05-14 16:56:24 -07:00
Ruslan Kuprieiev 1d5d608c21 test: make sure checksums are not recomputed on checkout
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-13 21:07:12 +03:00
Mr. Outis d3502ebfe3 add: warn when recursively adding large files
Close #1758
2019-04-30 12:33:28 -07:00
Ruslan Kuprieiev 9c77f2818c stage: commit cached stage to avoid data duplication
Fixes #1655

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 21:14:08 +03:00
Ruslan Kuprieiev 00cfb04fea remote: generalize file/dir handling
Fixes #1654
Fixes #1655

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 05:09:48 +03:00
Alexander Schepanovski 0c867399bb dvc: retain formatting and comments in .dvc files (#1885)
* test: remove never used copy of load_stage_file()

* dvc: save comments and custom formatting in stage files

* Add a couple of TODOs

* test: test repro doesn't clear comments and formatting in .dvc

As part of this I started writing pytest-like tests and added commonly
used fixtures.

* test: simplify and prettify dvc formatting retained test

Also test that line comments after md5 fields are retained.

* test: fix tests

* dvc: fix stage and metrics opens not closed

* dvc: always use utf-8 in stage files

It used to encode in system default encoding. Not sure if PyYAML was
handling this silently or we were just lucky all the way.

* test: unit test apply_diff()

* test: remove TODOs and ASK

* dvc: polish dvc-formatting PR

* dvc: move stage load/dump utils to a new stage utils file

* dvc: clean up apply_diff()

* test: test stage md5 ignores comments

* test: remove pytests starting marker
2019-04-20 01:12:53 +03:00
Ruslan Kuprieiev 6f11f1f51f test: move func tests to tests/func
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-16 20:19:22 +03:00