Commit Graph
3378 Commits
Author SHA1 Message Date
pared a3857fe747 install: add pre-push hook, refactor test 2019-04-30 20:51:09 -07:00
Ruslan Kuprieiev 40ace1702b Merge pull request #1950 from mroutis/fix-1839
status: optimization for already downloaded files
2019-05-01 04:33:10 +03:00
Mr. Outis 29f58bda9b status: optimization for already downloaded files
Fix #1839
2019-04-30 17:09:04 -07:00
Ruslan Kuprieiev 69a6993233 Merge pull request #1935 from mroutis/fix-1758
add: warn when recursively adding large files
2019-05-01 02:08:44 +03:00
Mr. Outis d3502ebfe3 add: warn when recursively adding large files
Close #1758
2019-04-30 12:33:28 -07:00
Ruslan Kuprieiev 622a089743 Merge pull request #1942 from pared/1941
repo: get default logger level from logger
2019-04-30 21:20:23 +03:00
Ruslan Kuprieiev e99fc2d8e8 Merge pull request #1946 from Suor/pbar-large-dir
dvc: add progress bar when hashing a large dir
2019-04-30 19:15:53 +03:00
Alexander Schepanovski a6823e0113 dvc: add progress bar when hashing a large dir
Added an easier way to show progress bar over for loops.
2019-04-30 20:26:02 +07:00
Ruslan Kuprieiev 8c02cefb9e Merge pull request #1947 from Suor/tests-log-level
test: fix tests being dependent on log level and abandon default DEBUG level
2019-04-30 15:22:35 +03:00
Alexander Schepanovski af4af45458 test: dedup some metrics tests
Closes #1928.
2019-04-30 18:01:54 +07:00
Alexander Schepanovski cdf318fff7 test: abandon default DEBUG log level in tests
Now simple `py.test` command won't use DEBUG log level and `py.test ...
--log-level=LEVEL` will do what it's supposed to.

Staging continues to use DEBUG log level.

This is part of #1888 resolution.
2019-04-30 17:43:39 +07:00
pared a1b12db00f repo: get default logger level from logger 2019-04-29 17:41:38 -07:00
pared 59b4090e64 metrics: show: do not throw on no metric present on branch 2019-04-29 15:16:05 -07:00
Ruslan Kuprieiev 435095e35f Merge pull request #1914 from pared/1890
repo: different cache link types info on long lasting operations
2019-04-29 19:14:27 +03:00
ag613915cao 0dabf7bbaa Replace yield_fixture()
Because yield_fixture() is deprecated since pytest 3.0 [1]. It is
not recommend to use the decorator instead of fixture().

[1] https://docs.pytest.org/en/latest/yieldfixture.html
2019-04-29 15:17:10 +07:00
Brett Barkley 4260a6246d dvc metrics show: implement jsonpath-ng
updated setup.py
updated test_metrics.py::assertMetricsHaveRelativePaths

Fixes #1919
2019-04-28 09:01:28 -04:00
Brett Barkley beef1e32b0 dvc metrics show: implement jsonpath-ng
Fixes #1919
2019-04-28 08:28:36 -04:00
pared f496ba4f03 remote: local: different cache link types info on long lasting linking 2019-04-26 10:17:16 -07:00
ag613915cao fd385c913b This commit intends to:
- Keep current SSH remote connection test on real system
- Faked and Mocked SSH server which running on both Linux and Windows
- Add corresponding tests

Fixed #1704
2019-04-26 18:07:16 +07:00
Alexander Schepanovski 7478e7d4b0 test: clean up test dirs (#1904)
* test: clean up test dirs

Closes #1770. Partially does #1888.

* dvc: close anything opened

* test: finalize git repo explicitly to remove test dir later

This was previously an issue on Windows.
2019-04-25 14:55:39 +03:00
Mr. Outis c0104e2e61 repro: implement downstream option (#1926)
* repro: implement downstream option

Close #1555

* Update scripts/completion/dvc.zsh

Co-Authored-By: mroutis <[email protected]>

* Update dvc/command/repro.py

Co-Authored-By: mroutis <[email protected]>
2019-04-25 04:50:44 +03:00
Mr. Outis d42c8fbd14 repro: persisting outputs invalidates build cache
Fix #1852
2019-04-24 16:45:03 -07:00
Ruslan Kuprieiev 9c77f2818c stage: commit cached stage to avoid data duplication
Fixes #1655

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 21:14:08 +03:00
Ruslan Kuprieiev 37ee55b1f8 test: test ssh
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 05:09:48 +03:00
Ruslan Kuprieiev b97b99cf15 test: func: use pytest for outputs/dependencies
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 05:09:48 +03:00
Ruslan Kuprieiev 44b95f23a3 remote: ssh: support directories
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 05:09:48 +03:00
Ruslan Kuprieiev 00cfb04fea remote: generalize file/dir handling
Fixes #1654
Fixes #1655

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 05:09:48 +03:00
Ruslan Kuprieiev 41b67c3730 move: move directory into an existing directory
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-24 04:56:33 +03:00
Alexander Schepanovski 277b9e566c test: use caplog.at_level() to restore log level after test 2019-04-24 02:45:02 +03:00
pared dd8598e53e tree: walk: operate on tree absolute root 2019-04-23 02:38:23 +03:00
Alexander Schepanovski 8ef52f92ee dvc: add meta key to stages to store user data 2019-04-22 16:26:57 +03:00
ag613915cao 47fb2d3a3b Add test for ssh connection
This commit intends to mock SSH server that not depends on
OS which the test running on for testing purpose.
2019-04-21 04:22:59 +03:00
Alexander Schepanovski 0c867399bb dvc: retain formatting and comments in .dvc files (#1885)
* test: remove never used copy of load_stage_file()

* dvc: save comments and custom formatting in stage files

* Add a couple of TODOs

* test: test repro doesn't clear comments and formatting in .dvc

As part of this I started writing pytest-like tests and added commonly
used fixtures.

* test: simplify and prettify dvc formatting retained test

Also test that line comments after md5 fields are retained.

* test: fix tests

* dvc: fix stage and metrics opens not closed

* dvc: always use utf-8 in stage files

It used to encode in system default encoding. Not sure if PyYAML was
handling this silently or we were just lucky all the way.

* test: unit test apply_diff()

* test: remove TODOs and ASK

* dvc: polish dvc-formatting PR

* dvc: move stage load/dump utils to a new stage utils file

* dvc: clean up apply_diff()

* test: test stage md5 ignores comments

* test: remove pytests starting marker
2019-04-20 01:12:53 +03:00
vyloy 60c89adeb5 import: use os.path.join instead of hardcode path in the test 2019-04-18 09:25:38 +08:00
vyloy 01a28f2de9 import: improve some details about the '--file' option. 2019-04-18 08:56:14 +08:00
vyloy b07c386d02 import: support the '--file' option 2019-04-18 08:56:14 +08:00
Ruslan Kuprieiev 9618b32fe8 pkg: comply with design of other dvc commands
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-17 23:24:27 +03:00
Ruslan Kuprieiev 6f11f1f51f test: move func tests to tests/func
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-16 20:19:22 +03:00
Yao Adzaku 6c730dd5cb pipeline: print locked DVC stages (#1882)
* pipeline: add option to display locked DVC stages.

The '--locked' option xplicitly displays only locked DVC stages.

Fixes #1864

* pipeline: support short option '-l' for displaying locked stages

* pipeline: minor cosmetic tweak

* pipeline: reformat code with black

* pipeline: test printing of locked stages

* pipeline: use caplog to ensure command outputs everything as expected

* pipeline: cleanup test

* pipeline: more strict comparison of test output
2019-04-16 16:46:50 +03:00
Paulo Olveira 75b4d0e483 s3: fix for corrupted cache on multipart uploads (#1867)
* s3: fixed wrong etag when copying multipart objects

The etag of multipart objects depends of the number of parts, when copying to the cache we should do so in the same number of parts that the original object was moved/uploaded in.

Fixes part of #1410

* s3: added check on copy for equal etag

* s3: added specific exception for ETag mismatch

* s3: use multipart copy to preserve etags

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* test: add tests for etag preservation on s3

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* test: requirements: use dev version moto

Specifically because of this https://github.com/spulec/moto/issues/1941

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* test: requirements: install dev moto without -e

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* test: stop using moto

Turned out to be quite buggy.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-15 21:03:22 +03:00
Ruslan Kuprieiev a74e4226a1 Merge pull request #1820 from pared/1499
ignore: introduce .dvcignore
2019-04-15 17:43:21 +03:00
pared c467ccebcb ignore: introduce .dvcignore, apply when searching for stage files 2019-04-15 14:31:34 +02:00
pared 074b6eaa9e test: move corrupted hardlink cache test test_repro -> test_run 2019-04-15 12:55:50 +02:00
pared 9049475412 test_repro: make cache use hardlink 2019-04-15 11:27:44 +02:00
pared e505c38f20 cache: make reflink, copy default links 2019-04-15 11:27:44 +02:00
Mr. Outis 1cac81430e tests: be paranoic with caplog 2019-04-11 14:09:02 -05:00
Ruslan Kuprieiev 4953110feb Merge pull request #1855 from mroutis/fix-1614
config: allow cross-referenced remotes
2019-04-09 15:19:41 +03:00
Ruslan Kuprieiev 56b52cc9bf checkout/remove: properly handle broken symlinks
Fixes #1856

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-09 14:09:54 +03:00
Mr. Outis d355b1e66e config: allow cross-referenced remotes
Close #1614
2019-04-08 22:10:03 -05:00
Mr. Outis 727bfd955e version: privatize version.py methods and vars 2019-04-07 16:48:34 -05:00