5845 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 749fec381f dvc: bump to 1.7.5 1.7.5 2020-09-24 16:26:04 +03:00
Ruslan Kuprieiev 21de21fe37 gha: adjust the test workflow (#4615)
* gha: move lint check to tests workflow

* gha: tests: don't fail fast

* scripts: remove redundant check
2020-09-24 16:13:03 +03:00
Ruslan Kuprieiev 0756af5012 setup: remove wheel from install_requires (#4614)
Kudos to @skshetry for catching this one in https://github.com/iterative/dvc/pull/4606
2020-09-24 15:39:54 +03:00
Ruslan Kuprieiev 2457633990 gha: build packages (#4607) 2020-09-24 14:48:04 +03:00
David Roberts 46bc89319a install: Potential fix for (non-fatal?) setup errors on Ubuntu 20.04. (#4606)
Wheels fail to build on initial install. Install `wheel` to fix this.

Original error message:
Building wheel for pyyaml (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/david/Documents/job-hunting/dvc/.env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nhmea2xe/pyyaml/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nhmea2xe/pyyaml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-v2ewd51w
       cwd: /tmp/pip-install-nhmea2xe/pyyaml/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for pyyaml
  Running setup.py clean for pyyaml

Potential fix applied from:
	https://stackoverflow.com/questions/34819221/why-is-python-setup-py-saying-invalid-command-bdist-wheel-on-travis-ci

Fixes #4605.
2020-09-24 02:02:15 +03:00
Ruslan Kuprieiev 4a82974da4 run-cache: respect cache.shared config option (#4596)
Fixes #4595
2020-09-24 01:00:53 +03:00
Ruslan Kuprieiev 688da04d4a tests: cache test directories (#4601)
* tests: ignore more hdfs tests

* tests: remove unused mark

* gha: don't limit the number of test workers

* cache created test dirs
2020-09-23 23:22:27 +03:00
Paweł Redzyński e1b34442aa plot: accept any viable target in dvc repo (#4602) 2020-09-23 17:02:12 +03:00
Ruslan Kuprieiev 38dd1c9fe7 tests: allow failure for hdfs tests (#4600) 2020-09-23 00:53:32 +03:00
Paweł Redzyński b31bd1cb30 github actions tests: initial (#4469)
* github actions tests: initial

* build: tests: add codecov action
2020-09-22 23:05:06 +03:00
Paweł Redzyński 2b3c30cb0c metrics: accept any viable target in DvcRepo (#4590) 2020-09-22 22:53:55 +03:00
ebaffd728e Restyle Do not require account-level permissions for Azure tree (#4583)
* Do not require account-level permissions for Azure tree

* Only allow the existence check to pass without error if it failed due to authorization issues

* Update dvc/tree/azure.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/tree/azure.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Restyled by isort

* Update azure.py

Co-authored-by: Matt Hollingsworth <[email protected]>
Co-authored-by: Matt Hollingsworth <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Restyled.io <[email protected]>
2020-09-22 00:07:56 +03:00
Ruslan Kuprieiev ece7d6ba67 dvc: bump to 1.7.4 1.7.4 2020-09-21 23:50:53 +03:00
Ruslan Kuprieiev cf5d461f4d api: enter state when trying to get a hash (#4584)
`get_hash` is an internal api, so conventions should be respected.

Fixes #4575
2020-09-21 23:37:51 +03:00
Ruslan Kuprieiev ff3b686058 dvc: bump to 1.7.3 1.7.3 2020-09-21 20:24:02 +03:00
Ruslan Kuprieiev 0b9cd4c93c gdrive: don't use hacks for empty files (#4588)
This bug is fixed in PyDrive2 1.6.2.
2020-09-21 20:23:19 +03:00
Robert LowryandRuslan Kuprieiev 15488ebefd scm: add config option to stage files automatically (#4543)
* scm: add "autostage" config option to stage files automatically

Fixes #4330

* Update tests/unit/scm/test_scm.py

* Update tests/unit/scm/test_scm.py

* scm_context, git: call reset_tracked_files from scm_context, remove output message

* Update dvc/repo/scm_context.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-09-21 11:59:25 +03:00
Peter Rowlands (변기호) 6a3c516b71 experiments: support committing to existing experiment branches (#4573)
* experiments: stash optional branch to extend

* experiments: support commit to existing experiment branch
2020-09-19 21:49:40 +03:00
Ruslan Kuprieiev de768375ed dvc: bump to 1.7.2 1.7.2 2020-09-15 12:14:49 +03:00
Ruslan Kuprieiev 2f88fa88c4 pydrive2: require six >= 1.13.0
https://github.com/iterative/PyDrive2/pull/56
2020-09-15 11:31:52 +03:00
01abf7c19c Fix error during external outputs checkout (#4570)
* Fix error during external outputs checkout

User in discord complained that external outputs' checkout
was failing. Looks like, during refactor of `HashInfo`,
this was forgotten, and as no tests existed before, we had
a regression from 1.6.5 onwards.

* fix tests

* Restyled by black (#4571)

Co-authored-by: Restyled.io <[email protected]>

Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <[email protected]>
2020-09-15 11:07:09 +03:00
Ruslan Kuprieiev 4ebacf1e0a pydrive2: require >=1.6.1
https://github.com/iterative/PyDrive2/pull/56
2020-09-15 11:06:36 +03:00
karajan1001 5e69e29b2f check-ignore show sub-repo patterns (#4564)
* `check-ignore` show sub-repo patterns

fix #4555
1. Sub-repo pattern now would be considered in `check-ignore` command.
2. New outputs for sub-repo patterns in a `detail`
3. Related tests.

* message change
2020-09-15 06:59:41 +03:00
Ruslan Kuprieiev bc5611968f dvc: bump to 1.7.1 1.7.1 2020-09-14 12:16:14 +03:00
Ruslan Kuprieiev 8c19492ba6 tree: validate resolved remote:// sections (#4569)
* tree: validate resolved remote:// sections

* dvc: use checksum_jobs instead of hash_jobs

This is a regression introduced during one of the refactorings.
2020-09-14 12:15:38 +03:00
Ruslan Kuprieiev 61be19b9b5 Update version.py 1.7.0 2020-09-14 07:31:49 +03:00
Ruslan Kuprieiev 424396b068 tree: resolve deep remote:// urls (#4568)
Fixes #4556
2020-09-14 05:49:54 +03:00
karajan1001andRuslan Kuprieiev 8b16af965c Print dvc version info for debugging (#4536)
* Print dvc version info for debugging

fix #4095
1. seperate info from dvc version.
2. print dvc version info when some unexpected error occured.

* Psutil NoneType error

* Update tests/func/test_version.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Move `package` outside dvc/utils/pkg.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-09-13 04:08:20 +03:00
Ruslan Kuprieiev cf7e534e2d tree: repo: handle dirty state (#4554) 2020-09-11 17:25:33 +03:00
Saugat Pachhai 2931d30ca5 dvc: change dvcfile.dump to do synchronized dump by default (#4544)
The API of `Dvcfile.dump` has been changed to
`dump(stage, update_pipeline=True, update_lock=True)`,
which will make it to always dump by default to both files,
and provide flags to suppress one or the other.

Fixes #4235
2020-09-10 13:12:00 +05:45
Robert F. DickersonandRuslan Kuprieiev 7fa90e425d Allow config option for HTTP method for uploads (Artifactory) (#4553)
* allow for a config option for http method

* add tests

* fix tests with method

* Update dvc/tree/http.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-09-10 05:11:41 +03:00
Victor 54a48e3256 Feature/http ssl verify configuration (#4547)
* add test for http ssl_verify configuration

* add ssl_verify flag to HTTPTree
2020-09-09 19:27:43 +03:00
Ruslan Kuprieiev 1ca2dc19c2 tree: open: handle dirty state in RepoTree (#4548)
We used to use that logic in DvcTree back before we had proper trees.
Now we have a RepoTree, that combines git/workspace tree with the
DvcTree and dirty state clearly belongs to the git/workspace and not
the dvc itself, as dvc state only depends on dvc-files and lock-files.

Prerequisite for #4523
2020-09-09 18:44:11 +03:00
Alexander Schepanovski 67ddb7b41f Make repo lock reentrable and provide a context manager (#4416) 2020-09-08 17:40:34 +03:00
Ruslan Kuprieiev 61bf1194c3 tests: add gs etag
Kudos to @skshetry
2020-09-08 09:29:56 +03:00
Ruslan Kuprieiev ccdcb1f4c1 codecov: remove old token
Kudos to @skshetry
2020-09-08 09:24:38 +03:00
Ruslan Kuprieiev 226322d312 gdrive: add workaround for #4507
Temporary workaround. Proper solution will be introduced in https://github.com/iterative/PyDrive2/pull/48
2020-09-08 05:27:20 +03:00
Alexander Schepanovski e1f43965a2 Skip no data plots while rendering (#4541) 2020-09-07 23:25:43 +03:00
Ruslan Kuprieiev 18e80b25ac codecov: add token 2020-09-07 23:16:30 +03:00
Saugat Pachhai 131099750e Disable cprofile on exit of the contextmanager profile (#4542)
Does not change anything, it's good to disable it just after yield to not show other stats by accident.
2020-09-07 14:50:36 +05:45
Peter Rowlands (변기호) d1cc143ef2 diff: add state for files missing from both workspace and cache (#4530)
* diff: support missing cache status for workspace diff

* diff: add --hide-missing option to hide cache missing status

* update command tests

* update diff func tests

* only show missing cache status when diffing local workspace

* suppress all output if no changes and --hide-missing

* use hide_missing instead of show_missing

* use is_dvc
2020-09-07 04:44:04 +03:00
Ruslan Kuprieiev 93a298343f run: consolidate run-cache logic (#4535)
Existing lock files are just an edge case of run-cache.

Prerequisite for #4223
2020-09-06 17:46:25 +03:00
Ruslan Kuprieiev 636a019289 dvc: compute dir hash without external cache (#4528)
Fixes #4144
2020-09-05 16:31:34 +03:00
Victor 79e8e4edaf dvc: override replace in HTTPURLInfo to include extra_parts (#4517)
* dvc: override replace in HTTPURLInfo to include extra_parts

The objective is to passtrough any parameter configured in http remotes.

fixes #4508

* add unit test for #4508

* Use HTTPURLInfo directly instead of decorator
2020-09-03 15:47:18 +03:00
Peter Rowlands (변기호) 1f495d66dd diff: use RepoTree to compare directory contents (#4518)
* diff: use RepoTree to get granular results for dir outs

* update tests for dir file granularity

* diff: prefer repo_tree.get_hash when possible

* DvcTree: handle extra files in dirty dvc out dirs
2020-09-03 15:12:57 +03:00
Saugat Pachhai 4564e8be30 support subrepos for get/list/api/import/ls (#4465)
* support subrepos

This adds support for get/list/api/import/ls on a subrepo.

Also fixes #3180, by adding a granular url support for `dvc.api.get_url`
And, of course it fixes #3369.

* Add tests

* Fix tests, started to get lock_error

* Fix tests for windows

* try to fix mac builds

* Fix comments, readd removed comments

* Fix test

* address review suggestions:

* use `spy` instead of `wraps`
* remove dvc fixture in a test
* set `cache_types` on Git repo
2020-09-03 13:47:14 +03:00
d460d18942 version: improve link type detection fall back (#4479)
* version: improve link type detection fall back

If failed to determine cache type print link to related web page.

Fixes #2788

* Update dvc/command/version.py

* fix formatting

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-09-03 12:03:56 +03:00
Ruslan Kuprieiev 4901765cd4 get/import: fix bug when working with mixed dirs (#4516)
Fixes #4510
2020-09-02 05:53:09 +03:00
Peter Rowlands (변기호) 24be96fce9 experiments: suppress repro --params option by default (#4515) 2020-09-02 10:17:01 +09:00
Ruslan Kuprieiev 981dd401fa cache: use get_file_hash instead of get_hash (#4514)
This is what we've been doing before. No need to try to pull it up from
the state db, as this is a clearly newly generated file that will get
deleted.
2020-09-02 03:37:17 +03:00