Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-10 23:10:42 +01:00
Code Issues Packages Projects Releases Wiki Activity
9,307 Commits 7 Branches 579 Tags
c514b0ea31339eada4da584063841cb303e0a67c
Commit Graph
9307 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Peter Rowlands (변기호) 256cb06921 move: validate new stage before move (#5953) 
* move: validate new stage before move

* verify that nothing was moved in tests
 2021-05-05 13:09:55 +09:00
Saugat Pachhai ff56379f8a tests: rerun gdrive and gs on failures (#5959) 
we just need to mark them with needs_internet marker.
Unfortunately, it's not possible to do that automatically
with the fixtures as fixtures are per-session and we need
something on more higher level (which the marker is not passed
through in the pytest).
 2021-05-04 17:59:50 +03:00
Saugat Pachhai 4733757bb1 use realpath for is_in_repo calculation (#5954) 
This is because, the repo root is used from a realpath, and the stage.wdir is abspath.
We might need to use same kinds of path, but this patch seems to be the easiest/fastest
way to fix the flaky tests (if all the tests agree).
 2021-05-04 17:59:09 +03:00
dependabot[bot] 3e9fd100ea build(deps): bump moto from 2.0.5 to 2.0.6 (#5955) 
Bumps [moto](https://github.com/spulec/moto) from 2.0.5 to 2.0.6.
- [Release notes](https://github.com/spulec/moto/releases)
- [Changelog](https://github.com/spulec/moto/blob/master/CHANGELOG.md)
- [Commits](https://github.com/spulec/moto/compare/2.0.5...2.0.6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-04 17:45:30 +03:00
dependabot[bot] c1a49b440d build(deps): bump pygments from 2.8.1 to 2.9.0 (#5956) 
Bumps [pygments](https://github.com/pygments/pygments) from 2.8.1 to 2.9.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.8.1...2.9.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-04 17:45:16 +03:00
dependabot[bot] c2734a2cd7 build(deps): bump codecov/codecov-action from v1.4.1 to v1.5.0 (#5957) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.4.1 to v1.5.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.1...a1ed4b322b4b38cb846afb5a0ebfa17086917d27)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-04 17:45:01 +03:00
Saugat Pachhai 71ab15771a exp show: decoupling tree and styling from row collection (#5903) 
* exp show: try to split presentation logic

Trying to remove tree and the bold/row-styling part even more

* fix naming of the experiment types
 2021-05-04 18:15:55 +05:45
Peter Rowlands (변기호) 53414a0618 dvcfile/stage: check if dvcfile would be gitignored on stage creation (#5907) 
* dvcfile: add check for gitignored when validating filenames

* update tests

* update exception using in tests

* review fixes

- clarify method name
- use FileIsGitIgnored
 2021-05-04 15:14:55 +09:00
Saugat Pachhai 708587a159 tests: mark test_stash_pop as xfail on linux (#5952) 
I tried increasing max_run to very high number: 1000. Even in those
cases, it failed 947 times. So I don't really see a reason to mark
it as flaky. Anyway, we are treating it as xfail by keeping it failing
in the CI, so this PR does not change anything for us. :)
 2021-05-04 11:22:57 +05:45
Ruslan Kuprieiev 3db25ea67d dvc: bump to 2.1.0 2.1.0 2021-05-03 16:29:49 +03:00
Saugat Pachhai d6afd26bea metrics diff/show: rollback to rounding fractional digits with precision (#5904) 2021-05-02 16:56:00 +03:00
Saugat Pachhai dba0aaeab7 accept dict for header styling (#5905) 
I found that it's harder to style a header with the list using index,
they are to be applied by header name. Otherwise, we might end up using
a different header for coloring, or worse throw some error because the
header name and style don't match.
Row does have it's property of index, so it makes sense there.
 2021-04-30 18:43:38 +05:45
Paweł Redzyński d509d77193 list: add --show-json option (#5888) 
* list: add --show-json option

* list: add test for --show-json
 2021-04-30 11:40:44 +02:00
Saugat Pachhai 102817e53a simplify experiments row collection logic (#5901) 2021-04-30 15:11:57 +05:45
Peter Rowlands (변기호) f771b1d5b0 exp run: error out when trying to resume from intermediate checkpoint w/no modifications (#5829) 
* apply: allow applying exp with multiple possible branches

* default to new checkpoint branch after applying ambiguous commit

* output shas in addition to exp names by default

* update show tests

* git: allow force checkout in detach_head context

* exp run: error when trying to resume from a non-head checkpoint without
modifications

* add test for apply/resume from intermediate checkpoint

* list some possible heads when multiple matches
 2021-04-30 15:12:44 +09:00
Peter Rowlands (변기호) 753803c177 pygit: force release of odb contexts after checkout/merge (#5849) 2021-04-30 10:18:37 +09:00
Ruslan Kuprieiev c9a748421a dependabot: add "maintenance" label (#5898) 2021-04-29 16:59:53 +03:00
Ruslan Kuprieiev cab4126ed4 output: relative paths outside of repo (#5893) 
* output: relative paths outside of repo

Fixes #5870

* scm: enhance FileNotInRepo message
 2021-04-28 16:13:01 +03:00
Batuhan Taskaya 2172add905 fs: introduce tree.find() (#5879) 2021-04-28 13:39:19 +03:00
Saugat Pachhai b2f71d459b Revert "build(deps): bump pylint-pytest from 1.0.3 to 1.1.2 (#5850)" (#5890) 
This reverts commit 8930be06e8.
 2021-04-27 13:32:39 +00:00
Saugat Pachhai 111c282808 exp show: separate ui representation of exp names and identifiers (#5885) 2021-04-27 18:37:35 +05:45
Saugat Pachhai 42cf394fde diff: use ui.table for --show-md (#5887) 2021-04-27 17:10:08 +05:45
dependabot[bot] 9c94b6dcaa build(deps): bump google-cloud-storage from 1.37.1 to 1.38.0 (#5883) 
Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 1.37.1 to 1.38.0.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/master/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-storage/compare/v1.37.1...v1.38.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-27 16:03:08 +05:45
dependabot[bot] a05be55e7e build(deps): bump pylint from 2.8.1 to 2.8.2 (#5884) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.8.1 to 2.8.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.8.1...v2.8.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-27 16:02:45 +05:45
Saugat Pachhai 8fd2eba9a2 refactor exp show: use new abstractions (#5880) 2021-04-27 13:14:56 +05:45
Saugat Pachhai f38d163d5c refactor diff and show to use new table ui and new data structure (#5872) 
* use new table data structure for diff and show

* fix params diff showing changes

* metrics show: use default precision

* skip precision format for exp diff params

* exp diff: separate tables in --quiet mode

* change precision to signify significants digits

changed f specifier to g, so that we show significant digits
instead of fractional digits. This does changes how precision
worked for metrics/params diff/show.
 2021-04-27 11:05:52 +05:45
Saugat Pachhai 0860e419e4 ui: tabular data (#5852) 
* introduce tabular data structure

* ignore ellipsis
 2021-04-26 14:54:23 +03:00
Saugat Pachhai 2978a5d7e9 dagascii: prune unused pager (#5876) 2021-04-26 14:52:29 +03:00
dependabot[bot] af6921e9ac build(deps): bump pre-commit/action from v2.0.2 to v2.0.3 (#5856) 
Bumps [pre-commit/action](https://github.com/pre-commit/action) from v2.0.2 to v2.0.3.
- [Release notes](https://github.com/pre-commit/action/releases)
- [Commits](https://github.com/pre-commit/action/compare/v2.0.2...9b88afc9cd57fd75b655d5c71bd38146d07135fe)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 14:08:40 +03:00
dependabot[bot] 8930be06e8 build(deps): bump pylint-pytest from 1.0.3 to 1.1.2 (#5850) 
Bumps [pylint-pytest](https://github.com/reverbc/pylint-pytest) from 1.0.3 to 1.1.2.
- [Release notes](https://github.com/reverbc/pylint-pytest/releases)
- [Changelog](https://github.com/reverbc/pylint-pytest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/reverbc/pylint-pytest/compare/pylint-pytest-1.0.3...pylint-pytest-1.1.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 14:05:33 +03:00
dependabot[bot] 9eba910023 build(deps): bump pytest-mock from 3.5.1 to 3.6.0 (#5874) 
Bumps [pytest-mock](https://github.com/pytest-dev/pytest-mock) from 3.5.1 to 3.6.0.
- [Release notes](https://github.com/pytest-dev/pytest-mock/releases)
- [Changelog](https://github.com/pytest-dev/pytest-mock/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest-mock/compare/v3.5.1...v3.6.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 13:21:58 +03:00
dependabot[bot] 0728777494 build(deps): bump pylint from 2.7.4 to 2.8.1 (#5873) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.7.4 to 2.8.1.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.7.4...pylint-2.8.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 13:21:51 +03:00
Jan 54c0675fc6 Add periods (#5867) 2021-04-23 13:58:12 +03:00
Peter Rowlands (변기호) e0d90cdda1 exp run: dvc commit DVC-tracked data deps when stashing an experiment (#5859) 
* stage,commit: add support for only committing data stages from
experiments

* exp run: `dvc commit` data deps when stashing exp changes

* add test for modified DVC-tracked data when generating experiment

* test with and without params change

* handle run without target
 2021-04-23 11:59:15 +09:00
Batuhan Taskaya 17d7b06f7b output: introduce BaseOutput.transfer (#5861) 
* output: introduce BaseOutput.transfer

* add: use BaseOutput.transfer for to-cache
 2021-04-22 15:01:44 +03:00
dependabot[bot] 8ac26b58ee build(deps): bump codecov/codecov-action from v1.4.0 to v1.4.1 (#5857) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.4.0 to v1.4.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.0...967e2b38a85a62bd61be5529ada27ebc109948c2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-22 13:50:46 +03:00
Batuhan Taskaya a42e363217 objects: don't recompute hash on odb.add() if verify is False (#5860) 2021-04-22 13:50:24 +03:00
Paweł Redzyński d87ace0968 html: enable providing custom html page templates (#5694) 
* html: enable providing custom html page templates

* plots: add html template configuration option
 2021-04-20 11:02:14 +02:00
Saugat Pachhai d275e6b7b5 cmd: stage list: use ui.table (#5847) 2021-04-20 12:04:26 +05:45
dependabot[bot] 71ee20412b build(deps): bump pyinstaller from 4.2 to 4.3 (#5843) 
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 4.2 to 4.3.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/develop/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v4.2...v4.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-20 07:13:26 +03:00
Batuhan TaskayaandRuslan Kuprieiev 68897aa0de update: only update the missing hashes with --to-remote (#5773) 
* objects: merge transfer() with stage()

* objects: drop streamfile

* objects: preserve the signature of get_file_hash

* objects: override hash_jobs if jobs= passed

* objects: support no progress bar directive

* tests: spy on _get_file_hash

* objects: update=... directive

* update: only transfer missing caches with --to-remote

* tests: remove local_remote from the primary test (due to it moves)

* objects: add move=True/False option

* objects: implement _get_file_obj on top of get_file_hash

* objects: use threading on save()

* objects: pass jobs= on to-cache as well

* objects: use in place tqdm

* objects: check for FileExistsError

* Update dvc/objects/__init__.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2021-04-19 18:09:35 +03:00
Batuhan Taskaya a8b7fda5a2 metrics: pass --precision to _show_metrics (#5839) 
* metrics: pass --precision to _show_metrics

* tests: return empty string
 2021-04-19 17:25:45 +03:00
Ruslan Kuprieiev 3472cb1a6a tests: don't set sensitive env secrets for all steps (#5823) 
Due to https://about.codecov.io/security-update/ , there is a potential that we've leaked these. This PR takes a more granular approach when setting secret env vars so that this doesn't happen in the future.

We'll change these secrets after this PR is merged.
 2021-04-19 16:39:37 +03:00
Peter Rowlands (변기호) 60ea24410e exp: use exp baseline for DVC commands that compare to HEAD during exp run (#5818) 
* exp: add util method for getting baseline HEAD during run

* exp: use baseline for dvc commands which compare to HEAD during exp run

* add test for new util method

* update plots diff test mock
 2021-04-19 14:42:58 +09:00
Naveen Sinha daf07451f8 Add support for kerberos token based auth (#5809) 
* Add support for kerberos token based auth

* Added kerb_ticket as attribute to HDFSFileSystem

* Lint fix for config schema

* Updated black style changes

* Removed staticmethod decorator from hdfs
 2021-04-17 10:02:58 +03:00
dependabot[bot] 48890c22b7 build(deps): bump codecov/codecov-action from v1.3.2 to v1.4.0 (#5838) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.3.2 to v1.4.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.3.2...0e28ff86a50029a44d10df6ed4c308711925a6a8)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-17 11:20:46 +05:45
Jorge Orpinel 5e92bef733 dvc: add -A flag to * --all-commits (#5783) 2021-04-16 14:19:44 +03:00
Bhagawan Subedi ca7a02d522 erepo: replace copy_tree from disutils with copytree of shutil (#5827) 
Fixes: #5817
 2021-04-16 15:02:16 +05:45
Peter Rowlands (변기호) 29b9c5370c exp run: disallow --rev for workspace runs (#5831) 2021-04-16 18:02:13 +09:00
Peter Rowlands (변기호) b08a3a8caa exp run: suppress "finished run" output on --dry (#5830) 2021-04-16 16:36:39 +09:00