Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
7,819 Commits 7 Branches 579 Tags
206e00ccbe7882e5a49a0c969d6f3890d2414359
Commit Graph
7819 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ivan Shcheklein 206e00ccbe guard rwlock file to prevent not locked repo edits 2022-08-11 15:54:42 +03:00
Peter Rowlands f13443477d exp run: account for repo root when staging git-deps 2022-08-11 13:48:28 +09:00
Dave Berenbaum 9837747eef exp run: ui tweak (#8124) 2022-08-11 09:14:43 +05:45
dependabot[bot] cf5dfc517a build(deps-dev): Bump filelock from 3.7.1 to 3.8.0 (#8125) 
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.7.1 to 3.8.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Changelog](https://github.com/tox-dev/py-filelock/blob/main/docs/changelog.rst)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.7.1...3.8.0)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-11 09:14:12 +05:45
dependabot[bot] c134d4ab8e build(deps): Bump dvc-data from 0.1.13 to 0.1.14 (#8117) 
Bumps [dvc-data](https://github.com/iterative/dvc-data) from 0.1.13 to 0.1.14.
- [Release notes](https://github.com/iterative/dvc-data/releases)
- [Commits](https://github.com/iterative/dvc-data/compare/0.1.13...0.1.14)

---
updated-dependencies:
- dependency-name: dvc-data
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-10 09:17:12 +05:45
skshetry 7a0b3e9377 Update tests/func/experiments/test_queue.py 2022-08-09 17:50:48 +08:00
karajan1001 d6f8e5a507 Add flaky to a queue related test. 
This is a flaky, the queue tasks status might remain "Running" even if
it had already been completed, beside sometimes I also noticed it raise
a pytest fixture error.
 2022-08-09 17:50:48 +08:00
Alex HallandDavid de la Iglesia Castro 2ca1d6c7c1 refactor: extract load_path function from common code using LOADERS (#7983) 
Following up on https://github.com/iterative/dvc/pull/7965#discussion_r912684074

Co-authored-by: David de la Iglesia Castro <[email protected]>
 2022-08-09 09:09:43 +00:00
Ivan Shcheklein f7fbac1eed stage add: fix hint message consistency (#8112) 2022-08-09 11:18:37 +05:45
skshetry 7c1e339650 data status: handle path joining properly for tree leaves/items (#8107) 2022-08-09 09:34:15 +05:45
Ivan Shcheklein b7c0c41262 lock test name fix 2022-08-09 00:47:04 +03:00
karajan1001 9180a718c1 queue status: show final experiment name 
fix: #8059

1. showing final experiment name in `queue status` for successful tasks.
2. Add a unit test for `celery_queue.status`
 2022-08-08 15:49:51 +09:00
karajan1001 306456d453 queue status: ERROR: Invalid experiment 
fix: #8014
> ERROR: Invalid experiment '{entry.stash_rev[:7]}'.

This happens when the queue task failed from a scm error.

{"exc_type": "GitMergeError", "exc_message": ["Cannot fast-forward HEAD
to '05100047a341f2fa4a02421289d48f84d8c45e86'"], "exc_module": "dvc.scm"}.

And the failed task neither create a infofile nor create a fail_stash.

1. Didn't raise DvcException if no infofile found for failed tasks.
   (success tasks still raise it)
2. Add a new unit test for this.
 2022-08-08 15:07:11 +09:00
Ivan Shcheklein 95421a5605 fix logic for unlock on unlocked lock calls 2022-08-08 14:25:10 +09:00
David de la Iglesia Castro 31635b8312 exp run: Use hydra for parsing --set-param. 
- Overriding a config value : foo.bar=value
- Appending a config value : +foo.bar=value
- Appending or overriding a config value : ++foo.bar=value
- Removing a config value : ~foo.bar, ~foo.bar=value

See https://hydra.cc/docs/advanced/override_grammar/basic/#modifying-the-config-object

---

Breaking changes:

To modify a list, `foo[0]=bar` must now be passed as `foo.0=bar`.

Modifying a nested list inside a dictionary is not supported by omegaconf.

---

Closes #4883
Closes #5868
Closes #6129
 2022-08-05 17:53:24 +02:00
David de la Iglesia Castro 45ac64ddde api: Fix params_show when stage has no params 
Also fix when `deps=True` and no params in the entire pipeline.

Closes #8064
2.17.0 		2022-08-05 16:26:19 +03:00
Sami Abdul-Wahid c7b22b467c Upgrade from toml to tomlkit library (#7468) 
* Upgrade from toml to tomlkit library
 2022-08-05 18:58:22 +05:45
karajan1001 ae83e3c787 Now exp remove can clean up failed tasks and associated successful tasks 
fix: #8018
fix: #8071

1. Add one line to make `exp remove` can remove failed tasks, and will
   also delete associated successful tasks
2. Add a new unit test for this behaviour.
3. Solve the unstable test_queue problem.
 2022-08-05 16:09:51 +09:00
karajan1001andPeter Rowlands f68c4e9a4b Refactor dvc.repo.experiment.remove to make matching exp by name also 
return its queue entry info.

Some review changes.

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
 2022-08-05 16:09:51 +09:00
karajan1001 2182c4508d Make exp remove flags and arguments mutually exclusive. 
1. Current behavior didn't follow the either or description.
2. Make the underlying logic more clear.
 2022-08-05 16:09:51 +09:00
karajan1001 eaab253e2b Refactor dvc.repo.experiments.queue.remove.py 
Make a better API for other place to call.
 2022-08-05 16:09:51 +09:00
karajan1001 905c416ffc Decouple queue clear and remove commands 2022-08-05 16:09:51 +09:00
karajan1001 a6063307b5 Replace the _clear_queue function with celery_queue.clear 2022-08-05 16:09:51 +09:00
Ruslan Kuprieiev 704191b15a output: resolve relative to stage.wdir only on the same fs 
Leftover from https://github.com/iterative/dvc/pull/8091
 2022-08-04 19:25:31 +03:00
dependabot[bot] 5625a94365 build(deps): Bump dvc-data from 0.1.12 to 0.1.13 
Bumps [dvc-data](https://github.com/iterative/dvc-data) from 0.1.12 to 0.1.13.
- [Release notes](https://github.com/iterative/dvc-data/releases)
- [Commits](https://github.com/iterative/dvc-data/compare/0.1.12...0.1.13)

---
updated-dependencies:
- dependency-name: dvc-data
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-08-04 20:26:18 +05:45
dependabot[bot] 634894d3b1 build(deps): Bump dvc-data from 0.1.11 to 0.1.12 
Bumps [dvc-data](https://github.com/iterative/dvc-data) from 0.1.11 to 0.1.12.
- [Release notes](https://github.com/iterative/dvc-data/releases)
- [Commits](https://github.com/iterative/dvc-data/compare/0.1.11...0.1.12)

---
updated-dependencies:
- dependency-name: dvc-data
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-08-04 13:22:48 +03:00
Ruslan Kuprieiev 0ce4e769f1 repo: make lock/rwlock kinda work on non local fs 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev 2c3655f7ff repo: use fs.makedirs instead of os.makedirs 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev 52bdd90840 repo: support basic workflow on s3fs/adlfs/etc 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev fcc7240617 odb: use odb.path instead of odb.cache_dir 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev ecd98bdf7d odbmgr: separate repo and local odbs 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev e525d9bedb deps: bump dvc-data to 0.1.11 2022-08-03 21:10:28 +03:00
Yury 27b2a5cd6d revert logger to lazy formatting 2022-08-03 13:18:10 +03:00
Peter Rowlands 8518fe639f dvc-task: update to 0.1.2 2022-08-03 16:44:36 +09:00
Ruslan Kuprieiev 496599518a deps: bump dvc-data to 0.1.9 2022-08-02 23:57:34 +03:00
Yury 116ad512e4 plots: remove legacy code from verify_metrics 
Fixes: #7988
 2022-08-02 14:38:29 +09:00
Ruslan Kuprieiev 6a7b55fdf7 datafs: use dvc-data's datafs 2022-08-02 03:13:06 +03:00
Paweł Redzyński ed62ddb373 cmd: plots: update messaging 
Related: iterative/dvc.org#3691
2.16.0 		2022-08-01 17:29:13 +02:00
Peter Rowlands 38b1a41f2b build: add pyinstaller hooks for dvc-task and celery 2022-08-01 12:26:53 +03:00
skshetry 6f0736142b deps: bump dvc-data to 0.1.6 2022-07-28 21:45:23 +05:45
Gao 737d7cda4b queue: missing f format for the error message. 2022-07-28 21:15:51 +08:00
dberenbaum 75745a70b4 readme: add exps to quickstart 2022-07-28 10:16:32 +02:00
Ivan Shcheklein 9ff18502b6 add gdrive remote profile field to config 2022-07-27 23:29:18 +03:00
Paweł Redzyński 44a4fb59a7 plots: account for wdir in stage plots 
Fixes: #8003
 2022-07-26 13:59:24 +02:00
Paweł Redzyński 6543683c0c project: update dvc render and dvclive test dependencies 2022-07-26 13:40:59 +02:00
pre-commit-ci[bot] 8fee26c3d2 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2022-07-26 15:02:09 +05:45
Gaoandskshetry 7c8994b538 Update dvc/repo/experiments/queue/workspace.py 
Co-authored-by: skshetry <[email protected]>
 2022-07-26 15:02:09 +05:45
karajan1001 9dfe88e2b1 exp run: workspace status wrong 
fix: #8045
 2022-07-26 15:02:09 +05:45
Saugat Pachhai (सौगात) d0eda1d538 data status: add spinners during diff 2022-07-25 19:32:19 +05:45
Saugat Pachhai (सौगात) 896acedb07 data status: optimize printing status list 
`--untracked-files` and `--unchanged` coupled with `--granular`
puts a lot of stress on `show_status`. Currently, with those options,
dvc spends ~26s on `_show_status` function.

Unhappy with the above result, I looked into improving the situation with
`console.out`. It did reduce that to 9s but it still felt like a lot.

Then I dropped to good old colorama, which dropped to 3s(term)/1.24(redirected).
I tested with buffering output which dropped this further to
~600ms(term)/~100ms(redirected).

This may seem like premature optimization, but this was preventing me
from figuring out perf issues in granular outputs, and I believe the
user should not be penalize for using feature that we already offer.
 2022-07-25 17:47:23 +05:45