Commit Graph
8177 Commits
Author SHA1 Message Date
Yury 6edca79b79 repro dry: show information if the stage is cached
Fixes: #8342
2022-11-24 21:49:52 +02:00
Ruslan Kuprieiev 2a888970e8 build: fpm: don't double quote cli option
Leftover from #8611
2.35.1
2022-11-24 01:34:18 +02:00
Ruslan Kuprieiev 73fdfc8059 build: fpm: don't create .build-id/* files
These are leaking into our rpm packages.

Related https://github.com/iterative/dvc-s3-repo/issues/86
2.35.0
2022-11-24 01:02:44 +02:00
Ruslan Kuprieiev 8dc11417d2 testing: api: test opening a file in subdir
Related to #8420
2022-11-23 17:47:13 +02:00
b02855fb93 ui: Fix WSL check in open_browser (#8604)
* ui: Fix WSL check in `open_browser`

* Update dvc/ui/__init__.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-11-23 09:57:57 +00:00
Peter Rowlands af255a46b4 worktree: use progress bars in push/fetch 2022-11-23 13:05:51 +09:00
Peter Rowlands 2748492f6b worktree push: don't push existing versions 2022-11-23 13:05:51 +09:00
Peter Rowlands c69d3188a6 deps: bump dvc-data to 0.28.0 2.34.3 2022-11-22 19:04:18 +09:00
karajan1001 9b9c04e633 Solve the problem during executor running
fix #7693

The problem was when we gathered experiments, and at the same time,
the executor are creating new commits. It will make the checkpoint
top/head return by `get_running_exps`/`get_ref`/`branch_revs` not the
same one. And cause the problem described in #7693

To solve this we did
1. move the `get_rev` to the beginning of `exp show` to make sure the
   return results in `branch_revs`
2. Add a new function to move all of the properties that only belong to
   `head` of the checkpoint from the wrong place to the top.
3. Add a new unit test for it.
4. Force fetch_running_exp to return data.
5. fix a bug in tmp_dir_executor.
2022-11-22 17:44:55 +08:00
karajan1001 350e0fb1f7 queue kill: we can manually mark problematic tasks as failure
fix: #8461
fix: #8473
For now, for those inactive but uncompleted tasks, they still shown as
Running in `exp show` and `task queue`, we cannot remove them because
they are regarded as running, we cann't kill them because their are no
real process behind them.

1. Now we can examine these false running tasks and mark them as failure
   in `queue kill` command. #8461
2. Better msg handle for initliaizing tasks in `queue kill`#8473
3. Add new unit tests for both cases.
4. bump into new dvc-task 0.1.6
2022-11-22 13:33:23 +08:00
dberenbaum f14358bc72 plots: add test for default x label 2022-11-21 21:05:12 +01:00
David de la Iglesia Castro 705bc6ec82 Added test_infer_x_label 2022-11-21 21:05:12 +01:00
dberenbaum 61a02e6f56 plots: set default x label 2022-11-21 21:05:12 +01:00
Saugat Pachhai (सौगात) 8e5226ece0 cli: remove version action 2022-11-21 20:37:05 +05:45
Peter Rowlands 2a1af0b880 add: merge existing file meta for versioned dirs 2022-11-21 19:18:41 +09:00
Peter Rowlands 014d149512 out: move restore_fields logic into Output 2022-11-21 19:18:41 +09:00
Peter Rowlands 77ae4aa92e deps: bump dvc-azure to 2.20.5 2022-11-21 18:21:02 +09:00
Peter Rowlands 588471a811 deps: bump dvc-data to 0.27.0 2022-11-21 18:21:02 +09:00
dependabot[bot] e59c19e752 build(deps-dev): Bump dvc-render from 0.0.13 to 0.0.14
Bumps [dvc-render](https://github.com/iterative/dvc-render) from 0.0.13 to 0.0.14.
- [Release notes](https://github.com/iterative/dvc-render/releases)
- [Commits](https://github.com/iterative/dvc-render/compare/0.0.13...0.0.14)

---
updated-dependencies:
- dependency-name: dvc-render
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-11-21 10:08:09 +01:00
David de la Iglesia CastroandDave Berenbaum b0a92414fa hydra: Use OmegaConf.to_yaml for dumping .yaml output. (#8587)
* hydra: Use `OmegaConf.to_yaml` for dumping `.yaml` output.

Combining `OmegaConf.to_object` with our yaml DUMPER (ruaml.yaml) was not correctly handling strings like 'no'.

`OmegaConf.to_yaml` uses a custom string representer that correctly handles those cases.

Fixes #8583

* Update dvc/utils/hydra.py

Co-authored-by: Dave Berenbaum <[email protected]>

Co-authored-by: Dave Berenbaum <[email protected]>
2022-11-19 12:37:18 +00:00
Peter Rowlands dae500d7c7 worktree: use view instead of full index in push/fetch 2022-11-19 09:55:58 +09:00
Peter Rowlands a01bc2918e push/fetch: move worktree logic into repo.worktree 2022-11-19 09:55:58 +09:00
David de la Iglesia Castro a33bbc97d5 parsing: Escape str interpolation in dict unpacking.
Use subprocess.list2cmdline (windows) / shlex.quote

Closes #8203
Closes #8370
2022-11-18 20:00:16 +01:00
Paweł Redzyński ba2e8bdac7 plots: data conversion: adjust for viewer backend (#8421)
* plots: vega: converter: support multi series data split and props inference

Fixes: #7754, #8399
Related: iterative/studio#4369

* plots: converter: make converter infer x and y labels if not provided

* plots: vega: converter: roll back to datapoints format

* plots: vega: converter: refactor

* plots: infer labels on

* plots: get_plot_id: fix naming

* plots: vega: converter: notes regarding converting
2022-11-18 12:36:00 +00:00
Peter Rowlands 4d6e045989 status -c: exclude push: false outs 2022-11-18 16:32:29 +09:00
Peter Rowlands 00eea4560e import-url: default to push: false for cloud-versioned imports 2022-11-18 16:32:29 +09:00
Peter Rowlands 6e02dddabd output: check push in get_used_objs 2022-11-18 16:32:29 +09:00
Peter Rowlands b550c5e625 output: add PARAM_PUSH 2022-11-18 16:32:29 +09:00
Ruslan KuprieievandPeter Rowlands b36861ada9 tests: import: use odb.clear()
Co-authored-by: Peter Rowlands <[email protected]>
2022-11-17 10:56:01 +02:00
Ruslan Kuprieiev d482e26c99 deps: bump dvc-data to 0.26.0 2022-11-17 10:56:01 +02:00
dependabot[bot] a8ad59ae4e build(deps-dev): Bump dvc-render from 0.0.12 to 0.0.13
Bumps [dvc-render](https://github.com/iterative/dvc-render) from 0.0.12 to 0.0.13.
- [Release notes](https://github.com/iterative/dvc-render/releases)
- [Commits](https://github.com/iterative/dvc-render/compare/0.0.12...0.0.13)

---
updated-dependencies:
- dependency-name: dvc-render
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-11-16 11:13:05 +01:00
Ivan Shcheklein 76cc649f5e fix/8550+: cover with test 2022-11-16 15:16:37 +05:45
Ivan Shcheklein d52e1aa17b fix: exp list ref heads handling 2022-11-16 15:16:37 +05:45
Saugat Pachhai (सौगात) 65f61acffa tests: garbage collect sqlite connections on Windows + Python 3.11
There seems to be a regression in Python 3.11, where the sqlite
connections are not deallocated, due to some internal changes in
Python 3.11, where they are now using LRU cache. They are not deallocated
until `gc.collect()` is not called.

See https://github.com/python/cpython/issues/97641.
This affects only Windows, because when we try to remove the tempdir for
the exp run, the sqlite connection is open which prevents us from
deleting that folder.

Although this may happen in real scenario in `exp run`, I am only fixing
the tests by mocking `dvc.close()` and extending it by calling
`gc.collect()` after it. We could also mock `State.close()` but didnot
want to mock something that is not in dvc itself.

The `diskcache` uses threadlocal for connections, so they are expected
to be garbage collected, and therefore does not provide a good way to
close the connections. The only API it offers is `self.close()` and that
only closes main thread's connection. If we had access to connection,
an easier way would have been to explicitly call `conn.close()`.
But we don''t have such option at the moment.

Related: https://github.com/iterative/dvc/pull/8404#issuecomment-1310024661
GHA Failure: https://github.com/iterative/dvc/actions/runs/3437324559/jobs/5731929385#step:5:57
2022-11-16 08:02:43 +05:45
Peter Rowlands fdc689954f pyinstaller: drop google-api-client hooks 2.34.2 2022-11-15 22:07:05 +09:00
Peter Rowlands 076c48532d deps: bump dvc-gdrive to 2.19.1 2022-11-15 22:07:05 +09:00
David de la Iglesia Castro e4ea463c73 hydra: Raise error when name and sweeps. 2022-11-14 21:34:55 +01:00
Saugat Pachhai (सौगात) 87d8269365 typecheck: fix type error in _git_info 2022-11-14 10:21:31 +05:45
Peter Rowlands a753ee215f output: fix get_entry dep object generation for imports 2022-11-13 14:41:21 +02:00
dependabot[bot] 014a38fae2 build(deps-dev): Bump iterative-telemetry from 0.0.5 to 0.0.6
Bumps [iterative-telemetry](https://github.com/iterative/telemetry-python) from 0.0.5 to 0.0.6.
- [Release notes](https://github.com/iterative/telemetry-python/releases)
- [Commits](https://github.com/iterative/telemetry-python/compare/0.0.5...0.0.6)

---
updated-dependencies:
- dependency-name: iterative-telemetry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2.34.1
2022-11-11 14:54:07 +09:00
Peter Rowlands 8913f5a6fe mypy: allow implicit-optional 2022-11-11 14:17:17 +09:00
dependabot[bot] 4ce9825c2b build(deps-dev): Bump mypy from 0.982 to 0.990
Bumps [mypy](https://github.com/python/mypy) from 0.982 to 0.990.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](https://github.com/python/mypy/compare/v0.982...v0.990)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-11-11 14:17:17 +09:00
dependabot[bot] e9c5b5fa84 build(deps-dev): Bump dvc-gs from 2.19.1 to 2.20.0
Bumps [dvc-gs](https://github.com/iterative/dvc-gs) from 2.19.1 to 2.20.0.
- [Release notes](https://github.com/iterative/dvc-gs/releases)
- [Commits](https://github.com/iterative/dvc-gs/compare/2.19.1...2.20.0)

---
updated-dependencies:
- dependency-name: dvc-gs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-11-11 13:33:10 +09:00
karajan1001 fa54c1af7c Solve run_all flaky tests
1. Modify run all to include currently running exps.
2. bump dvc-task to 0.1.5
2022-11-11 07:04:43 +05:45
karajan1001 9fbbfd07a6 Fix some celery queue realted ci failure
fix: #8403
1. remove some of the flaky mark
2. In `get_result` make sure the celery task is completed.
2022-11-11 07:04:43 +05:45
karajan1001 929de7c651 Solve the crash on getting name of applied experiment branch
fix: #8526
1. The current `get_names` will crash if an experiment branch had
   already been given a new branch name to it.

1. Solve the crash when we getting the name of a newly fixed experiment
   branch.
2. Add a new unit test to it.
2022-11-10 13:18:45 +08:00
blakeNaccarato 49af6e1435 plots: support svg
Add SVG to parser and parametrize an existing functionality test on SVG as well as JPG.

Fixes #6714.
2022-11-09 19:23:49 +02:00
skshetry a6bfc7db4e ci: test with 3.11 on all OSes (#8543)
* ci: test with 3.11 on all OSes

* Update tests.yaml

* Update tests.yaml
2022-11-09 12:05:27 +00:00
Yury 872a0d663e dvc exceptions CyclicGraphError: add more clear message for the exception
Fixes: #7915
2022-11-09 14:33:08 +05:45
dependabot[bot] c01583f3dd build(deps-dev): Bump dvc-task from 0.1.4 to 0.1.5
Bumps [dvc-task](https://github.com/iterative/dvc-task) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/iterative/dvc-task/releases)
- [Commits](https://github.com/iterative/dvc-task/compare/0.1.4...0.1.5)

---
updated-dependencies:
- dependency-name: dvc-task
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-11-09 16:37:14 +09:00