Commit Graph
3241 Commits
Author SHA1 Message Date
skshetry 3c570a2177 exp push: improve ui (#9308)
* exp push: improve ui

* fix misc suggestions
- show url even when the refs are up-to-date.
- show url at the end
- add Git remote information in up-to-date UI message.

* fix diverged exps message
2023-04-06 09:54:39 +05:45
David de la Iglesia Castro 3312a90949 exp run: Add --copy-paths arg.
List of paths to copy inside the temp directory. Only used if `--temp` or `--queue` is specified.

Closes #5800
2023-04-05 23:17:07 +02:00
Alexander Guschin 32bebc0246 Add top-level artifacts: section (#9220)
* add artifacts section

* adding and removing artifacts

* delete code for add/remove; add tests

* fix pre-commit

* remove extra piece of writing to dvc.yaml

* remove extra piece

* simplify kwargs

* allow same names for artifacts in different dvc.yaml files

* set ID as path; don't return dvc.yaml(s) with no artifacts

* fix PR feedback

* issue warning at incorrect artifact name

* issue warning at incorrect artifact name

* reference right schema

* add r to regexp

* add tests for regexp

* set artifacts for SingleStageFile

* fix windows test

* make path required; simplify regexp and remove slash from it

* allow to use gto's artifacts.yaml

* remove extra piece of code for migration from gto
2023-04-05 13:31:35 +00:00
David de la Iglesia Castro 35648134f7 exp run: git add param files that will be modified.
Only if they are currently untracked and `--temp` or `--queue` is passed.

Closes #7842
2023-04-05 10:28:18 +02:00
Ruslan Kuprieiev 40f7f34889 api: data: set default remote instead of using it directly
Fixes #9139
Fixes #7341 (see https://github.com/iterative/dvc/pull/7363#issuecomment-1032959843)
2023-04-05 02:47:35 +03:00
skshetry 8be71be540 utils.studio: disallow redirects, misc API refactoring (#9301) 2023-04-04 13:15:08 +05:45
skshetry 7c3c47b4a2 bump ruff to 0.0.260 and enable bandit (#9300)
* bump ruff to 0.0.260 and enable bandit

* install toml extra for bandit

* suppress insecure use of hash md5
2023-04-04 10:01:43 +05:45
David de la Iglesia Castro 3a2259821b exp save: Include LOCK_FILE 2023-04-03 10:33:52 +02:00
Ruslan Kuprieiev ecdc44e010 dep: repo: use dvcfs
Makes dvc get/import/etc use dvcfs, which already supports cloud versioning,
circular imports and other stuff.

Also makes dvc import behave more like dvc import-url, so that we can use
the same existing logic for fetching those using index instead of objects.

Fixes #8789
Related iterative/studio#4782
2023-04-02 03:55:02 +03:00
David de la Iglesia Castro 24e10e0278 exp save: Track top level files 2023-03-31 19:23:40 +02:00
David de la Iglesia Castro 7c8e6e3156 exp run: Track top level files. 2023-03-31 19:23:40 +02:00
David de la Iglesia Castro d2812fa48f studio: Don't fail on empty params. 2023-03-31 19:23:40 +02:00
skshetry 086d51d9de plots json: return definition errors unrelated to renderers (#9293) 2023-03-31 16:34:26 +05:45
David de la Iglesia Castro 1d8b2e5fa6 exp run: Propagate .dvc/config.local in TempDirExecutor.
Closes #7254
2023-03-31 09:02:22 +02:00
skshetry 696fabc745 exp-push: show url to the Studio project (#9283) 2023-03-30 12:52:09 +05:45
skshetry 780dc72301 Revert "exp-push: show url to the Studio project" (#9282)
Revert "exp-push: show url to the Studio project (#9278)"

This reverts commit 1ca3ef14a3.
2023-03-30 08:28:44 +05:45
skshetry 1ca3ef14a3 exp-push: show url to the Studio project (#9278) 2023-03-30 08:27:47 +05:45
skshetry 593f1064cf utils.studio: refactor to return json response (#9277) 2023-03-29 20:02:40 +05:45
Ruslan Kuprieiev 35bb261b89 config: don't look for .dvc dir implicitly
Currently if dvc_dir doesn't exists (e.g. we are creating an uninitialized
repo), config will try looking for `.dvc` dir in the current directory,
which might be inside of another dvc repository resulting in mixed configs.

Use explicit `from_cwd` method to make `Config` try to find repo configs
if available to avoid the usage mixup.
2023-03-29 16:27:54 +03:00
David de la Iglesia Castro fe6d543c1f tests: Add UnresolvedExpNamesError check in test_remove_remote 2023-03-29 12:56:59 +02:00
skshetry cc4816f652 utils.studio: require token (#9255)
This is a simple refactoring which makes the caller to choose appropriate token instead of
the studio.notify_refs making a decision for them.
2023-03-28 13:32:05 +05:45
skshetry faa1bdfbce use git remote to find out remote repo url when pinging Studio (#9254) 2023-03-28 12:17:48 +05:45
skshetry 1bc4c1dceb exp share refactor - ping on exp:remove, do not fail on error (#9248)
studio experiments share refactor

1) Pings Studio on exp remove
2) Does not fail on any error during request.post
3) UI improvement on exp remove
2023-03-27 22:51:50 +05:45
skshetry 531b2110cf Revert "exp push: add trailing slash to the endpoint url" (#9243)
Revert "exp push: add trailing slash to the endpoint url (#9240)"

This reverts commit 5485275b92.
2023-03-27 14:19:12 +05:45
skshetry 43195a806d plots: return errors in json format (#9146)
* plots: return errors in json format

* plots --json: move plots data into data key

* rename  revision key to rev

* move imports to top-level

* add tests
2023-03-27 12:53:43 +05:45
David de la Iglesia CastroandDave Berenbaum e9fb0b92aa exp branch: Make branch arg optional.
Default to `f"{ref_info.name}-branch"` where ref_info is inferred from the `exp_rev` argument.

Co-authored-by: Dave Berenbaum <[email protected]>
2023-03-24 19:11:06 +01:00
skshetry 5485275b92 exp push: add trailing slash to the endpoint url (#9240) 2023-03-24 22:35:20 +05:45
David de la Iglesia Castro 6b6dc01ac4 exp save: Reintroduce implementation based on executor/queue.
Closes #9058 .

Ensures behavior of `_stash_exp` in `exp run` is matched in `exp save` without duplicating logic.

The simpler implementation ended up being worse for maintenance when wanting to make `exp run` and `exp save` behavior in sync.
2023-03-24 10:43:24 +01:00
Peter Rowlands 91d3ff0f69 exp: mark q remove test as flaky 2023-03-24 17:53:40 +09:00
Dave Berenbaum da3bbccf66 Update duplicate output error msg (#9138)
* update dup output error msg

* update dup output error msg

* stage add: add test for duplicated output error msg

* cleanup OutputDuplicationError code
2023-03-23 15:54:44 +00:00
pre-commit-ci[bot]andDavid de la Iglesia Castro 48b56c83dc [pre-commit.ci] pre-commit autoupdate (#9168)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/charliermarsh/ruff-pre-commit: v0.0.254 → v0.0.257](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.254...v0.0.257)
- [github.com/codespell-project/codespell: v2.2.2 → v2.2.4](https://github.com/codespell-project/codespell/compare/v2.2.2...v2.2.4)

* pre-commit: fix ruff errors

* pre-commit: fix flake8

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: David de la Iglesia Castro <[email protected]>
2023-03-23 13:05:31 +00:00
Peter Rowlands f8bbf20b77 exp: use individual q for timeout dependent tests 2023-03-23 19:18:52 +09:00
Peter Rowlands cd2c4d0caf tests: remark flaky queue tests as xfail on windows 2023-03-23 19:18:52 +09:00
David de la Iglesia Castro 9b329d01bf api: Add exp_save. 2023-03-23 10:33:55 +01:00
Ruslan Kuprieiev 2d5c15bee1 erepo: move legacy exceptions to api (#9228)
* erepo: get rid of NoRemoteInExternalRepoError

This exception hasn't been making any sense at all since we've introduced
get and import, as they can be chained in repos.

* erepo: get rid of NoOutputInExternalRepoError

This is dead code these days.

* erepo: get rid of FileMissingError and PathMissingError

* api: preserve legacy exc wrappers
2023-03-23 01:40:42 +02:00
Peter Rowlands 7868c1301c exp: enforce waiting for task completion in CI 2023-03-22 18:58:42 +09:00
Ruslan Kuprieiev dfe76c54d7 erepo: set subrepos/uninitialized explicitly 2023-03-22 02:26:44 +02:00
David de la Iglesia Castro 121d56b739 hydra: resolve OmegaConf interpolations. (#9212)
Closes #9196
2023-03-20 21:38:22 +05:45
skshetry 8399bf612a sconfig: add support for studio token (#9213)
config: add support for studio token
2023-03-20 20:59:52 +05:45
daniele 6c1b786564 parsing: fix foreach resolution with int keys (#8904) 2023-03-20 18:40:30 +05:45
David de la Iglesia Castro 2d820cccad deps: bump dvc-render>=0.3.1<0.4.0.
For https://github.com/iterative/dvc-render/pull/124
2023-03-16 21:36:02 +01:00
skshetry 58029e9719 exp push: push experiments revs to studio (#9160)
* exp push: push experiments revs to studio

* log json data that is sent
2023-03-16 11:24:50 +05:45
Ruslan Kuprieiev 990e834f1d repo: open: handle bare git repos
If we are asked to open a potentially unitialized repo, we need to make sure
it is not a bare git repo, so that we don't accidentally start putting dvc
files inside git control dir.

Discovered while working on #8789
2023-03-16 02:03:31 +02:00
Dave Berenbaum 12955da3c3 exp: fail fast in empty git repo (#8840) 2023-03-14 17:16:17 +00:00
David de la Iglesia Castro e73cc9b40a exp: Include STUDIO_REPO_URL in run_env.
Closes #8951.
2023-03-14 13:43:03 +01:00
Ruslan Kuprieiev b56f6b69dd exp: push/pull: don't include baseline commit
Fixes #6592
2023-03-14 14:00:14 +02:00
skshetry 16ecd53b71 tests: remove gc.collect() from windows (#9159) 2023-03-13 13:37:57 +05:45
Ruslan Kuprieiev 612f3039e0 repo: move state/index/data_index to site_cache_dir
We use all of these for optimization and should not depend on a potentially slow or broken
filesystem on which repository resides (could be HDD or CIFS/NFS/overlayfs/etc).

This PR means a few things:
    * all odd filesystems that dvc repo might be on will work quickly out of the box
    * when we start caching cloned repositories for `dvc get/import`, we will be
      able to cache indexes for them as well (note that bare clones don't have a
      worktree so there is no place for .dvc/tmp with the old approach).
    * `index.dir` and `state.dir` are now obsolete.
    * `dvc init` will cleanup site_cache_dir.

Related #5562
2023-03-13 02:58:23 +02:00
skshetry aaa825a9a8 config: ignore unspecified keys from feature section (#9153)
A warning is shown to the user for the unsupported/ignored keys, and then ignored.
The warning is raised only once per interpreter session.
2023-03-10 21:52:05 +05:45
Ruslan Kuprieiev 3aa9f51e36 output: suppress cloud.pull errors on cache collection
Fixes https://github.com/iterative/studio/issues/5261
2023-03-09 22:44:32 +02:00