skshetry
83dd2b2a7b
test: import-db: test for external model ( #10112 )
...
* test: import-db: test for external model
* close repo
2023-11-27 16:13:44 +05:45
Ivan Shcheklein
d238350a2d
fix(config): respect global config env variable
2023-11-23 20:51:52 +02:00
skshetry
3d7bbb8492
tests: add functional test for db ( #10101 )
...
* tests: add functional test for db
* fix test
2023-11-22 16:15:07 +05:45
skshetry
fd7c891217
implement import-db command ( #10040 )
...
* implement import-db command
* generate .dvc file
* update
* fix tests
* pin dbt-fal
* try to run on ci
* use os.chdir
* fix
* revert
* require dbt-core>=1.5
* add db config schema
* remove platformdirs constraints
* use logger.getChild
* improve error messaging
* rename cli flag --export-format to --output-format
* use feature flag for db config
* hide command from dvc commands list
* refactor dependency._get
* remove todo
* refactor dependency._get
* fix dbt installation check
2023-11-21 19:38:50 +05:45
Ruslan Kuprieiev
8c9358957c
push: fix cloud versioning
2023-11-15 16:28:17 +02:00
Ivan Shcheklein
86dd0785a0
add tests for the index param keys
2023-11-14 13:21:44 +02:00
skshetry
d678773b53
remove more pylint suppression comments ( #10084 )
2023-11-10 15:19:52 +05:45
skshetry
519cce3fe4
logger: type-safe logger.trace ( #10083 )
...
* use logger.getChild from `dvc.log` instead of `logging.getLogger`
I got tired adding `# type: ignore[attr-defined]` while using `logger.trace`.
So, now you need to import logger from `from dvc.log import logger` and
create a child logger using `logger.getChild(__name__)`. `logger.getChild`
will return a logger type-hinted as a logger having a `trace` method.
So, from now on, instead of:
```python
import logging
logger = logging.getogger(__name__)
```
you'll have to do following:
```python
from dvc.log import logger
logger = logger.getChild(__name__)
```
* remove all dvc imports from dvc.logger
* use Tqdm back
2023-11-10 12:55:49 +05:45
skshetry
4ee8f554a5
use ruff formatter ( #10081 )
...
Replaces black with ruff-formatter, and isort with ruff.
2023-11-09 20:58:58 +05:45
dberenbaum
4a2c4f76e5
fix for DVCFileSystem().ls()
2023-10-31 23:38:21 +02:00
Piotr Bielak
23d2ed3d59
Unroll dict-comprehension in matrix test
2023-10-31 08:05:38 -05:00
Piotr Bielak
fcc69ab6eb
parsing: Add key to matrix stages
...
This commit adds the `key` property to the templaing context of `matrix`
stages. Similarly to `foreach` stages, the `key` is available using
`${key}` substitution.
Fixes #10043
2023-10-31 08:05:38 -05:00
dberenbaum
fc5147767f
api: make exp_show return float instead of None for zeroes
2023-10-30 13:54:31 -05:00
skshetry
89b4beb7a4
daemon: ipc to log pid from the child ( #10058 )
...
* daemon: ipc to log pid from the child
* fix tests to use pid
* also test for updater/analytics keyword
2023-10-30 20:47:52 +05:45
skshetry
48f1f52c1b
refactor: dvc.daemon ( #10045 )
...
* minor refactor to dvc.daemon
* fix tests
* run daemon only inside a subprocess
2023-10-30 11:09:55 +05:45
Ruslan Kuprieiev
3fee81f1be
push: use index push
2023-10-26 20:58:45 +03:00
Peter Rowlands
d9b2e67a0b
Revert "Enable forcing exp apply (unsaved files in the workspace)"
...
This reverts commit 3661b8998b
2023-10-26 14:48:41 +09:00
skshetry
b23ba58cd6
tests: daemon: log body of the request ( #10039 )
...
log body of the request
2023-10-19 17:16:34 +05:45
skshetry
8fb67b71a5
expose test webserver as an executable for easier testing ( #10037 )
2023-10-19 16:28:47 +05:45
skshetry
5693775459
refactor: use int as the factory ( #10036 )
...
makes it thread-safe if we need in the future. Also looks a bit clean.
2023-10-19 15:31:33 +05:45
skshetry
17f5508f49
daemon: add logging support, tests and generalize popen based runs for future ( #10033 )
...
* use popen to run updater/analytics in detached mode
subprocess does not provide a way to avoid a zombie process. Not to confuse with orphaned processes, which will be `wait`-ed by `pid 1`.
It might not work for long-running processes, but I don't think zombie processes are going to be a problem for us, as the parent process immediately exits after spawning a child process.
Even in case of `queue` based experiment runner, parent will exit before than the child process. So I don't think it's a requirement here. But it does feel wrong to leak the processes, so I have to sleep on that.
---
This PR:
1. makes `daemonized` process inherit the logging verbosity, so if the parent process used `-vv`, the daemonized process will run the same way.
2. Add `DVC_DAEMON_LOGFILE` envvar, that the daemon will log to the file. By default, the verbosity is the same as running parent processes. Without `DVC_DAEMON_LOGFILE`, the output will be redirected to `/dev/null`.
3. Adds debug/trace for updater/analytics.
I hope by doing this, we can make our life easier on debugging and make daemon more transparent/observable.
* add tests
* fixes
* use kwargs
* pass all os.environ with some modified
2023-10-19 15:14:41 +05:45
Ruslan Kuprieiev
46c65fdba2
worktree: don't modify index entries in place ( #10024 )
2023-10-17 16:30:19 +03:00
Matt Seddon
5f8afb911e
Remove VegaConverter TODO and update misleading comment
2023-10-16 20:42:48 -04:00
Heinz-Alexander Fuetterer
8ea9d3edef
chore: fix typos ( #10022 )
2023-10-14 12:54:57 +05:45
Ruslan Kuprieiev
72fd265a8f
get-url/ls-url: support remote:// with in-repo config
2023-10-13 18:58:23 +03:00
Brady Johnson
c2f8115c9a
exp rename: verify exp name follows ref format
2023-10-13 11:27:05 +09:00
Ruslan Kuprieiev
50d6788411
exp: preserve hidden alias -M for --message ( #10006 )
...
Followup for #10005
2023-10-10 04:41:43 +03:00
Matt Seddon
37a4a36dc7
exp save: use -m for --message short option
2023-10-09 09:03:44 -04:00
Dave Berenbaum
279f8cb68f
repro --pull: don't fail for cloud-versioned remotes ( #10001 )
...
* repro --pull: don't fail for cloud-versioned remotes
* raise RunCacheNotSupported for cloud-versioned remotes
2023-10-06 18:11:30 +03:00
dberenbaum
8a70877748
fix artifact name test
2023-10-05 13:39:06 -04:00
Matt Seddon
3661b8998b
Enable forcing exp apply (unsaved files in the workspace)
2023-10-04 16:12:52 -04:00
Dave Berenbaum
bc1a700254
gc: add dry option and log num removed objs ( #9952 )
...
Closes #9330
2023-09-30 02:33:44 +00:00
Ruslan Kuprieiev
4d5a0163ea
diff: handle granular targets
2023-09-28 21:54:06 +03:00
David de la Iglesia Castro
a6e0d68cd2
run-cache: Load newest instead of first found in os.listdir. ( #9908 )
...
Closes https://github.com/iterative/dvc/issues/9420
2023-09-26 15:07:13 +03:00
Peter Rowlands (변기호)
12b572596f
repo: add API and CLI for reading artifacts ( #9770 )
...
* deps: add GTO as a dependency
* deps: bump dvc-studio-client to 0.13.0
* fs: return downloaded count in fs.download()
* utils: add type hints for resolve_output
* artifacts: add `dvc artifacts get`
* api: add `api.artifacts_show`
* artifacts get: support --remote, --remote-config
* gto: update to 1.3.0
2023-09-25 20:21:34 +05:45
dberenbaum
e8362db9bc
exp: force add include_untracked paths
2023-09-25 19:02:38 +09:00
BradyJ27 and daavoo
51dc78a135
exp: add rename command ( #9812 )
...
* Add renaming for finished experiments
* Update arg parser
* Add rename by rev
* Add rename tests
* Change remove to rename; Refactor arguments error
* Remove --rev; Use resolve_name; Add test
* Update testing; Fix python3.10 rename failure
* Update rename test
* Update tests and error handling
* exp rename: Add `--force`.
Fail if experiment new_name exists but allow to overwrite it.
---------
Co-authored-by: daavoo <[email protected]
2023-09-18 17:01:32 +00:00
dberenbaum
bf2af86397
drop irrelevant errors in tests/unit/render/test_match.py
2023-09-14 10:36:46 -04:00
dberenbaum
08d37912d6
add util function to group plot definitions
2023-09-14 10:36:46 -04:00
dberenbaum
0e66aa2b6a
add integration test for dvclive 2.x->3.x plots
2023-09-14 10:36:46 -04:00
dberenbaum
f9340ed7b9
add test cases for plots group_definitions
2023-09-14 10:36:46 -04:00
dberenbaum
e1599ae280
add test for merging plot ids across config files
2023-09-14 10:36:46 -04:00
dberenbaum
7e7d0e1dd8
merge plot ids across dvc.yaml files
2023-09-14 10:36:46 -04:00
dberenbaum
d81a1bbf56
make exp run --dry imply --temp
2023-09-13 15:06:46 +09:00
Ruslan Kuprieiev
e9c7cd63a2
dvcfs: add tests for fsid
2023-09-11 21:20:12 +03:00
skshetry
f2673acb53
metrics/params: support reading from cached files and directories, use repo-relative path consistently ( #9909 )
...
* metrics
* fix path handling
* fix dvc metrics diff
* params show/diff
* share common utils
* remove onerror_collect from exp_show
* extract read_params
* move around code
* fix path handling on Windows
* rename _show to _gather_metrics
* remove _collect_vars
* add tests
* Revert "remove _collect_vars"
This reverts commit 6d2cb500c32a882497406d66fb13e613790c3436.
* bring back _collect_vars
* fix tests
* fix paths
2023-09-11 17:41:37 +05:45
David de la Iglesia Castro
7e52c233ae
import: Expose --force flag ( #9927 )
2023-09-08 18:43:13 +05:45
David de la Iglesia Castro
7fb8a9a427
checkout: Expose --allow-missing arg. ( #9919 )
2023-09-07 09:55:50 +00:00
Ruslan Kuprieiev
f47d21b0c2
[WIP] ls-url/get-url/import-url: introduce --fs-config ( #9888 )
...
ls-url/get-url/import-url: introduce --fs-config
Fixes #9886
2023-09-01 02:17:17 +03:00
Matt Seddon
5b69a16ea7
Add DVC_ROOT env var into queued experiment runs ( #9879 )
2023-08-28 22:37:03 +00:00
skshetry
Ivan Shcheklein
Ruslan Kuprieiev
dberenbaum
Piotr Bielak
Peter Rowlands
Matt Seddon
Heinz-Alexander Fuetterer
Brady Johnson
Dave Berenbaum
David de la Iglesia Castro
Matt Seddon