Commit Graph
17 Commits
Author SHA1 Message Date
pre-commit-ci[bot]andSaugat Pachhai 502b3df6c9 [pre-commit.ci] pre-commit autoupdate (#9756)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.9.0 → v3.10.1](https://github.com/asottile/pyupgrade/compare/v3.9.0...v3.10.1)
- [github.com/astral-sh/ruff-pre-commit: v0.0.278 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.278...v0.0.281)
- [github.com/PyCQA/flake8: 6.0.0 → 6.1.0](https://github.com/PyCQA/flake8/compare/6.0.0...6.1.0)

* fix issues reported by ruff

- also remove redundant flake8 plugins
- remove beautysh as we don't have any scripts now
- remove pyupgrade as it is redundant due to ruff

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-08-02 18:08:30 +05:45
pre-commit-ci[bot]andSaugat Pachhai ca3704997b [pre-commit.ci] pre-commit autoupdate (#9632)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.3.1 → v3.9.0](https://github.com/asottile/pyupgrade/compare/v3.3.1...v3.9.0)
- https://github.com/charliermarsh/ruff-pre-commit → https://github.com/astral-sh/ruff-pre-commit
- [github.com/astral-sh/ruff-pre-commit: v0.0.261 → v0.0.277](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.261...v0.0.277)
- [github.com/codespell-project/codespell: v2.2.4 → v2.2.5](https://github.com/codespell-project/codespell/compare/v2.2.4...v2.2.5)

* fix ruff complaints

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-07-14 20:11:15 +05:45
skshetry 1de37556d5 bump ruff to v0.0.247 (#9050)
* bump ruff to v0.0.247

* ruff: enable pep8-naming check

rename some camelCased APIs, especially in logger
rename G variables to graph
ignore some variable namings and help messages in dvc.commands
2023-02-18 16:50:35 +05:45
karajan1001 6bfce5a5db Use a more granular lock on exp operations.
For now in `experiment.new`, `init_executor` and `collect_exectuor` we
lock the whole scm repo. We can make a more granular lock in these
operations.
2022-11-28 17:04:16 +08:00
karajan1001 6bd14b21ef Move infofile operation to executor and
Currently infofile dump was spread in both executor and queue classes.
put them all into executor to make them more managable. Move all ending
dump operations to the cleanup function, because the collect_result progress
might failed because of git operations, in which can guarantee them to
be run in a `finally` scope.
2022-09-26 16:25:47 +08:00
karajan1001 f23bc71eea Use task status to replace collected.
fix: #8088

1. Use status to replace collected.
2. Make status show more accurate.

Fix lint
2022-09-26 16:25:47 +08:00
karajan1001 90109bccd7 Refactor seperate the initialization of executor and setup environment
Reason:
1. Need some more granular control over initialization progress of
   executor.

What done:
1. Seperate `init_git` and `init_cache` progress out from setup_executor
2. Move `set_ref` into `init_git`
2022-09-26 16:25:47 +08:00
Ruslan Kuprieiev 3fc9f323a7 fs: don't import SSHFileSystem 2022-08-18 00:23:40 +03:00
Peter Rowlands 7113568c60 exp: refactor base.py into refs.py and exceptions.py 2022-07-12 21:39:49 +09:00
Saugat Pachhai (सौगात) 570e3a5359 dvc.fs: move fs to dvc.objects.fs
Some of the APIs/FileSystems are reimported inside dvc.fs module.
2022-05-16 13:15:19 +03:00
Peter Rowlands dd5d999644 exp: add exp run --machine flag 2022-01-13 16:34:20 +09:00
Peter Rowlands 3c7695b39d exp: add SSHExecutor.reproduce via exp exec-run 2022-01-13 16:34:20 +09:00
Peter Rowlands 84da78a62b exp: initial SSHExecutorManager implementation 2022-01-13 16:34:20 +09:00
Peter Rowlands 37f8010b42 exp: initial executor manager refactor
- introduce ExecutorManager classes
- move executor init + execution from experiments.__init__ into the
  manager classes
- make executors serializable via ExecutorInfo
2021-12-15 15:55:32 +09:00
Ruslan Kuprieiev b6f2a8e653 tests: remove redundant remote tests
With introduction of https://github.com/iterative/dvc-plugins,
all important tests are being ran there, so there is no need
for dvc repo to run them again.

This PR also removes some redundant/broken remote tests.
2021-12-03 22:45:41 +02:00
Batuhan Taskaya 9aec8f1926 dvc: get rid of path_info
Intermediate PR in removing path_info from our code base.

It is completely removed from the main codebase and is still
being used in tests, which will be migrated in the followups.

For fs-specific simple path manipulation, we've introduced
`fs.path` helper (similar to `os.path`).

Fixes #6774
2021-11-10 00:52:48 +02:00
Peter Rowlands (변기호) 0d895e2661 exp: add initial ssh executor (#6833)
* executor: move local specific git initialization out of base

* machine: return kwargs for executor init instead of executor object

* executor: add initial ssh executor

* ssh-executor: disable strict host checking for dvc-machine instances

* initial tests for ssh executor

* drop GIT_SSH_COMMAND usage

* push: accept destination odb as a parameter

* separate cache initialization into executor.init_cache

* add test for ssh executor cache init

* remove dead code

* machine: fixture-ize default test scenarios

* test SSHExecutor.from_machine

* tpi: return default SSH port for constructing executors
2021-10-26 16:08:17 +09:00