Commit Graph
80 Commits
Author SHA1 Message Date
Mr. Outis 0d46921ec7 py2: add unicode literals
Fix #1512
2019-01-27 20:40:21 -06:00
Ruslan Kuprieiev 6fec8b16cd RFC: conform to pylint
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-01-16 14:12:19 +02:00
Mr. Outis 7be42c8402 run: Parse and quote correctly the command
Fix: #1478
2019-01-09 00:01:55 -06:00
Mr. Outis 4c420d5f4d logger [rfc]: Use a module instead of a class
- Remove deprecation warnings
- Format error output and remove duplicated info

Close #1086
2018-12-23 21:06:39 -06:00
Ruslan Kuprieiev 83ee6e46e0 run: introduce --remove-outs option
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-12-04 09:15:23 +02:00
Ruslan Kuprieiev ac82426788 run: introduce --overwrite-dvcfile and --ignore-build-cache
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-12-04 07:17:16 +02:00
Ruslan Kuprieiev e81c6f2b34 run: introduce --force option
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-12-03 05:13:54 +02:00
Ruslan Kuprieiev 298ae356cd run: make --deterministic default
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-12-03 04:50:48 +02:00
username b3a6d70d91 [WIP] Add dvc run --deterministic option.
Tests currently failing.
2018-12-02 00:35:20 -08:00
Ruslan Kuprieiev ff577d7a87 run: use quotes to wrap arguments with spaces in them
Fixes #1372

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-11-30 10:11:49 +02:00
Ruslan Kuprieiev b159ace6d7 run: add -y|--yes option to bypass prompts
Fixes #1072

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-30 00:38:28 +03:00
Ruslan Kuprieiev 0e9774a3f8 RFC: follow flake8 rules
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-25 17:28:58 +03:00
Ruslan Kuprieiev 2cab8e03be command: remove unused code
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-19 07:00:39 +03:00
Ruslan Kuprieiev 3410f8048a dvc: optimize imports to speedup startup time
Speeds up `time dvc` from 1.2 seconds to <0.2 seconds.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-18 11:20:05 +03:00
Ruslan Kuprieiev f1a28c3812 dvc: support dvc add for external files
Fixes #890

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-16 16:34:16 +03:00
Ruslan Kuprieiev c733b8130b dvc: add dvc metrics add/show/remove and dvc run -M|--metrics-no-cache
Fixes #656

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-16 16:01:32 +03:00
Ruslan Kuprieiev d15143bb6f run: properly generate automatic name when output is a directory and no -f specified
Fixed #720

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-05-21 12:38:21 +03:00
Ruslan Kuprieiev 48e3ce03d6 dvc: run: fix dvcfile name generation
Fixes #625

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-06 02:11:48 +03:00
Ruslan Kuprieiev 14fdc18948 dvc: chain exceptions
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-06 18:38:29 +02:00
Ruslan Kuprieiev 6a6c11ffdb dvc: remove lock
As discussed with @dmpetrov, we don't really see a good usage scenario
for this and thus removing it from dvc.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-13 01:54:20 +02:00
Ruslan Kuprieiev e4b9718f34 run: add --no-exec option
Fixes #385

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-08 13:19:42 +02:00
Ruslan Kuprieiev 14181ede5f dvc: remove -D option
Fixes #370

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-04 13:34:31 +02:00
Dmitry 7a300131ce Small bugfix: args_outs --> args_outs[0] 2017-12-24 23:31:54 -08:00
Dmitry 1987693f34 Refactoring: extract method stage_file_basename() 2017-12-24 22:40:08 -08:00
Dmitry 863e0e4b14 Use output filename as a default stage file name but do not change dir 2017-12-24 18:31:15 -08:00
Ruslan Kuprieiev a779424a36 WIP RFC
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-23 09:37:40 +02:00
Dmitry e9b50093e9 Keep dvc files i the currend dir by default 2017-12-16 14:03:13 -08:00
Dmitry 282b7c6805 Use output filename as a defult stage file name. Close #351 2017-12-14 02:13:39 -08:00
Ruslan Kuprieiev 2ded6bf409 run: add -c|--cwd option
Fixes #343

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-10 06:06:29 +02:00
Ruslan Kuprieiev 927d58cfad dvc: introduce stage files instead of state files
These describe single stage of pipeline and are allowed to
be placed anywhere in current or parent directories(up to
project's root) of the data file.

By default, 'dvc run' writes stage file to ./stage.dvc, but
it can be changed by using '-f' option(see functest/test-repro.py).

For now we're traversing project tree to find needed stage files for
specific data files. This is very wasteful and not optimal, but will
do for now, as this patch is aimed at stabilizing design of our repo
files.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-07 22:52:18 +02:00
Ruslan Kuprieiev 0b639a9c09 state: rework state format
Dependencies are now specified in a more natural way:

deps:
  path/to/file1: <md5>
  path/to/file2: <md5>

And outputs also have md5 sum, that allows us to track
them by it and not by separate md5 field in state file,
which will allow us to remove strict data <-> data.state
relation in the future.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-07 01:58:13 +02:00
Dmitry 075415f6bc Git file output fix 2017-12-06 00:33:59 -08:00
Ruslan Kuprieiev bbe5e46b5c dvc: remove CommandFile
As we're moving to more user friendly single *.dvc file approach, separate
command files become uneeded.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-05 20:56:17 +02:00
Ruslan Kuprieiev e853fc76aa dvc: get rid of state dir and cache_state in favor of single .dvc
From now on everything is stored in filename + .dvc path, except for local_state.
This allows us to be more explicit and help users see everything they need in relation
to version control of their own data files.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-05 20:33:21 +02:00
Dmitry 4c388089af Rename -r to -g (--out-git). Close #322 2017-12-05 01:03:06 -08:00
Dmitry 2a980f6e57 Fix func tests: get rid of python3 lazy evaluation 2017-11-27 00:33:40 -08:00
Dmitry 4b3827255d Get rid of code duplication 2017-11-26 22:57:39 -08:00
Dmitry 40660b5120 Refactoring: extract CommandFile class 2017-11-26 21:49:06 -08:00
Dmitry 50f736d9d9 dvc run: regular file output. Close #314 2017-11-26 21:40:55 -08:00
Ruslan Kuprieiev d9e06c07b0 state: use yaml for everything
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 15:36:07 +02:00
Ruslan Kuprieiev be159e7798 dvc: fix misc dvc stuff
Please don't merge pull requests that don't pass our checks.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:42:47 +02:00
Ruslan Kuprieiev 5acdfcde51 run: use REMAINDER for command
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:42:47 +02:00
Ruslan Kuprieiev 7620e14c82 repro: state: rework state format and repro internals
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:42:46 +02:00
Ruslan Kuprieiev 3083604a9c dvc: remove 'import' and replace it with 'add'
Allows us to unify state files and drop another level of
abstraction.

Fixes #304

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:38:23 +02:00
Ruslan Kuprieiev 1fb81447f7 run: add cmdfile
Fixes #299

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:38:23 +02:00
Ruslan Kuprieiev a180be7d66 dvc: use md5 for all deps
Fixes #267

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:38:23 +02:00
Dmitry db625e8213 Use DvcLock from CmdBase class only 2017-11-19 04:14:12 -08:00
Dmitry 1349589ec7 Move CmdBase class to common dir 2017-11-19 03:12:37 -08:00
Dmitry 8d6c134780 Extract cache_dir\state_dir as a CongigI class variable 2017-11-13 00:38:44 -08:00
Ruslan Kuprieiev 4ffac507df dvc: remove data dir
From now on, we're no longer tied to fixed data dir, so we can
track individual data items without forcing user to use separate
dir for them. This way we add much more flexibility for dvc, so
users can use their own project structure.

Fixes #68

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-10-30 05:05:28 +02:00