Commit Graph
35 Commits
Author SHA1 Message Date
Mr. Outis 94a0914bba py3: get rid of future imports 2019-12-30 19:35:48 -06:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Casper da Costa-Luis 9ce3b0c50c add: output improvements (#2546)
* some doc improvements

* add: intermediate progress

* update after rebase

* linting

* fix checksum progress cleanup by ensuring gc

* progress-aware logger.info

* re-fix checksum pbar closure

* multi-file add

* make stage.dump() less verbose.

TODO: check all instances of stage.dump and possibly make verbose

* decrease verbosity of checksum calculations

* leave dvc.add stats

* update/fix logger tests

* minor doc reword

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* tidy API target_;list => targets

- as per https://github.com/iterative/dvc/pull/2546#discussion_r337042535

* tidy RecursiveAddingWhileUsingFilename message duplication

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337044670

* minor logging uncomment

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337051097

* minor doc improvements

- https://github.com/iterative/dvc/pull/2546#discussion_r337116255
- https://github.com/iterative/dvc/pull/2546#discussion_r337116809

* tidy exception

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337434142

* minor doc update

- fixes https://github.com/iterative/dvc/pull/2546/files#r337294491
- related to https://github.com/iterative/dvc.org/issues/719

* Adding => add

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337037137

* various review fixes

- minor comment as per
  https://github.com/iterative/dvc/pull/2546#discussion_r337927837
- revert warning style fix
  https://github.com/iterative/dvc/pull/2546#discussion_r338046035
- Abstract error consistency
  https://github.com/iterative/dvc/pull/2546#discussion_r337931070
- minor comment as per
  https://github.com/iterative/dvc/pull/2546#discussion_r338254958

* remove diabling progress option

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r338314050
2019-10-25 01:09:19 +03:00
Jorge Orpinel 936f941bf0 help: update usage for commands that use the targets argument 2019-06-26 23:25:58 -05:00
Jorge Orpinel 3dffe8e5c0 dvc: i/DVC file[s]?/ -> /DVC-file($1)/
Standardizes way to refer to DVC-file(s)

(Related to #2102)
2019-06-07 03:33:20 -05:00
vyloy 01a28f2de9 import: improve some details about the '--file' option. 2019-04-18 08:56:14 +08:00
Mr. Outis 85cd344152 logger: Use a more _pythonic_ logging
- Uses a formatter for coloring output
- Move the `progress_aware` decorator logic to the logger itself
- Refactor modules so each one uses their own instance of a logger

Close #1753
2019-04-06 18:19:32 -06:00
Ivan Shcheklein 4a9188c0b5 Move man doc links to descriptions, fix some help copies (#1805)
* move man doc links to descriptions, fix some help copies

* minor init force message fix

* deal with ugly inheritance in lock base

* remove unnecessary super calls

* add return back to python lock/unlock run functions
2019-03-29 03:56:41 +02:00
Lee Yi Jie Joel 2ba657c77a Add links to documentation for all commands (#1767) 2019-03-22 20:32:06 +02:00
Mr. Outis b319e18510 add: support the --file option
Fix #1356
2019-03-06 16:18:29 -06:00
Ruslan Kuprieiev 99f4107d58 add: introduce --no-commit
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-02-16 21:18:54 +02:00
Ruslan Kuprieiev ba8d83e9b1 dvc: refactor dvc project class
1) split Project's methods into separate files;
2) rename Project into Repo(similar to gitpython's Repo)

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-02-13 03:31:23 +02:00
Ruslan Kuprieiev 4896f08ddf dvc: comply with both flake8 and black
flake8 provides some additional features like unused vars check, etc, so
it makes total sense to use it as well.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-02-04 00:10:36 +02:00
Ruslan Kuprieiev 4069e34130 dvc: use black instead of flake8
Fixes #1546

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-01-30 03:48:36 +02:00
Mr. Outis 0d46921ec7 py2: add unicode literals
Fix #1512
2019-01-27 20:40:21 -06:00
Ruslan Kuprieiev 6fec8b16cd RFC: conform to pylint
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-01-16 14:12:19 +02:00
Mr. Outis 4c420d5f4d logger [rfc]: Use a module instead of a class
- Remove deprecation warnings
- Format error output and remove duplicated info

Close #1086
2018-12-23 21:06:39 -06:00
Ruslan Kuprieiev e02a33f934 logger: shortcut logger in quiet and normal modes
In scenarios with millions of files, late shorcutting of
debug msgs results in massive overhead. In order to avoid
that, just shortcut logger messages ASAP if verbose mode
is not enabled.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-12-11 12:07:28 +02:00
Ruslan Kuprieiev c6f642fd6b dvc: add --recursive
Fixes #1026

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-21 10:41:19 +03:00
Ruslan Kuprieiev 0e9774a3f8 RFC: follow flake8 rules
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-25 17:28:58 +03:00
Ruslan Kuprieiev 2cab8e03be command: remove unused code
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-19 07:00:39 +03:00
Ruslan Kuprieiev db5cb125fb add: fix error msg
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-03 01:23:03 +03:00
Ruslan Kuprieiev 14fdc18948 dvc: chain exceptions
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-06 18:38:29 +02:00
Ruslan Kuprieiev aa396d1b42 add/remove: catch and report errors
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-07 16:11:24 +02:00
Ruslan Kuprieiev a779424a36 WIP RFC
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-23 09:37:40 +02:00
Dmitry e9b50093e9 Keep dvc files i the currend dir by default 2017-12-16 14:03:13 -08:00
Ruslan Kuprieiev 927d58cfad dvc: introduce stage files instead of state files
These describe single stage of pipeline and are allowed to
be placed anywhere in current or parent directories(up to
project's root) of the data file.

By default, 'dvc run' writes stage file to ./stage.dvc, but
it can be changed by using '-f' option(see functest/test-repro.py).

For now we're traversing project tree to find needed stage files for
specific data files. This is very wasteful and not optimal, but will
do for now, as this patch is aimed at stabilizing design of our repo
files.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-07 22:52:18 +02:00
Ruslan Kuprieiev 0b639a9c09 state: rework state format
Dependencies are now specified in a more natural way:

deps:
  path/to/file1: <md5>
  path/to/file2: <md5>

And outputs also have md5 sum, that allows us to track
them by it and not by separate md5 field in state file,
which will allow us to remove strict data <-> data.state
relation in the future.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-07 01:58:13 +02:00
Ruslan Kuprieiev bbe5e46b5c dvc: remove CommandFile
As we're moving to more user friendly single *.dvc file approach, separate
command files become uneeded.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-05 20:56:17 +02:00
Ruslan Kuprieiev e853fc76aa dvc: get rid of state dir and cache_state in favor of single .dvc
From now on everything is stored in filename + .dvc path, except for local_state.
This allows us to be more explicit and help users see everything they need in relation
to version control of their own data files.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-12-05 20:33:21 +02:00
Dmitry 40660b5120 Refactoring: extract CommandFile class 2017-11-26 21:49:06 -08:00
Dmitry 50f736d9d9 dvc run: regular file output. Close #314 2017-11-26 21:40:55 -08:00
Ruslan Kuprieiev be159e7798 dvc: fix misc dvc stuff
Please don't merge pull requests that don't pass our checks.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:42:47 +02:00
Ruslan Kuprieiev 7620e14c82 repro: state: rework state format and repro internals
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:42:46 +02:00
Ruslan Kuprieiev 3083604a9c dvc: remove 'import' and replace it with 'add'
Allows us to unify state files and drop another level of
abstraction.

Fixes #304

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2017-11-26 13:38:23 +02:00