Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-01 23:08:43 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,504 Commits 7 Branches 579 Tags
8955a606c4acaf3a567daacb2cdf82118da6afbc
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Mr. Outis a17caefd50 integrate @efiop 's suggestions 2019-08-10 11:37:35 -05:00
Mr. Outis 163d872c92 fix tests for windows 2019-08-09 22:46:25 -05:00
Mr. Outis fa04c27580 add tests for recursive operations and file not found 2019-08-09 22:06:32 -05:00
Mr. Outis 0e46edd9bb fail silently but with a more sound error message 
* Add a `_missing` key to the return value of `repo.metrics.show` to
include targets not recognized as metrics or not found.
 2019-08-09 21:55:19 -05:00
Mr. Outis d4ef110794 exit with 1 if targets were not found 2019-08-06 21:10:48 -05:00
Mr. OutisandRuslan Kuprieiev 21a2d3c6ff remove no-op 
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
 2019-08-05 15:30:28 -05:00
Mr. Outis 44f1cc6200 bring back debug message 2019-08-05 15:27:17 -05:00
Mr. OutisandRuslan Kuprieiev dc5dd1e443 change set.update to set.add 
created a set with `{}` but resulted in a dictionary, making me think about `set` not supporting the `add` method.

Thanks for the correction, @efiop !

Co-Authored-By: Ruslan Kuprieiev <[email protected]>
 2019-07-27 17:44:45 -05:00
Mr. Outis 405dfde23f use set instead of list for deduplication 2019-07-27 16:34:22 -05:00
Mr. Outis 9ca5452603 scm: reset tracked files after every operation 
Fix #1475
 2019-07-23 15:10:48 -05:00
Mr. Outis dda8368f20 ssh: increase jobs and parallelize batch_exists 
- SSH number of jobs increased from 4 to 8.

- `batch_exists` now opens all the SFTP sessions available to
distribute the work between them. Number of sessions is restricted
by the `MaxSessions` option configurable by the `sshd_config`.

- Fix #2104

With the optimizations mentioned above, we increased the speed by 8x
(using the default `MaxSessions` configuration -- 10).

Shout-outs to @efiop, @shcheklein, and @dmpetrov for suggesting the
optimization techniques.
 2019-06-19 17:45:59 -05:00
Mr. Outis 223f0f6c0f status: show a progress bar while gathering info 
Fix #1762
Fix #1825
 2019-06-14 16:39:30 -05:00
Mr. Outis d4f0d311fc remote: add other way to perform cache_exists 
Close #1762
 2019-06-11 08:55:48 -05:00
Mr. Outis e3b3f3f371 progress: bug fix with --quiet 
Related to #1377
 2019-05-29 16:45:10 -07:00
Mr. Outis 037f32c8cc docs: add some docstrings about computing status (#2048) 2019-05-23 16:16:02 -07:00
Mr. Outis 7c0c89a688 upgpkg: requests >= 2.22.0 
Close #2008
 2019-05-16 13:28:36 -07:00
Mr. Outis 15f074fd96 refactor: rely on RemoteBasic._changed_dir_cache 
Defining `_changed_cache_dir` for the sole purpose of executing
code before `_changed_dir_cache` was not that clear.

Now, it overwrites `_changed_dir_cache` but reference the
base method.
 2019-05-15 19:02:58 -07:00
Mr. Outis a91b198954 status: optimize the _fill_statuses method 
Close #1975
 2019-05-10 19:20:13 -07:00
Mr. Outis c39c2f087a install: append commands if hook already exists 
Close #1964
 2019-05-10 16:20:01 -07:00
Mr. Outis 9978bc87a4 logger: fix bug that ignored the progress bar 
Close #1973
 2019-05-10 00:17:32 -07:00
Mr. Outis 30e38c91dc tests: use UTF-8 when creating metrics files (#1956) 2019-05-01 20:07:37 -07:00
Mr. Outis 60e83285e1 config: implement bool_schema 💅 
Extracted from https://github.com/iterative/dvc/pull/1804/
 2019-05-01 17:37:47 -07:00
Mr. Outis 29f58bda9b status: optimization for already downloaded files 
Fix #1839
 2019-04-30 17:09:04 -07:00
Mr. Outis d3502ebfe3 add: warn when recursively adding large files 
Close #1758
 2019-04-30 12:33:28 -07:00
Mr. Outis c0104e2e61 repro: implement downstream option (#1926) 
* repro: implement downstream option

Close #1555

* Update scripts/completion/dvc.zsh

Co-Authored-By: mroutis <[email protected]>

* Update dvc/command/repro.py

Co-Authored-By: mroutis <[email protected]>
 2019-04-25 04:50:44 +03:00
Mr. Outis d42c8fbd14 repro: persisting outputs invalidates build cache 
Fix #1852
 2019-04-24 16:45:03 -07:00
Mr. Outis 1cac81430e tests: be paranoic with caplog 2019-04-11 14:09:02 -05:00
Mr. Outis f417f49c6e dvc: deprecate --remove-outs 
Close #1853
 2019-04-10 18:58:32 -05:00
Mr. Outis 059aef3ef0 logger: use exception instead of error 
The problem was happening because `logger.error` doesn't include any
`exc_info`, and the `parse_exc` logic was previously to rely on the log
record's `exc_info` instead of the global stack trace.

There's still use cases when you want to just `log` an error without
printing the stack trace, those are left untouched.
You can search for them with: `grep -R 'logger.error' dvc/`

---

Fix #1869
 2019-04-10 16:26:14 -05:00
Mr. Outis 75e16cdc09 version: show when the release version is dirty 
- Silent stderr when calling `git describe` (fix #1866)
 2019-04-10 13:14:23 -05:00
Mr. Outis a90a3beba0 logger: don not log without a message 
Close #1858
 2019-04-09 14:59:10 -05:00
Mr. Outis d355b1e66e config: allow cross-referenced remotes 
Close #1614
 2019-04-08 22:10:03 -05:00
Mr. Outis 727bfd955e version: privatize version.py methods and vars 2019-04-07 16:48:34 -05:00
Mr. Outis 921b39dbfe tests: use pytest caplog instead of custom mock 
Close #1753
 2019-04-06 20:35:30 -06:00
Mr. Outis 85cd344152 logger: Use a more _pythonic_ logging 
- Uses a formatter for coloring output
- Move the `progress_aware` decorator logic to the logger itself
- Refactor modules so each one uses their own instance of a logger

Close #1753
 2019-04-06 18:19:32 -06:00
Mr. Outis b4d0d37ac8 dvc: refactor package versioning logic 
- Use `version.py` to compute a dynamic version stored in `__version__`.
  Version information was scattered across the source code
  (`VERSION_BASE`, `VERSION`, `__version__`, and `version.py`).

- Read `__version__` from `setup.py` and **pin** it using custom
  `build_py` command class.
  This way, we can maintain consistancy between the build version
  and the runtime one.
  Also, this gives us the ability to run `setup.py` without depending on
  on `dvc` itself.

Close #1824
 2019-04-06 18:06:11 -06:00
Mr. Outis 298770efe9 metrics: shows formatted multiline metrics 
Close #1716
 2019-03-29 16:24:54 -07:00
Mr. Outis 1df5dea351 cache: provide at least one value in parser 
Fix #1747
 2019-03-21 12:28:46 -07:00
Mr. Outis ce65771596 scripts: update completion for zsh and bash 
Fix #1732
 2019-03-21 09:28:54 -07:00
Mr. Outis e4bc59ca03 cli: Use correct color codes on Windows 
Fix #1698
 2019-03-16 17:51:28 -06:00
Mr. Outis 21778f6719 stage: convert wdir path to posix 
Fix #1721
 2019-03-14 14:40:28 -06:00
Mr. Outis b319e18510 add: support the --file option 
Fix #1356
 2019-03-06 16:18:29 -06:00
Mr. Outis 599ee58a5f stage: always reproduce when --force is given 
Fix #1667
 2019-03-05 22:36:23 -06:00
Mr. Outis 1c850af12d remote: upload/download to a temp file 
Fix #1674
 2019-03-02 10:26:13 -06:00
Mr. Outis 93920baaec logger: Format progress bar interruption 
Fix #1602
 2019-02-27 21:24:22 -06:00
Mr. Outis 600514cba9 ssh: upload to a tmp file to prevent data corruption 
Fix #1570
 2019-02-22 10:40:55 -06:00
Mr. Outis 5cfcc956db status: imply --cloud when a remote is given 
Fix #1617
 2019-02-21 18:32:48 -06:00
Mr. Outis d640859fc4 stage: always reproduce callback stages 
Fix #1634
 2019-02-21 15:56:38 -06:00
Mr. Outis d32705f01b py2: raise OSErrors on makedirs 2019-02-21 11:34:09 -06:00
Mr. Outis 758c71e249 move: awareness of cwd when moving files 
fix #1636
 2019-02-19 11:55:43 -06:00
Mr. Outis 5e7a152b53 ssh: use sftp when possible 
Related #830 [1/2]
 2019-02-18 19:03:06 -06:00
Mr. Outis ec77bfa1af remote: define command error on the base 
fix: #1484
 2019-02-09 12:03:17 -06:00
Mr. Outis 0d51d10ccf remote: notify when operation is not supported 
close #1591
 2019-02-08 19:54:54 -06:00
Mr. Outis d6b5729105 status: update --quiet help message 2019-02-08 00:48:42 -06:00
Mr. Outis 2be7232d1a py2: Add unicode support for open and os.walk 2019-02-07 23:33:30 -06:00
Mr. Outis 2c354cffe0 style: Add flake8-docstrings 2019-02-04 17:07:59 -06:00
Mr. Outis 3009a25ae2 remote http: use Content-MD5 header if there is not ETag available 
Fix #1412
 2019-01-28 19:02:41 -06:00
Mr. Outis 4218f85853 status: --quiet reflects changes on its exit code 
Fix #1519
 2019-01-28 11:28:44 -06:00
Mr. Outis 0d46921ec7 py2: add unicode literals 
Fix #1512
 2019-01-27 20:40:21 -06:00
Mr. Outis 185600d344 project: Detect circular dependencies on graph 
Fix #1285
 2019-01-22 18:39:56 -06:00
Mr. Outis 823077aa2b stage: Checkout cached output when possible 
Fix #1348
 2019-01-19 00:05:25 -06:00
Mr. Outis e600e3a736 remote: Use checksum instead of MD5/ETag 2019-01-19 00:02:28 -06:00
Mr. Outis d768607733 💅 Fix some typos and aesthetic changes (#1508) 2019-01-18 23:47:27 -06:00
Mr. Outis 0372e1d87d move: Take into account suffix 
🔨 Refactor

Fix #1491
 2019-01-11 12:57:42 -06:00
Mr. Outis 7be42c8402 run: Parse and quote correctly the command 
Fix: #1478
 2019-01-09 00:01:55 -06:00
Mr. Outis 3ddc54403d init: Boxify message about analytics for more visibility 2018-12-28 13:59:28 -06:00
Mr. Outis 2d21c4a6e8 init: Do not show "now commit" message with --no-scm 2018-12-28 13:59:03 -06:00
Mr. Outis 4c420d5f4d logger [rfc]: Use a module instead of a class 
- Remove deprecation warnings
- Format error output and remove duplicated info

Close #1086
 2018-12-23 21:06:39 -06:00
Mr. Outis e8a081030f repro: Fix --all-pipelines option 
Fix #1435
 2018-12-16 18:31:40 -06:00
Mr. Outis 4142f65531 Add tests for pipelines 2018-12-16 18:06:37 -06:00
Mr. Outis 948021f5b0 rfc: Prompt as functions instead of a class 2018-12-14 10:47:18 -06:00
Mr. Outis df044c60ac rfc: Move welcome message to a private function 2018-12-14 10:45:58 -06:00
Mr. Outis cb142f9f0e project: Refactor pipeline 
What's a pipeline if not the weekly connected component subgraphs of a
directed acyclic graph?
 2018-12-13 13:37:45 -06:00
Mr. Outis 21f19b1296 fix logger issue 2018-12-11 15:06:09 -06:00
Mr. Outis a1ecbdd001 Merge branch 'master' into fix-1255 2018-12-11 14:56:05 -06:00
Mr. Outis 9971f7a624 run: Prevent argument duplication with deps/outs 
Fix #1255
Fix #1168
Close #1259

RFC:
- Add / fix docstrings and privatize methods
- Add `__repr__` method for core objects
- Change the name of methods to make them more readable
 2018-12-11 14:54:45 -06:00
Mr. Outis d1009a7489 remote s3: Add option to disable the use of SSL 2018-12-11 14:24:43 -06:00
Mr. Outis 19007ff081 scripts: Add file completion to dvc add command (#1423) 2018-12-07 22:39:24 -06:00
Mr. Outis b3b2c97f4b logger: Prevent adding multiple handlers 
Fix #1365
 2018-12-04 22:33:06 -06:00
Mr. Outis 22d38cda9f run: Check CWD specified as output in any stage (#1375) 
* run: Check CWD specified as output in any stage

Fix #1123

* Fix some typo

Co-Authored-By: mroutis <[email protected]>

* normalize paths and make stage an obligatory argument

* Pass stages() to _check_output_duplication

* Update dvc/project.py

Co-Authored-By: mroutis <[email protected]>
 2018-11-29 21:03:10 +02:00
Mr. Outis 44e67d4cdc progress: Respect the --quiet option (#1377) 
* progress: Respect the --quiet option

Fix #1308

* Use a dynamic import to prevent circular dependency

* test: add test for quiet mode

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-11-29 08:10:43 +02:00
Mr. Outis 38d666d77d import: Output as on optional argument (#1351) (#1378) 
* import: Output as on optional argument

Close #1335

* Use tempfile to prevent circular dependencies
 2018-11-28 23:35:37 +02:00
Mr. Outis c6b8be8f8a import: Output as on optional argument (#1351) 
* import: Output as on optional argument

Close #1335

* Use tempfile to prevent circular dependencies
 2018-11-28 20:00:58 +02:00
Mr. Outis 6d54a114ae local: Don't prompt at removal if file is already cached (#1367) 
* local: Don't prompt at removal if file is already cached

Fix #1366

* [fix] Do `remove` at `save_remove`

* Use `state.update` to compute checksum instead of `save_info`

* PEP8 compilant
 2018-11-26 21:25:54 +02:00
Mr. Outis 1e3d752917 remote: HTTP implemented (support only for downloading) (#1350) 
* remote: HTTP implemented (support only for downloading)

* Remove duplicated content in the schema

* Chomp trailing spaces in test_remote

* remote http: Implement pull/fetch

* add some tests

* Update dvc/remote/http.py

Co-Authored-By: mroutis <[email protected]>

* Update dvc/remote/http.py

Co-Authored-By: mroutis <[email protected]>

* correct test

* rehprase error message

* add __init__ for utils

* try to fix windows test
 2018-11-24 21:22:05 +02:00
Mr. Outis 43e7fec5d3 skip verification if file doesn not exist 2018-11-24 11:57:19 -06:00
Mr. Outis 02e589a25c init: Add a welcome message 
Close #1184
 2018-11-19 23:09:51 -06:00
Mr. Outis 3910eb6065 metrics add: Raise exception when path does not exist 
Fix #1316
 2018-11-19 20:52:28 -06:00
Mr. Outis f26c1fffef pull: Add --force flag that permeates down to checkout 2018-11-18 09:27:42 -06:00
Mr. Outis 1ff5e2c163 stage: When loading an empty Dvcfile default to dict() 
Fix #1339
 2018-11-17 20:24:59 -06:00
Mr. Outis 2906a31b66 tests: dvc checkout --force 2018-11-17 19:47:50 -06:00
Mr. Outis e57480eb46 Always perfom a safe remove on checkout 2018-11-17 19:36:47 -06:00
Mr. Outis d406361bd3 checkout: Prompt when deleting files from working dir 
Close #1328
 2018-11-17 19:36:47 -06:00
Mr. Outis b4588671a0 Add a test for it 2018-11-14 11:03:53 -06:00
Mr. Outis 423d620b89 checkout: Clean working directory changes of tracked dir 
Fix #1326
 2018-11-14 10:53:02 -06:00
Mr. Outis 88ed542b94 gitignore: Add temporary SQLite files (#1322) 
Fix #1319
 2018-11-13 19:29:21 -06:00
Mr. Outis 55eadba0ec cache: Cosmetic changes 💅 (#1321) 2018-11-13 13:30:34 -06:00
Mr. Outis fc7545b6cf checkout: Remove selectively on directory files when changed (#1320) 
Fix: #1220
 2018-11-13 13:30:19 -06:00
Mr. Outis 4d03f69a86 remote: Show progress bar when linking large dirs (#1309) 
* remote: Show progress bar when linking large dirs

Close #1222

* Fix pycodestyle

* Use Logger.debug instead of local verbose argument

* Show the progress bar also during checkout
 2018-11-10 23:56:50 +02:00
Mr. Outis 4ee615e5b4 Prefer using ENV_VARS before dvc config file 2018-11-07 19:11:18 -06:00