Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
5,767 Commits 7 Branches 579 Tags
fcca6361edefbf453a7271a1cc61b4d980c5a9ae
Commit Graph
5767 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
karajan1001andSaugat Pachhai fcca6361ed Outputs as target supporting for dvc status (#4433) 
* Outputs as target supporting for `dvc status`

fix #4191
1. Add a related test which would fail on current version.

* Pass the tests

1. add deps to the tests.
2. make change to pass the tests.

* Update tests/func/test_status.py

Co-authored-by: Saugat Pachhai <[email protected]>

* Solve some change request.

Co-authored-by: Saugat Pachhai <[email protected]>
 2020-08-25 13:51:27 +02:00
Peter Rowlands (변기호) 714503a9e8 experiments: only show queued experiments for the current commit (#4462) 2020-08-25 14:33:46 +03:00
Peter Rowlands (변기호) 01a6580b3d experiments: fix inconsistent dvc exp checkout behavior (#4461) 
* experiments: stage workspace changes before using stash apply

* add test for dvc exp checkout
 2020-08-25 14:33:14 +03:00
Ruslan Kuprieiev 026c4fa47d dvc: bump to 1.6.1 1.6.1 2020-08-25 02:03:35 +03:00
Ruslan Kuprieiev 6c352eeaf0 hdfs: implement isdir and isfile (#4460) 
Fixes #4332
 2020-08-25 02:02:33 +03:00
Peter Rowlands (변기호) f789cdff89 experiments: fix bug where wrong git branch checked-out before pull (#4450) 2020-08-24 15:47:25 +03:00
Peter Rowlands (변기호) bfb98e9288 experiments: support binary and untracked files when generating (#4452) 
experiment patches
 2020-08-24 15:19:57 +03:00
Paweł Redzyński a82d6e4173 metrics: don't raise on dir during collection (#4443) 2020-08-24 02:55:20 +03:00
Puneetha Pai 05c1e76091 diff command: output in markdown format with hash (#4435) 
* diff: Add tests to hash formatting function

* diff: Add test for show_md with hash

* diff: Add stub test for _show_md from command
 2020-08-24 02:25:09 +03:00
Saugat Pachhai adce620621 add json serialization utils (#4441) 
* add json serialization utils

* Update dvc/utils/serialize/_json.py
 2020-08-21 17:48:26 +03:00
Saugat Pachhai 1c14a7e202 metadata() on RepoTree (#4434) 
* WIP: metadata api on RepoTree

* Make outs optional

* add tests + misc. changes

* reorder tests and fields
 2020-08-21 17:34:35 +03:00
Peter Rowlands (변기호) dac17e247d experiments: abstract executor tmpdir population using tree upload (#4432) 
* executor: abstract tmpdir population using tree upload

* experiments: unmark flaky test

* fix wrong tree download

* LocalTree: use os.replace instead of os.rename in _upload
 2020-08-21 17:32:39 +03:00
Ruslan Kuprieiev d89c16dfe8 deps: freeze pylint at 2.5.3 2020-08-21 17:31:36 +03:00
Peter Rowlands (변기호) a2a6f8bd58 tree: add overwrite parameter in tree.upload (#4442) 
* tree: add overwrite flag for tree.upload

* tests: update tests

* fix test file permissions

* default to overwrite on upload
 2020-08-21 17:30:27 +03:00
Saugat Pachhai 1d196b4df6 Modify yaml/toml using contextmanager, removing boilerplate (#4426) 2020-08-21 04:47:58 +03:00
Saugat Pachhai 2f2e7121eb Enable profiling from cli (#4437) 2020-08-20 23:22:39 +03:00
Saugat Pachhai de0fe94cd0 Make serializing utils use tree, misc changes (#4425) 
* Also fixes plots not getting dumped on flow style
* Make serialization utils aware of trees
* Directly load yamls/tomls using utils, rather than using parser
 2020-08-20 20:38:17 +03:00
Saugat Pachhai 5d1ad03fe9 git: do not ask for gpg signoff on merge (#4430) 2020-08-20 13:47:19 +03:00
Saugat Pachhai 9f1bd98fcc Mark test_new_simple as flaky (#4429) 
It is for short period of time till it gets resolved
See: #4401
 2020-08-20 10:37:58 +05:45
Ruslan Kuprieiev 41e6c22e7d dvc: move Dvc/RepoTree to dvc/tree (#4427) 
This is where it really belongs.
 2020-08-19 22:30:55 +03:00
Ruslan Kuprieiev b82104a608 dvc: bump to 1.6.0 1.6.0 2020-08-19 21:51:20 +03:00
Saugat Pachhai 14745039a4 Move to yaml1.2 (#4415) 
* use yaml 1.2 everywhere

This commit:
* gets rid of pyyaml
* uses ruamel everywhere
* tries to unite toml and yaml

However, this will require a few more changes that will come soonish.
Eg: Current `parse_yaml` and `parse_toml` behavior could be error-prone,
so, it's better to introduce `tree` inside these serialization methods
(which I will try to tackle next).

Another is, yaml load can be slow for json. We could speed it up just by
using `json.load(s)` here (also in upcoming patch).

* Fix tests
 2020-08-19 17:56:38 +03:00
Peter Rowlands (변기호) 958aa265e7 experiments: abstract executor output collection using trees (#4402) 
* experiments: use RepoTree for executor output collection

* experiments: use tree.download for output collection

* cleanup docstring

* use string name for tqdm
 2020-08-19 17:56:00 +03:00
Ruslan Kuprieiev ac24b5c6f0 dvc: introduce merge-driver (#4298) 
Related to #4162
 2020-08-19 17:55:12 +03:00
Peter Rowlands (변기호) 5b352491d4 git: handle rev_parse exception (#4422) 2020-08-19 21:21:16 +09:00
Ruslan Kuprieiev 32b5b33329 dvc: bump to 1.5.1 1.5.1 2020-08-19 00:10:06 +03:00
Peter Rowlands (변기호) 4aecb12988 experiments: fix flaky tests (#4414) 
* experiments: lock exp workspace when using git

* remove timestamp from flaky exp show test
 2020-08-18 19:58:30 +03:00
Paweł Redzyński ef944e6509 http: replace HEAD with streamed GET (#4390) 2020-08-18 12:45:06 +03:00
Saugat Pachhai e31364979b Move toml/yaml serialization to dvc.utils.serialize (#4411) 2020-08-18 13:01:16 +05:45
nik123 8cd4eeac18 status: check if target is missing both in cache and remote (#4398) 
* status: check if target is missing both in cache and remote

* refactor: additional comments
 2020-08-17 20:08:10 +03:00
Paweł Redzyński 6ae0be2179 metrics: show: debug info on parsing problems (#4370) 2020-08-17 17:37:15 +03:00
Peter Rowlands (변기호) 70ddb7c931 experiments: include experiment timestamps in dvc exp show output (#4389) 
* experiments: include experiment commit times in show

* --no-timestamp can be used to hide timestamps

* update tests

* clean up kwargs
 2020-08-17 17:35:43 +03:00
Saugat Pachhai c4bc4c83e8 upgrade to isort5 (#4399) 
* Imports are no longer moved to top.
* `seed-isort-config` is no longer needed.
* Has a `black`-compatible profile/config.
* Also `isorts` the imports anywhere in the code
  (not just at the top).

Also bumps `restyled-isort` to newest version
 2020-08-17 19:59:40 +05:45
Saugat Pachhai 31e9cc2e40 cleanup RepoTree (#4405) 
1. Rename `_get_tree_pairs` to `_get_tree_pair`.
2. Add comments, docstrings and type-annotations.
3. Rename variable: `_discovered_subrepos` to `_subrepos_trie`.
4. Merge some nested ifs.
 2020-08-17 19:42:45 +05:45
karajan1001 b8e40f5091 Solve check-ignore broken in a Linux pipeline (#4409) 
fix #4407
1. Update test, now it can simulate a pipeline situation.
2. Solve remove `isatty` check in `dvc check-ignore --stdin` command.
 2020-08-17 16:17:45 +03:00
Saugat Pachhai 1cfccef22e update flake8 to latest version (#4403) 2020-08-17 14:26:10 +05:45
nik123 44392941af ui: '-j' option now mentions docs (#4382) 
* ui: '-j' option now mentions docs

* ui: copied '-j' default value desc. from cmd ref
 2020-08-15 21:41:42 +03:00
Ruslan Kuprieiev 023dec4aa6 tree: make get_hash return type and hash pair (#4397) 
Currently we kinda assume that whatever is returned by `get_file_hash`
is of type self.PARAM_CHECKSUM, which is not actually true. E.g. for
http it might return `etag` or `md5`, but we don't distinguish between
those and call both `etag`. This is becoming more relevant for dir
hashes that are computed a few different ways (e.g. in-memory md5 or
upload to remote and get etag for the dir file).

Prerequisite for #4144 and #3069
 2020-08-15 20:35:39 +03:00
Saugat Pachhai b773ba73cf tests: add .dvcignore on the tests expectations (#4396) 
Since #4229, we have started creating .dvcignore file after `dvc init`
This commit only adds that in assertions
 2020-08-15 21:24:29 +05:45
Saugat Pachhai 0b1d4f10d6 RepoTree: add support for subrepo traversal (#4381) 
Right now, no actual "functionality" is surfaced as
`subrepos` is always set to False by default for now.
 2020-08-15 17:27:00 +03:00
Christoph Berganski 8b41655894 Derive WebDAVURLInfo from URLInfo instead of HTTPURLInfo (#4391) 
Unused, empty http extra parts (params,query,fragment) cause problems
with pushing/pulling - lead to violation of the assumption where to look
for hash values in file/path names.

Context:
    https://discordapp.com/channels/485586884165107732/485596304961962003/743454659733094520
    https://discordapp.com/channels/485586884165107732/485596304961962003/743497749290287215
    https://discordapp.com/channels/485586884165107732/485596304961962003/743571921622401136
 2020-08-14 19:46:03 +03:00
Paweł Redzyński 35f125f3df outputs: validate that output is not ignored (#4386) 2020-08-14 14:30:12 +03:00
Robert Lowry 00925029b4 exception text: link to troubleshooting page (#4339) 2020-08-13 13:09:58 +03:00
Paweł Redzyński 9c772f5f6c get-url: allow weak etags (#4372) 2020-08-13 12:03:47 +03:00
karajan1001 e3cfa7348c Result inconsistency of the dvcignore interfaces (#4378) 
Fix #4333, as `is_ignored_file` and `is_ignored_dir` and `__call__`
share the same call `DvcIgnorePatterns.matches`, we only need to focus
on the out of repo situation.
1. Add tests for the `is_ignored_file` and `is_ignored_dir` for the out
of repo condition.
2. Now `is_ignored_file` and `is_ignored_dir` returns false for those
files outside the repo.
 2020-08-13 01:22:03 +03:00
Peter Rowlands (변기호) 9b5740555c experiments: add options to include/exclude columns in exp show (#4375) 
* experiments: add options to include/exclude specific metrics/params columns in
`dvc exp show`

* allow param/metric file to be provided to keep consistency with other
dvc commands

* match nested params/metrics names
 2020-08-13 01:21:12 +03:00
Jorge OrpinelandRuslan Kuprieiev ce9205a3cc term: review "URL" term usage in remote cmds, and get/import-url (#4364) 
* term: review upper case URL usage, and remote add url arg desc

* get/imp-url: better url arg help out

* Update dvc/command/get_url.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-08-13 01:12:45 +03:00
AnatolyandRuslan Kuprieiev 5cb36fc983 Special dir warning #3436 (#4229) 
* add special folder warning

* add special folder warning

* remove special folder warning, start working on default dvcignore

* add default .dvcignote and templates

* remove dvcignore templates

* change text and add tests

* refactor

* dvcignore: ignore comments

* adjust tests

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-08-13 01:03:34 +03:00
Ruslan Kuprieiev 64f038fda8 dvc: bump to 1.5.0 1.5.0 2020-08-12 19:08:38 +03:00
Ruslan Kuprieiev a8f0c5241c dvc: migrate to azure-storage-blob >= 12.0 (#4379) 
There has been some pretty drastic changes in 12.0, so migration
is a little bit more involved.

Fixes #3546
 2020-08-12 19:07:28 +03:00