* plots: allow dir plots
* first scenario
* separate dvclive tests
* rename index to step
* another test
* refactor
* its ok to have step in data
* sort plots
* introduce --logs
* add logs to run commands
* variable name refactoring
* fix logs option
* fixup
* cmd: run: fix unit tests
* plots: modify: fix tests
* improve dir handling
* fixup
* dvclive: logs summary
* api: introduce logs summarization method
* handle no repo case
* fix granular dir plots
* api: dvclive: simplify
* dvclive: roll back logs flag
* cmd: introduce logs
* api: adjust to logs
* test: dvclive fix
* rename logs to dvclive
* live: command: add uninitialized
* outs: dvclive type
* dvclive: support providing config via evn variables
* dvclive: treat as metrics
* dvclive integration: add support for summary on/off
* dvc integration: add temporary test for configuration export
* remove requirements
* cmd: introduce logs
* rename logs to dvclive
* cmd: introduce logs
* remove logs leftowvers
* dvclive: env: adjust to chekpoints env variables
* command: live: remove unnecessary help
* fix linter suggestions
* type hinting
* review refactor
* replace dvclive with live
* stage: loading: move live loading to separate method
* stage: serialize: allow only single live output
* live: add show/diff
* stage: serialize: roll back single live enforcing
* fixup
* rebase fixups
* output: live: force single instance
* live: command: add unit tests
* Update dvc/output/base.py
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Update dvc/stage/__init__.py
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Update dvc/stage/serialize.py
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* review v2
* live: fix summary tracking by experiments
* naming fixups
* api: live: add tests
* tests: api: separate live tests
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Imports are no longer moved to top.
* `seed-isort-config` is no longer needed.
* Has a `black`-compatible profile/config.
* Also `isorts` the imports anywhere in the code
(not just at the top).
Also bumps `restyled-isort` to newest version
This PR is just a showcase to see how we like it. I've converted a few
tests to it, but there will be much more to come.
```
assert (tmp_dir / "file") == "contents"
assert (tmp_dir / "dir").read_text() == {"subdir": {"file": "hello"}}
```
This approach is nicer because it allows us to use a declaratory
asserts instead of relying on fs files. The problem becomes much more
apparent once you get into remote workspaces, where we `trees_equal`
doesn't work.
* remote.http: support _upload() via HTTP POST
- uploaded files are sent as chunked encoding POST data
Fixes#3247
* remote.http: add support for HTTP basic and digest auth
- username, password, ask_password options work the same way as for
ssh remotes
- basic_auth (bool) and digest_auth (bool) options added for http(s)
remotes
- digest_auth takes precedence over basic_auth if both are enabled
* remote.http: auth_method() unit tests
* tests: Add functional tests for HTTP remotes
- HTTP remotes now tested locally using a SimpleHTTPServer instance that
allows reading/writing to a temp directory
* tests: fix http tests on py3.5, 3.6
* remote.http: add custom auth method
* s/custom_header/custom_auth_header/
* Apply suggestions from code review
Co-Authored-By: Saugat Pachhai <[email protected]>
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Saugat Pachhai <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
This employs generating temporary signed public url and then streaming
from it. The bulk of the code might be reused for other remotes.
Url streaming does resume on connection error using "Range" header and
regenerates public url too.
If remote class doesn't provide .open() then this falls back to pulling
a file and opening it locally.
* state: use ignore file handler when accessing metadata
* fs: don't add dir size to overall size for dir
* ignore: use posix-like path on windows
* ignore: remove tree from dvc_walk arguments
* ignore: handle DvcIgnoreFilter creation outside of DVC repo
* ignore: throw on dvcignore file under output dir, but not dependency dir
* ignore: move ignore initialization to Repo init
* ignore: work in progress
* ignore: work in progress 2
* ignore: throw on .dvcignore upon collecting dir
* ignore: refactor
* ignore: loading upper level ignores in stages()
* Update dvc/repo/__init__.py
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* dvcignore review refactor
* test: remove never used copy of load_stage_file()
* dvc: save comments and custom formatting in stage files
* Add a couple of TODOs
* test: test repro doesn't clear comments and formatting in .dvc
As part of this I started writing pytest-like tests and added commonly
used fixtures.
* test: simplify and prettify dvc formatting retained test
Also test that line comments after md5 fields are retained.
* test: fix tests
* dvc: fix stage and metrics opens not closed
* dvc: always use utf-8 in stage files
It used to encode in system default encoding. Not sure if PyYAML was
handling this silently or we were just lucky all the way.
* test: unit test apply_diff()
* test: remove TODOs and ASK
* dvc: polish dvc-formatting PR
* dvc: move stage load/dump utils to a new stage utils file
* dvc: clean up apply_diff()
* test: test stage md5 ignores comments
* test: remove pytests starting marker
Paweł RedzyńskiandRuslan Kuprieiev
Saugat Pachhai
Casper da Costa-Luis
Fábio Santos
Mr. Outis
Ruslan Kuprieiev
Alexander Schepanovski
Andrew Grigorev
Dmitry