This permites URLInfo construction from parts without building/reparsing
string, which makes it around 20% faster. Code is also simpler and a bit
shorter now.
Other improvements:
- added URLInfo.replace(path=...)
- URLInfo._path now stringifies properly
- protected against passing path not starting with /
- do not allow query, params, fragments and passwords in urls (they were
silently ignored previously)
Change messages output after return of some commands:
* Remove warning for 'dvc checkout' & commands calling it under the hood
* Add an informational message output for 'dvc pull' command
Fixes#2685.
Default chunk size for gs is 100M, which might be too much for
particular network. So if we are getting ConnectionError, we
should try lowering the chunk size until we reach the minimum
allowed chunk size of 256K. Also note that `chunk_size` must be a
multiple of 256K per the API specification.
Fixes#2572
Hi team,
I just [noticed](https://github.com/iterative/dvc/pull/2548#issuecomment-536255251) that there are could be quite a number of false-positives in `dvc/utils/compat.py` because of, well, the things we have to do for compatibility. I would recommend adding that file to `exclude_patterns` to prevent the noise. I've proposed that in this PR.
We've seen some other projects report this as well, and we're working on proactively suggesting this via the product UI to make it easier.
Hope this helps!
Best,
Sanket
Founder, DeepSource
* s3: list_objects -> list_objects_api
* s3: use path_info for _list_paths
* s3: separate _list_paths from _list_objects
* s3: support max items for listing objects/paths
* s3: take into account directories on `exists`
* s3: overwrite `isdir` method
* s3: custom collect directory entry for s3
* s3: strip checksum when collecting directories
* remote: change walk implementation to walk_files
* s3: add comment about isdir logic
* s3: use `dir_path` correctly on `isdir`
* tests: add a few unit tests for S3
* s3: use path_info join instead of posixpath
* tests: use parametrize for s3 tests
* remote: use posixpath accordingly
* tests: remove undeeded AWS variables from S3 test
* tests: rearrange parametrize values for readability
* tests: add encoding, use a single s3 instance, for loop instead of parameterize
* py2: fix encoding
We are incorrectly using `simple_cycle` here, `find_cycle` is much more
suitable for this application and much faster too. For example:
```
$ cat test_networkx.py
import networkx as nx
from dvc.repo.graph import check_acyclic
G = nx.scale_free_graph(1200)
list(nx.find_cycle(G, orientation='ignore'))
list(nx.simple_cycles(G))
$ python -mcProfile -scumtime test_networkx.py &> log.log
```
shows ~100s for `simple_cycles` and ~0s for `find_cycle`.
Fixes#2666
Ivan Shcheklein
Alexander Schepanovski
n3hrox
Ruslan Kuprieiev
Sadie Bartholomew
Ruslan Kuprieiev
Aman Sharma
Mr. Outis
Jorge Orpinel
Sagun Karanjit
Sanket Saurav
Casper da Costa-Luis
Dashamir Hoxha