Commit Graph
38 Commits
Author SHA1 Message Date
Saugat Pachhai e33274723d Move flags from tests to config (#4943) 2020-11-23 16:21:24 +05:45
Saugat Pachhai 5faae2a6e9 test: allow selectively running tests for remotes (#4920)
Some of the tests are run by default, other requires
`--enable-{remote}` to run. The remotes that are run by default
also have a `--disable-{remote}` flag. All of the remotes can be
enabled using `--all` flag.

To use this functionality in the fixture,
`test_config` fixture has to be used, and requires to call
`test_config.requires(remote)`.
In an individual test, you can simply use `pytest.mark.remote`
to make them skip or run.
2020-11-23 11:43:39 +05:45
Ruslan Kuprieiev 76228a2118 dvc: split remotes from trees (#4212)
* dvc: remove tree-specific wrappers from remote

This helps combat the remote/tree/cache confusion.

* dvc: split remotes from trees

Related to #4050
2020-07-15 23:44:09 +03:00
Saugat Pachhai ad306d7bf3 Enable most of the pylint checks (#4111) 2020-06-25 18:10:14 +03:00
Saugat Pachhai b81ace54df Enable pylint checks (#4092)
* pylint: enable checks for imports

* ci: add github action for pylint

* enable pylint on pre-commit

* Install requirements

* use local pylint installation rather than of pre-commit's

* fix import errors

* more checks

* Silence unused imports

* Make a copy of dict
2020-06-24 18:02:39 +03:00
Ruslan Kuprieiev 51a8c782ae tests: create remote fixtures (#4063)
* tests: use remote fixtures

* tests: use remote fixtures for data cloud tests

* tests: treat remotes as tmp dirs
2020-06-18 01:36:13 +03:00
Ruslan Kuprieiev 02a52d038a tests: make ssh_server session scoped (#3898)
Fixes #3888
2020-05-28 16:09:58 +03:00
Saugat Pachhai b8aa8ad951 pyupgrade: 3.6+ (#3805)
* pyupgrade: 3.6+

* tests: pyupgrade 3.6+
2020-05-15 16:48:02 +03:00
Casper da Costa-Luis 2bb7257a06 lint: isort pre-commit & misc improvements (#3749)
* test: clean flake8 config root clutter

* test: pre-commit: add isort

Compatible with black (see https://github.com/timothycrosley/isort/issues/694)

* lint: reorder imports

* lint: more complex imports

* lint: fix deepsource

* config: isort: add first party dvc

* tests: restyled: add isort

* temp commit to revert before merge

* Revert "temp commit to revert before merge"

This reverts commit 11db0fd8e95fe891597cbb9bf804480909b19e24.
2020-05-06 23:56:07 +03:00
68e543f7e1 remotes.http: Support dvc push for http remotes (#3343)
* remote.http: support _upload() via HTTP POST

- uploaded files are sent as chunked encoding POST data

Fixes #3247

* remote.http: add support for HTTP basic and digest auth

- username, password, ask_password options work the same way as for
  ssh remotes
- basic_auth (bool) and digest_auth (bool) options added for http(s)
  remotes
- digest_auth takes precedence over basic_auth if both are enabled

* remote.http: auth_method() unit tests

* tests: Add functional tests for HTTP remotes

- HTTP remotes now tested locally using a SimpleHTTPServer instance that
  allows reading/writing to a temp directory

* tests: fix http tests on py3.5, 3.6

* remote.http: add custom auth method

* s/custom_header/custom_auth_header/

* Apply suggestions from code review

Co-Authored-By: Saugat Pachhai <[email protected]>
Co-Authored-By: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Saugat Pachhai <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-02-21 12:05:55 +02:00
pawel c2f5e35f1d conftest: remove repo_dir 2020-01-16 10:43:48 +01:00
pawel a4b55a00ec conftest: remove dvc_repo, erepo, git, git_erepo 2020-01-16 10:43:48 +01:00
Mr. Outis b844aea42c py3: remove cast_bytes method 2019-12-31 20:14:25 -06:00
Mr. Outis 94a0914bba py3: get rid of future imports 2019-12-30 19:35:48 -06:00
prd d40588aefa temporary_windows_drive fixture: move to dir_helpers 2019-12-16 13:15:53 +01:00
Mayur Kularni ca20a1c93a incorporated feedback 2019-12-08 18:56:38 +05:30
Alexander Schepanovski fcebad821e test: refactor tmp dir helper fixtures
- `tmp_dir` descends from Path
- convenient template methods to generate files and dirs, add them to
dvc and git and commit to git
- independent of old basic_env structures
- can go in any order
- modular: start with empty `tmp_dir`, initialize git with `scm`, dvc
repo with `dvc` add "repo template" with `repo_template`, ...
2019-12-02 23:31:54 +01:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Casper da Costa-Luis 1e01ce3ac9 isatty (#2550)
* suppress Tqdm when isatty

- fixes #2549

* fix tests

* unconditionally disable Tqdm if not isatty

- fixes https://github.com/iterative/dvc/pull/2550#discussion_r329386950

* add TQDM_DISABLE env var

- update tests
- add documentation

* TQDM_DISABLE => TQDM_ISATTY

* tests: mock.patch sys.stderr

* TQDM_ISATTY => DVC_ISATTY

as per https://github.com/iterative/dvc/pull/2550#discussion_r334565120

* dvc.progress.Tqdm file kwarg

- fixes https://github.com/iterative/dvc/pull/2550/files/ad1f245107c405697c4b61fe58a92fc0265c6588#r335706955

* dynamic env DVC_ISATTY

- fixes
  https://github.com/iterative/dvc/pull/2550/files/ad1f245107c405697c4b61fe58a92fc0265c6588#r335811454

* DVC_ISATTY in tests

- fixes https://github.com/iterative/dvc/pull/2550/files/ad1f245107c405697c4b61fe58a92fc0265c6588#r335811099

* re-add and abstract `env2bool` to `dvc.utils`

- partially reverts commit 524a0df188f566fb3e103f6965f1b6ac9c530886
- fixes
  https://github.com/iterative/dvc/pull/2550/files/ad1f245107c405697c4b61fe58a92fc0265c6588#r335811639

* env2bool tidy

* rename DVC_ISATTY => DVC_IGNORE_ISATTY

- fixes https://github.com/iterative/dvc/pull/2550#discussion_r335947382
2019-10-20 00:54:06 +03:00
pawel 4236f18e0c repo: get: intercept and convert NotDvcRepoErro to UrlNotDvcRepoError 2019-09-03 13:59:04 +02:00
RahulR19 371c57ac24 Renamed git to repo in dvc/scm/git along with changes in some tests. 2019-08-04 23:26:45 +05:30
Alexander Schepanovski 6afc2a38ba test: close pools on tests cleanup
This avoids weird errors as pools are collected by gc in broken env.
2019-07-21 13:40:29 +07:00
Alexander Schepanovski ed387d824d remote: do not swallow exceptions by ssh pool 2019-07-17 22:20:43 +07:00
Ruslan Kuprieiev 918ef646dc appveyor: test py2 and py3
Related to #2166

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-23 13:33:47 +03:00
Ruslan Kuprieiev 1ba8e0171d dvc: replace pkg concept with external repo
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-21 01:16:00 +03:00
prd 0615ff8730 pathinfo: extract relative path method to standalone method 2019-06-18 12:08:19 +02:00
Ruslan Kuprieiev 5941ace270 dvc: initial 'dvc pkg' implementation
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-17 18:12:36 +03:00
Ruslan Kuprieiev f41dcea248 tests: don't use var before assignment
```
repo_dir = <tests.basic_env.TestDirFixture object at 0x7f36c3f3d650>                  
git = <git.Repo "/tmp/dvc-test.2894.0LMoaT.eefefa48-172d-44e4-8c3d-4f65f76c78ee/.git">
                                                                                      
    @pytest.fixture                                                                   
    def dvc_repo(repo_dir, git):                                                      
        try:                                                                          
            dvc = DvcRepo.init(repo_dir._root_dir)                                    
            dvc.scm.commit("init dvc")                                                
            yield dvc                                                                 
        finally:                                                                      
>           dvc.scm.git.close()                                                       
E           UnboundLocalError: local variable 'dvc' referenced before assignment      
                                                                                      
/home/efiop/git/dvc/tests/conftest.py:76: UnboundLocalError                           
```
2019-06-06 13:44:13 +03:00
pawel 01c2f6c174 tests: rename dvc repo fixture 2019-05-21 15:04:43 -07:00
prd 1cb4e5233e path: local: str return full path on value error 2019-05-14 16:56:24 -07:00
Ruslan Kuprieiev 755a8bab06 Merge pull request #1949 from Suor/tests-extra-processes
test: never run updater nor analytics in tests
2019-05-01 15:50:53 +03:00
Alexander Schepanovski 6e0e5b426b test: never run updater nor analytics in tests
Previously that was only the case when running tests via `python
-mtests` wrapper.
2019-05-01 13:48:57 +07:00
Alexander Schepanovski cdf318fff7 test: abandon default DEBUG log level in tests
Now simple `py.test` command won't use DEBUG log level and `py.test ...
--log-level=LEVEL` will do what it's supposed to.

Staging continues to use DEBUG log level.

This is part of #1888 resolution.
2019-04-30 17:43:39 +07:00
ag613915cao 0dabf7bbaa Replace yield_fixture()
Because yield_fixture() is deprecated since pytest 3.0 [1]. It is
not recommend to use the decorator instead of fixture().

[1] https://docs.pytest.org/en/latest/yieldfixture.html
2019-04-29 15:17:10 +07:00
ag613915cao fd385c913b This commit intends to:
- Keep current SSH remote connection test on real system
- Faked and Mocked SSH server which running on both Linux and Windows
- Add corresponding tests

Fixed #1704
2019-04-26 18:07:16 +07:00
Alexander Schepanovski 7478e7d4b0 test: clean up test dirs (#1904)
* test: clean up test dirs

Closes #1770. Partially does #1888.

* dvc: close anything opened

* test: finalize git repo explicitly to remove test dir later

This was previously an issue on Windows.
2019-04-25 14:55:39 +03:00
Alexander Schepanovski 277b9e566c test: use caplog.at_level() to restore log level after test 2019-04-24 02:45:02 +03:00
Alexander Schepanovski 0c867399bb dvc: retain formatting and comments in .dvc files (#1885)
* test: remove never used copy of load_stage_file()

* dvc: save comments and custom formatting in stage files

* Add a couple of TODOs

* test: test repro doesn't clear comments and formatting in .dvc

As part of this I started writing pytest-like tests and added commonly
used fixtures.

* test: simplify and prettify dvc formatting retained test

Also test that line comments after md5 fields are retained.

* test: fix tests

* dvc: fix stage and metrics opens not closed

* dvc: always use utf-8 in stage files

It used to encode in system default encoding. Not sure if PyYAML was
handling this silently or we were just lucky all the way.

* test: unit test apply_diff()

* test: remove TODOs and ASK

* dvc: polish dvc-formatting PR

* dvc: move stage load/dump utils to a new stage utils file

* dvc: clean up apply_diff()

* test: test stage md5 ignores comments

* test: remove pytests starting marker
2019-04-20 01:12:53 +03:00