remotes.http: Support dvc push for http remotes (#3343)

* remote.http: support _upload() via HTTP POST

- uploaded files are sent as chunked encoding POST data

Fixes #3247

* remote.http: add support for HTTP basic and digest auth

- username, password, ask_password options work the same way as for
  ssh remotes
- basic_auth (bool) and digest_auth (bool) options added for http(s)
  remotes
- digest_auth takes precedence over basic_auth if both are enabled

* remote.http: auth_method() unit tests

* tests: Add functional tests for HTTP remotes

- HTTP remotes now tested locally using a SimpleHTTPServer instance that
  allows reading/writing to a temp directory

* tests: fix http tests on py3.5, 3.6

* remote.http: add custom auth method

* s/custom_header/custom_auth_header/

* Apply suggestions from code review

Co-Authored-By: Saugat Pachhai <[email protected]>
Co-Authored-By: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Saugat Pachhai <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>

tests/conftest.py

@@ -4,6 +4,7 @@ import mockssh
 import pytest
 
 from dvc.remote.ssh.connection import SSHConnection
+from tests.utils.httpd import PushRequestHandler, StaticFileServer
 from .dir_helpers import *  # noqa
 
 
@@ -57,3 +58,9 @@ def _close_pools():
 
     yield
     close_pools()
+
+
+@pytest.fixture
+def http_server(tmp_dir):
+    with StaticFileServer(handler_class=PushRequestHandler) as httpd:
+        yield httpd