Commit Graph
8 Commits
Author SHA1 Message Date
skshetry b647475a64 drop Python 3.8 support (#10265)
* drop Python 3.8

Note that this PR only makes typing related changes.
Other changes can be done once this gets merged, they
are relatively few in numbers.

* bump build python version

* add typing_extensions to lint group
2024-01-30 11:33:25 +02:00
skshetry 4eb073868c update ruff to 0.1.15 (#10263)
update ruff
2024-01-30 13:09:55 +05:45
skshetry 519cce3fe4 logger: type-safe logger.trace (#10083)
* use logger.getChild from `dvc.log` instead of `logging.getLogger`

I got tired adding `# type: ignore[attr-defined]` while using `logger.trace`.
So, now you need to import logger from `from dvc.log import logger` and
create a child logger using `logger.getChild(__name__)`. `logger.getChild`
will return a logger type-hinted as a logger having a `trace` method.

So, from now on, instead of:
```python
import logging

logger = logging.getogger(__name__)
```

you'll have to do following:
```python
from dvc.log import logger

logger = logger.getChild(__name__)
```

* remove all dvc imports from dvc.logger

* use Tqdm back
2023-11-10 12:55:49 +05:45
skshetry 89b4beb7a4 daemon: ipc to log pid from the child (#10058)
* daemon: ipc to log pid from the child

* fix tests to use pid

* also test for updater/analytics keyword
2023-10-30 20:47:52 +05:45
skshetry b23ba58cd6 tests: daemon: log body of the request (#10039)
log body of the request
2023-10-19 17:16:34 +05:45
skshetry 8fb67b71a5 expose test webserver as an executable for easier testing (#10037) 2023-10-19 16:28:47 +05:45
skshetry 5693775459 refactor: use int as the factory (#10036)
makes it thread-safe if we need in the future. Also looks a bit clean.
2023-10-19 15:31:33 +05:45
skshetry 17f5508f49 daemon: add logging support, tests and generalize popen based runs for future (#10033)
* use popen to run updater/analytics in detached mode

subprocess does not provide a way to avoid a zombie process. Not to confuse with orphaned processes, which will be `wait`-ed by `pid 1`.

It might not work for long-running processes, but I don't think zombie processes are going to be a problem for us, as the parent process immediately exits after spawning a child process.

Even in case of `queue` based experiment runner, parent will exit before than the child process. So I don't think it's a requirement here. But it does feel wrong to leak the processes, so I have to sleep on that.

---

This PR:

1. makes `daemonized` process inherit the logging verbosity, so if the parent process used `-vv`, the daemonized process will run the same way.
2. Add `DVC_DAEMON_LOGFILE` envvar, that the daemon will log to the file. By default, the verbosity is the same as running parent processes. Without `DVC_DAEMON_LOGFILE`, the output will be redirected to `/dev/null`.
3. Adds debug/trace for updater/analytics.

I hope by doing this, we can make our life easier on debugging and make daemon more transparent/observable.

* add tests

* fixes

* use kwargs

* pass all os.environ with some modified
2023-10-19 15:14:41 +05:45