The package-selected-packages is just the list of installed packages, doesn't load anything.
You can set
(setq use-package-always-defer t)
so anything you load with use-package will only be loaded on-demand. Or you can do it per package with e.g. `(use-package csv-mode :defer t)`.
Also make sure you don't set use-package-always-ensure and don't `:ensure t` or it'll try to install anything that's not installed on startup. If you want fast startup, it's better to explicitly install with M-x package-install
Always defer is a bad idea because some packages are required at startup or look for variables defined at startup (which is why `:init`) is a thing. Also, `:ensure t` only installs what is not installed at startup, but it does so only once and the following updates are to be made by hand. Once this is done, it does not try to install the packages anymore unless you remove them. This is how you get both fast startup and feature availability. Agreed on not using `always-ensure`, there are builtin packages that do not need it and will error out, this is why you should `:defer` and `:ensure` yourself.
You should have everything managed by use-package for coherence, even when you don't defer. For example,
(use-package org-evil
:ensure t ;; download if not present
:defer t ;; defer loading
hook (org-mode . org-evil-mode)) ;; start this mode in org buffers
(use-package csv
:ensure t
:defer t) ;; modes are usually smart enough to know in which buffer they start
and so on for all your packages. If you NEED something to start when Emacs open, you can remove the `:defer t` macro. There are also other useful things like `:init` `:config` `custom`, you should check out the docs!
