graphics – Forcing /usr/lib/xorg/Xorg to not use the NVIDIA GeForce GPU in Ubuntu 20.04

I use my NVIDIA GeForce 1650 Ti which just has <~= 4G GPU memory for deep learning applications. However, I am noticing other applications are using it quite a lot.

Are there ways to force these other applications to use the other GPU? And is it recommended to do so?

$ nvidia-smi
Tue Dec 29 17:21:16 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.38       Driver Version: 455.38       CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX 165...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   59C    P8     3W /  N/A |   1092MiB /  3911MiB |      4%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       981      G   /usr/lib/xorg/Xorg                133MiB |
|    0   N/A  N/A      1735      G   /usr/lib/xorg/Xorg                637MiB |
|    0   N/A  N/A      1906      G   /usr/bin/gnome-shell              116MiB |
|    0   N/A  N/A      2264      G   ...gAAAAAAAAA --shared-files      169MiB |
|    0   N/A  N/A      3148      G   /usr/lib/firefox/firefox            1MiB |
|    0   N/A  N/A      7685      G   /usr/lib/firefox/firefox            1MiB |
|    0   N/A  N/A     11044      G   ...AAAAAAAA== --shared-files        8MiB |
|    0   N/A  N/A     25167      G   /usr/lib/firefox/firefox            1MiB |
|    0   N/A  N/A     25343      G   /usr/lib/firefox/firefox            1MiB |
|    0   N/A  N/A     25434      G   /usr/lib/firefox/firefox            1MiB |
+-----------------------------------------------------------------------------+

I have:

$ lsb_release -a
LSB Version:    core-11.1.0ubuntu2-noarch:security-11.1.0ubuntu2-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.1 LTS
Release:    20.04
Codename:   focal

and

$ lspci
00:00.0 Host bridge: Intel Corporation 10th Gen Core Processor Host Bridge/DRAM Registers (rev 02)
00:01.0 PCI bridge: Intel Corporation 6th-9th Gen Core Processor PCIe Controller (x16) (rev 02)
00:02.0 VGA compatible controller: Intel Corporation UHD Graphics (rev 05)
00:04.0 Signal processing controller: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem (rev 02)
00:08.0 System peripheral: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model
00:12.0 Signal processing controller: Intel Corporation Comet Lake PCH Thermal Controller
00:14.0 USB controller: Intel Corporation Comet Lake USB 3.1 xHCI Host Controller
00:14.2 RAM memory: Intel Corporation Comet Lake PCH Shared SRAM
00:14.3 Network controller: Intel Corporation Comet Lake PCH CNVi WiFi
00:16.0 Communication controller: Intel Corporation Comet Lake HECI Controller
00:1c.0 PCI bridge: Intel Corporation Device 06b8 (rev f0)
00:1c.7 PCI bridge: Intel Corporation Device 06bf (rev f0)
00:1d.0 PCI bridge: Intel Corporation Comet Lake PCI Express Root Port #9 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Device 068e
00:1f.3 Multimedia audio controller: Intel Corporation Comet Lake PCH cAVS
00:1f.4 SMBus: Intel Corporation Comet Lake PCH SMBus Controller
00:1f.5 Serial bus controller (0c80): Intel Corporation Comet Lake PCH SPI Controller
01:00.0 VGA compatible controller: NVIDIA Corporation TU117M (GeForce GTX 1650 Ti Mobile) (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)
04:00.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge (Titan Ridge 4C 2018) (rev 06)
05:00.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge (Titan Ridge 4C 2018) (rev 06)
05:01.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge (Titan Ridge 4C 2018) (rev 06)
05:02.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge (Titan Ridge 4C 2018) (rev 06)
05:04.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge (Titan Ridge 4C 2018) (rev 06)
06:00.0 System peripheral: Intel Corporation JHL7540 Thunderbolt 3 NHI (Titan Ridge 4C 2018) (rev 06)
2c:00.0 USB controller: Intel Corporation JHL7540 Thunderbolt 3 USB Controller (Titan Ridge 4C 2018) (rev 06)
54:00.0 Unassigned class (ff00): Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader (rev 01)
55:00.0 Non-Volatile memory controller: SK hynix Device 1639

and

$ sudo lshw -C video
  *-display                 
       description: VGA compatible controller
       product: TU117M (GeForce GTX 1650 Ti Mobile)
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:179 memory:ad000000-adffffff memory:80000000-8fffffff memory:90000000-91ffffff ioport:3000(size=128) memory:ae080000-ae0fffff
  *-display
       description: VGA compatible controller
       product: UHD Graphics
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       logical name: /dev/fb0
       version: 05
       width: 64 bits
       clock: 33MHz
       capabilities: pciexpress msi pm vga_controller bus_master cap_list rom fb
       configuration: depth=32 driver=i915 latency=0 mode=3840x2160 visual=truecolor xres=3840 yres=2160
       resources: iomemory:600-5ff iomemory:400-3ff irq:177 memory:6040000000-6040ffffff memory:4000000000-400fffffff ioport:4000(size=64) memory:c0000-dffff

and

$ sudo lshw -C display
  *-display                 
       description: VGA compatible controller
       product: TU117M (GeForce GTX 1650 Ti Mobile)
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:179 memory:ad000000-adffffff memory:80000000-8fffffff memory:90000000-91ffffff ioport:3000(size=128) memory:ae080000-ae0fffff
  *-display
       description: VGA compatible controller
       product: UHD Graphics
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       logical name: /dev/fb0
       version: 05
       width: 64 bits
       clock: 33MHz
       capabilities: pciexpress msi pm vga_controller bus_master cap_list rom fb
       configuration: depth=32 driver=i915 latency=0 mode=3840x2160 visual=truecolor xres=3840 yres=2160
       resources: iomemory:600-5ff iomemory:400-3ff irq:177 memory:6040000000-6040ffffff memory:4000000000-400fffffff ioport:4000(size=64) memory:c0000-dffff

and

$ lspci | grep ' VGA ' | cut -d" " -f 1 | xargs -i lspci -v -s {}
00:02.0 VGA compatible controller: Intel Corporation UHD Graphics (rev 05) (prog-if 00 (VGA controller))
    Subsystem: Lenovo Device 22c0
    Flags: bus master, fast devsel, latency 0, IRQ 177
    Memory at 6040000000 (64-bit, non-prefetchable) (size=16M)
    Memory at 4000000000 (64-bit, prefetchable) (size=256M)
    I/O ports at 4000 (size=64)
    Expansion ROM at 000c0000 (virtual) (disabled) (size=128K)
    Capabilities: <access denied>
    Kernel driver in use: i915
    Kernel modules: i915

01:00.0 VGA compatible controller: NVIDIA Corporation TU117M (GeForce GTX 1650 Ti Mobile) (rev a1) (prog-if 00 (VGA controller))
    Subsystem: Lenovo Device 22c0
    Flags: bus master, fast devsel, latency 0, IRQ 179
    Memory at ad000000 (32-bit, non-prefetchable) (size=16M)
    Memory at 80000000 (64-bit, prefetchable) (size=256M)
    Memory at 90000000 (64-bit, prefetchable) (size=32M)
    I/O ports at 3000 (size=128)
    Expansion ROM at ae080000 (virtual) (disabled) (size=512K)
    Capabilities: <access denied>
    Kernel driver in use: nvidia
    Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia