В чем причина выключения и перезагрузки моей машины при использовании 4 графических процессоров для обучения нейронной сети?

Моя новая система linux отключается во время обучения нейронной сети, используя все 4 моих графических процессора.

nvidia-smi указал, что температура и мощность были в пределах нормы. Я пытался отслеживать эту проблему в течение нескольких недель. Я посмотрел журналы, но как новичок в Linux я не могу точно сказать, что актуально, а что нет. Вот некоторые из результатов

            [   11.404701] scsi 10:0:0:0: Direct-Access     Samsung  Portable SSD T3  0    PQ: 0 ANSI: 6
            [   11.407116] input: HDA Intel PCH Front Mic as /devices/pci0000:00/0000:00:1b.0/sound/card0/input6
            [   11.407165] input: HDA Intel PCH Rear Mic as /devices/pci0000:00/0000:00:1b.0/sound/card0/input7
            [   11.407199] input: HDA Intel PCH Line as /devices/pci0000:00/0000:00:1b.0/sound/card0/input8
            [   11.407226] input: HDA Intel PCH Line Out Front as /devices/pci0000:00/0000:00:1b.0/sound/card0/input9
            [   11.407254] input: HDA Intel PCH Line Out Surround as /devices/pci0000:00/0000:00:1b.0/sound/card0/input10
            [   11.407287] input: HDA Intel PCH Line Out CLFE as /devices/pci0000:00/0000:00:1b.0/sound/card0/input11
            [   11.407321] input: HDA Intel PCH Front Headphone as /devices/pci0000:00/0000:00:1b.0/sound/card0/input12
            [   11.453914] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 655360 ms ovfl timer
            [   11.453914] RAPL PMU: hw unit of domain pp0-core 2^-14 Joules
            [   11.453915] RAPL PMU: hw unit of domain package 2^-14 Joules
            [   11.453915] RAPL PMU: hw unit of domain dram 2^-16 Joules
            [   11.787404] AVX2 version of gcm_enc/dec engaged.
            [   11.787405] AES CTR mode by8 optimization enabled
            [   11.806169] kvm: disabled by bios
            [   11.847001] EDAC MC: Ver: 3.0.0
            [   11.848827] EDAC sbridge: Seeking for: PCI ID 8086:6fa0
            [   11.848833] EDAC sbridge: Seeking for: PCI ID 8086:6fa0
            [   11.848837] EDAC sbridge: Seeking for: PCI ID 8086:6ffc
            [   11.848838] EDAC sbridge: Seeking for: PCI ID 8086:6ffc
            [   11.848841] EDAC sbridge: Seeking for: PCI ID 8086:6ffd
            [   11.848842] EDAC sbridge: Seeking for: PCI ID 8086:6ffd
            [   11.848845] EDAC sbridge: Seeking for: PCI ID 8086:6f60
            [   11.848848] EDAC sbridge: Seeking for: PCI ID 8086:6fa8
            [   11.848849] EDAC sbridge: Seeking for: PCI ID 8086:6fa8
            [   11.848852] EDAC sbridge: Seeking for: PCI ID 8086:6f71
            [   11.848853] EDAC sbridge: Seeking for: PCI ID 8086:6f71
            [   11.848855] EDAC sbridge: Seeking for: PCI ID 8086:6faa
            [   11.848857] EDAC sbridge: Seeking for: PCI ID 8086:6faa
            [   11.848859] EDAC sbridge: Seeking for: PCI ID 8086:6fab
            [   11.848861] EDAC sbridge: Seeking for: PCI ID 8086:6fab
            [   11.848863] EDAC sbridge: Seeking for: PCI ID 8086:6fac
            [   11.848865] EDAC sbridge: Seeking for: PCI ID 8086:6fac
            [   11.848867] EDAC sbridge: Seeking for: PCI ID 8086:6fad
            [   11.848869] EDAC sbridge: Seeking for: PCI ID 8086:6fad
            [   11.848871] EDAC sbridge: Seeking for: PCI ID 8086:6faf
            [   11.848873] EDAC sbridge: Seeking for: PCI ID 8086:6faf
            [   11.848875] EDAC sbridge: Seeking for: PCI ID 8086:6f68
            [   11.848877] EDAC sbridge: Seeking for: PCI ID 8086:6f68
            [   11.848878] EDAC sbridge: Seeking for: PCI ID 8086:6f79
            [   11.848881] EDAC sbridge: Seeking for: PCI ID 8086:6f6a
            [   11.848884] EDAC sbridge: Seeking for: PCI ID 8086:6f6b
            [   11.848887] EDAC sbridge: Seeking for: PCI ID 8086:6f6c
            [   11.848890] EDAC sbridge: Seeking for: PCI ID 8086:6f6d
            [   11.848893] EDAC sbridge: ECC is disabled. Aborting
            [   11.849390] EDAC sbridge: Couldn't find mci handler
            [   11.849915] EDAC sbridge: Failed to register device with error -19.
            [   11.870580] sd 10:0:0:0: Attached scsi generic sg4 type 0
            [   11.870957] sd 10:0:0:0: [sde] 976773168 512-byte logical blocks: (500 GB/466 GiB)
            [   11.871036] sd 10:0:0:0: [sde] Write Protect is off
            [   11.871037] sd 10:0:0:0: [sde] Mode Sense: 43 00 00 00
            [   11.871196] sd 10:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
            [   11.873856]  sde: sde1
            [   11.874574] sd 10:0:0:0: [sde] Attached SCSI disk
            [   11.875753] intel_rapl: Found RAPL domain package
            [   12.535729] Adding 7906300k swap on /dev/sda2.  Priority:-1 extents:1 across:7906300k SSFS
            [   12.541868] EDAC sbridge: Seeking for: PCI ID 8086:6fa0
            [   12.541872] EDAC sbridge: Seeking for: PCI ID 8086:6fa0
            [   12.541878] EDAC sbridge: Seeking for: PCI ID 8086:6ffc
            [   12.541879] EDAC sbridge: Seeking for: PCI ID 8086:6ffc
            [   12.541881] EDAC sbridge: Seeking for: PCI ID 8086:6ffd
            [   12.541882] EDAC sbridge: Seeking for: PCI ID 8086:6ffd
            [   12.541884] EDAC sbridge: Seeking for: PCI ID 8086:6f60
            [   12.541886] EDAC sbridge: Seeking for: PCI ID 8086:6fa8
            [   12.541887] EDAC sbridge: Seeking for: PCI ID 8086:6fa8
            [   12.541889] EDAC sbridge: Seeking for: PCI ID 8086:6f71
            [   12.541890] EDAC sbridge: Seeking for: PCI ID 8086:6f71
            [   12.541891] EDAC sbridge: Seeking for: PCI ID 8086:6faa
            [   12.541892] EDAC sbridge: Seeking for: PCI ID 8086:6faa
            [   12.541894] EDAC sbridge: Seeking for: PCI ID 8086:6fab
            [   12.541895] EDAC sbridge: Seeking for: PCI ID 8086:6fab
            [   12.541897] EDAC sbridge: Seeking for: PCI ID 8086:6fac
            [   12.541898] EDAC sbridge: Seeking for: PCI ID 8086:6fac
            [   12.541900] EDAC sbridge: Seeking for: PCI ID 8086:6fad
            [   12.541901] EDAC sbridge: Seeking for: PCI ID 8086:6fad
            [   12.541902] EDAC sbridge: Seeking for: PCI ID 8086:6faf
            [   12.541903] EDAC sbridge: Seeking for: PCI ID 8086:6faf
            [   12.541905] EDAC sbridge: Seeking for: PCI ID 8086:6f68
            [   12.541906] EDAC sbridge: Seeking for: PCI ID 8086:6f68
            [   12.541908] EDAC sbridge: Seeking for: PCI ID 8086:6f79
            [   12.541910] EDAC sbridge: Seeking for: PCI ID 8086:6f6a
            [   12.541912] EDAC sbridge: Seeking for: PCI ID 8086:6f6b
            [   12.541915] EDAC sbridge: Seeking for: PCI ID 8086:6f6c
            [   12.541917] EDAC sbridge: Seeking for: PCI ID 8086:6f6d
            [   12.541920] EDAC sbridge: ECC is disabled. Aborting
            [   12.542266] EDAC sbridge: Couldn't find mci handler
            [   12.542609] EDAC sbridge: Failed to register device with error -19.
            [   12.583474] audit: type=1400 audit(1512326286.816:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/ippusbxd" pid=1065 comm="apparmor_parser"
            [   12.583632] audit: type=1400 audit(1512326286.816:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/cups-browsed" pid=1063 comm="apparmor_parser"
            [   12.583857] audit: type=1400 audit(1512326286.816:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/snapd/snap-confine" pid=1062 comm="apparmor_parser"
            [   12.583858] audit: type=1400 audit(1512326286.816:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=1062 comm="apparmor_parser"
            [   12.583964] audit: type=1400 audit(1512326286.816:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/tcpdump" pid=1067 comm="apparmor_parser"
            [   12.584280] audit: type=1400 audit(1512326286.816:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/sbin/dhclient" pid=1057 comm="apparmor_parser"
            [   12.584282] audit: type=1400 audit(1512326286.816:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=1057 comm="apparmor_parser"
            [   12.584282] audit: type=1400 audit(1512326286.816:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=1057 comm="apparmor_parser"
            [   12.584283] audit: type=1400 audit(1512326286.816:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=1057 comm="apparmor_parser"
            [   12.607980] NVRM: Your system is not currently configured to drive a VGA console
                           on the primary VGA device. The NVIDIA Linux graphics driver
                           requires the use of a text-mode VGA console. Use of other console
                           drivers including, but not limited to, vesafb, may result in
                           corruption and stability problems, and is not supported.
            [   12.684737] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready
            [   12.901781] IPv6: ADDRCONF(NETDEV_UP): eno1: link is not ready
            [   12.903895] IPv6: ADDRCONF(NETDEV_UP): enp14s0: link is not ready
            [   12.948573] IPv6: ADDRCONF(NETDEV_UP): enp14s0: link is not ready
            [   13.718428] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:08.0/0000:06:00.1/sound/card3/input17
            [   13.718516] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:08.0/0000:06:00.1/sound/card3/input18
            [   13.718584] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:08.0/0000:06:00.1/sound/card3/input19
            [   13.718651] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:08.0/0000:06:00.1/sound/card3/input20
            [   13.718678] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:02.0/0000:07:00.0/0000:08:08.0/0000:0a:00.1/sound/card1/input25
            [   13.718865] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:02.0/0000:07:00.0/0000:08:08.0/0000:0a:00.1/sound/card1/input26
            [   13.721913] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:02.0/0000:07:00.0/0000:08:08.0/0000:0a:00.1/sound/card1/input27
            [   13.722090] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:02.0/0000:07:00.0/0000:08:10.0/0000:09:00.1/sound/card2/input13
            [   13.722271] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:02.0/0000:07:00.0/0000:08:08.0/0000:0a:00.1/sound/card1/input28
            [   13.723585] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:10.0/0000:05:00.1/sound/card4/input21
            [   13.723698] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:02.0/0000:07:00.0/0000:08:10.0/0000:09:00.1/sound/card2/input14
            [   13.723996] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:02.0/0000:07:00.0/0000:08:10.0/0000:09:00.1/sound/card2/input15
            [   13.724157] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:10.0/0000:05:00.1/sound/card4/input22
            [   13.724308] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:10.0/0000:05:00.1/sound/card4/input23
            [   13.724459] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:02.0/0000:07:00.0/0000:08:10.0/0000:09:00.1/sound/card2/input16
            [   13.724633] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:03.0/0000:03:00.0/0000:04:10.0/0000:05:00.1/sound/card4/input24
            [   14.576237] igb 0000:0e:00.0 enp14s0: igb: enp14s0 NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX
            [   14.576913] IPv6: ADDRCONF(NETDEV_CHANGE): enp14s0: link becomes ready
            [   15.892367] nvidia-modeset: Allocated GPU:0 (GPU-bffcceff-4388-ca5f-54f1-661f60ed3f91) @ PCI:0000:05:00.0
            [   16.733705] nvidia-modeset: Allocated GPU:1 (GPU-7385a2d1-521a-231a-3552-488574e039f1) @ PCI:0000:0a:00.0
            [   17.367479] nvidia-modeset: Allocated GPU:2 (GPU-709514aa-7aaf-e268-56cf-b986961602e4) @ PCI:0000:09:00.0
            [   18.003041] nvidia-modeset: Allocated GPU:3 (GPU-61598621-29d4-8d9b-abfc-87f004019f3e) @ PCI:0000:06:00.0

Я прошу прощения за этот огромный поток информации - я подозреваю, что проблема связана с моими графическими процессорами. Мой блок питания 1600 Вт, так что должно быть в порядке.