MuHack - User contributions [en]

Gaspare (Server)

2025-05-13T21:00:43Z

Cammo: /* Using your Debian 12 LXC */

{{Service
|Service URI=gpu.muhack
|Location=Aula associazioni
|Contact email=webmaster@muhack.org
|Operational Status=Working
|Super Users=Ceres-c, Mrmoddom, Cammo
|Owner=MuHack
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.muhack''' (reachable only locally or through the MuHack Tailscale network)
* Management : Proxmox web UI @ port 8006 → [https://gpu.muhack:8006 Link]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''
{{Note|type=warn|text=After the first login you have to re-login a second time to let proxmox setup correctly your account's group }}

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the following script:
<syntaxhighlight lang="bash">
#!/bin/bash
wget https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb
apt install -y ./cuda-keyring_1.1-1_all.deb
apt update
rm cuda-keyring_1.1-1_all.deb
apt install -y cuda-toolkit-12-4

echo "export PATH=\${PATH}:/usr/local/cuda/bin" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=\${LD_LIBRARY_PATH}:/usr/local/cuda/lib64" >> ~/.bashrc
</syntaxhighlight>

After running the script, apply the environment changes by sourcing your `.bashrc`:

<syntaxhighlight lang="bash">
source ~/.bashrc
</syntaxhighlight>

You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Tailscale VPN

2025-05-13T20:57:40Z

Cammo:

= Accessing Internal Services via Tailscale =

Many of our non-public services, those hosted on servers physically located in the ''Aula Associazioni'', are only accessible from within our VPN.

== VPN Access via Tailscale ==

We use [https://tailscale.com/ Tailscale] to manage our internal VPN. Tailscale is a mesh VPN that simplifies access to our local infrastructure.

=== Headscale Deployment ===

We run a self-hosted [https://github.com/juanfont/headscale Headscale] instance, which serves as the coordination server for our Tailscale network. You will need to authenticate through it in order to join the VPN.

=== Important Notes ===
# This VPN does not provide an exit node: your external IP address will remain unchanged.
# The VPN only allows access to internal services—we do not route general internet traffic through it.
# Once connected, you'll be able to reach services hosted in the ''Aula Associazioni'' using their Tailscale IPs or hostnames thanks to the [https://tailscale.com/kb/1081/magicdns MagicDNS]. On each [[:Category:Services|service]] page, the [[Property:Service URI]] (and port, if necessary) should be available in the infobox on the right side.

== Access ==
# Follow the Tailscale client [https://tailscale.com/kb/1347/installation install guide] for your operating system.
# Use the following command to login our tailnet
<syntaxhighlight lang="bash">
tailscale login --login-server=https://vpn.muhack.org
</syntaxhighlight>
Then open the link you'll get in the terminal, and log in with your [[MuHack Account|MuHack workspace account]]

Note: this also works with the mobile app

{{Note|text=On MacOS
<syntaxhighlight lang="bash">
brew install tailscale
tailscale login --login-server=https://vpn.muhack.org
</syntaxhighlight>
}}

{{Note|text=On LXC Container
if you're trying to run tailscale on you LXC Container on one of our servers, you need to run it in [https://tailscale.com/kb/1112/userspace-networking userspace-networking mode]
<syntaxhighlight lang="bash">
tailscaled --tun=userspace-networking
tailscale login --login-server=https://vpn.muhack.org
</syntaxhighlight>
}}

Gaspare (Server)

2025-05-13T20:46:55Z

Cammo: /* Using your Debian 12 LXC */

{{Service
|Service URI=gpu.muhack
|Location=Aula associazioni
|Contact email=webmaster@muhack.org
|Operational Status=Working
|Super Users=Ceres-c, Mrmoddom, Cammo
|Owner=MuHack
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.muhack''' (reachable only locally or through the MuHack Tailscale network)
* Management : Proxmox web UI @ port 8006 → [https://gpu.muhack:8006 Link]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''
{{Note|type=warn|text=After the first login you have to re-login a second time to let proxmox setup correctly your account's group }}

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
#!/bin/bash
wget https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb
apt install -y ./cuda-keyring_1.1-1_all.deb
apt update
rm cuda-keyring_1.1-1_all.deb
apt install -y cuda-toolkit-12-4

echo "export PATH=\${PATH}:/usr/local/cuda/bin" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=\${LD_LIBRARY_PATH}:/usr/local/cuda/lib64" >> ~/.bashrc
</syntaxhighlight>

After running the script, apply the environment changes by sourcing your `.bashrc`:

<syntaxhighlight lang="bash">
source ~/.bashrc
</syntaxhighlight>

You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T18:12:10Z

Cammo:

{{Service
|Service URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Location=Aula associazioni
|Contact email=webmaster@muhack.org
|Operational Status=Working
|Super Users=Ceres-c, Mrmoddom, Cammo
|Owner=MuHack
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI @ port 8006 → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T18:11:12Z

Cammo:

{{Service
|Service URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Location=Aula associazioni
|Operational Status=Working
|Super Users=Ceres-c, Mrmoddom, Cammo
|Owner=MuHack
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI @ port 8006 → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

User:Cammo

2025-05-10T18:10:40Z

Cammo: Created page with "I do stuff @ muHack"

I do stuff @ muHack

Gaspare (Server)

2025-05-10T18:10:10Z

Cammo:

{{Service
|Service URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Location=Aula associazioni
|Operational Status=Working
|Super Users=Ceres-c, Mrmoddom
|Owner=MuHack
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI @ port 8006 → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T18:04:10Z

Cammo:

{{Service
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack (reachable on the MuHack Tailscale net)
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI @ port 8006 → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T18:02:52Z

Cammo:

{{Service
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack (reachable only through the MuHack Tailscale network)
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI @ port 8006 → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T18:02:31Z

Cammo: /* Fast facts */

{{Service
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI @ port 8006 → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T18:01:49Z

Cammo:

{{Service
|Physical Location=Aula associazioni
|URI=gpu.vpn.muhack
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T18:01:08Z

Cammo:

{{Service
|Physical Location=Aula associazioni
|URI=https://gpu.vpn.muhack
|Project Status=Active
|Participants=Ceres-c, Mrmoddom, Cammo
}}
= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T17:57:36Z

Cammo:

{{Service
|Project Status = Active
|Participants = Ceres-c, Mrmoddom, Cammo
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T17:06:44Z

Cammo: /* Troubleshooting quick-ref */

{{Project
|Project Status = Active
|Participants = Ceres-c, Mrmoddom, Cammo
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* '''BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee'''

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MuHack meeting.

Gaspare (Server)

2025-05-10T17:06:23Z

Cammo: /* Good-citizen checklist */

Gaspare (Server)

2025-05-10T17:02:58Z

Cammo:

{{Project
|Project Status = Active
|Participants = Ceres-c, Mrmoddom, Cammo
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MUHACK meeting.

Gaspare (Server)

2025-05-10T17:02:50Z

Cammo:

{{Project
|Project Status = Active
|Participants = Ceres-c, Mrmoddom, Cammo
|Host name = gpu.vpn.muhack (reachable only through the MuHack Tailscale network)
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MUHACK meeting.

Gaspare (Server)

2025-05-10T17:02:19Z

Cammo:

{{Project
|Project Status = Active
|Participants = Ceres-c, Mrmoddom, Cammo
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MUHACK meeting.

Gaspare (Server)

2025-05-10T17:01:39Z

Cammo:

{{Project
|Project Status = Active
|Host name = gpu.vpn.muhack (reachable only through the MuHack Tailscale network)
|Management = Proxmox web UI on port 8006 → [https://gpu.vpn.muhack:8006]
|Login credentials = your MuHack account
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MUHACK meeting.

Gaspare (Server)

2025-05-10T17:01:18Z

Cammo:

{{Project
|Project Status = Active
|Partecipants = Ceres-c, Mrmoddom, Cammo
|Host name = gpu.vpn.muhack (reachable only through the MuHack Tailscale network)
|Management = Proxmox web UI on port 8006 → [https://gpu.vpn.muhack:8006]
|Login credentials = your MuHack account
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MUHACK meeting.

Gaspare (Server)

2025-05-10T17:00:56Z

Cammo:

{{Project
|Project Status = Active
|Sysadmins = Ceres-c, Mrmoddom, Cammo
|Host name = gpu.vpn.muhack (reachable only through the MuHack Tailscale network)
|Management = Proxmox web UI on port 8006 → [https://gpu.vpn.muhack:8006]
|Login credentials = your MuHack account
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MUHACK meeting.

Gaspare (Server)

2025-05-10T17:00:14Z

Cammo:

{{Project
|Project Status = Active
|Sysadmins = Ceres-c, Mrmoddom, Cammo
|Host name = gpu.vpn.muhack (reachable only through the MuHack Tailscale network)
|Management = Proxmox web UI on port 8006 → [1]
|Login credentials = your MuHack account
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MUHACK meeting.

Gaspare (Server)

2025-05-10T16:56:38Z

Cammo:

{{Project
|Project Status = Active
|Participants = Ceres-c, Mrmoddom, Cammo
}}

= MUHACK GPU SERVER =
A shared machine that gives all members access to serious GPU horsepower for AI/ML, rendering, data-science and other compute-intensive experiments.

== What this box is for ==

* Training / fine-tuning neural networks
* CUDA / OpenCL development and compilation
* 3-D rendering, video encoding or scientific workloads
* General “I-need-48-GB-of-VRAM” tinkering

The host runs Proxmox VE, so we can either
# carve the GPU into vGPU slices for VMs, or
# passthrough the full card to Debian 12 LXC containers.

== Fast facts ==

* Host name : '''gpu.vpn.muhack''' (reachable only through the MuHack Tailscale network)
* Management : Proxmox web UI → [https://gpu.vpn.muhack:8006]
* Sysadmins : '''Ceres-c, Mrmoddom, Cammo'''
* Contact : [mailto:webmaster@muhack.com webmaster@muhack.com] • or in person every Tuesday evening at the MuHack meeting

== Hardware ==

* CPU : Intel Core i7-4930K — 6 cores / 12 threads @ 3.40 GHz (3.90 GHz Turbo)
* RAM : 64 GB DDR3-1866
* GPU-A : NVIDIA Quadro RTX 8000 — 48 GB GDDR6 (compute card)
* GPU-B : NVIDIA GeForce GT 710 — emergency host console only
* Disk : 1 TB PNY SATA SSD + additional SATA ports free for expansion
* NICs :
** Intel 82574L Gigabit
** Intel 82579V Gigabit

== Software stack ==

* Proxmox VE 8.x (kernel 6.8)
* “Merged” NVIDIA driver 550.90.07 (both vGPU and passthrough available)

'''Pre-built images'''
* Debian 12 LXC (default; CUDA-ready; shared GPU)
* Ubuntu 22.04 / Windows VM (on request, with dedicated vGPU slice, only for specific necessities)

== Getting your own environment ==

# Make sure your laptop/PC is connected to the MuHack Tailscale tail-net and can <tt>ping gpu.vpn.muhack</tt>.
# Talk to '''any sysadmin''' (email or in person on Tuesday). Tell us
## what you plan to run (ML, rendering, etc.)
## an estimate of how much GPU RAM / runtime you’ll need
# We’ll create a container (or VM) for you and hand back:
## ID (e.g. <tt>ct104</tt> or <tt>vm203</tt>)
## a '''random initial password'''

''Typical turnaround time: usually same day or next-day. However might take more time if the sysadmins are busy.''

== Using your Debian 12 LXC ==

Need CUDA toolkit? Run the helper script already placed in <tt>/root</tt>:
<syntaxhighlight lang="bash">
/root/install-cuda-toolkit.sh
source ~/.bashrc
nvcc --version
</syntaxhighlight>
You’ll get CUDA toolkit 12.4 that matches the host driver.

== Fair-share policy ==

* Jobs longer than '''12 h''' → announce them at the Tuesday meeting or in the telegram channel.
* Sysadmins may pause/stop workloads that block others
* Crypto-mining strictly forbidden

== Good-citizen checklist ==

* Before logging out, run <tt>watch -n60 nvidia-smi</tt> and confirm your process is gone and is not hogging the gpu.
* Keep <tt>/tmp</tt> and your home directory tidy; disk space is shared
* BACK UP YOUR OWN DATA: we have no data persistency or backup guarantee

== Extras ==

* vGPU-enabled Windows or Ubuntu VMs — ask if needed
* Docker-inside-LXC supported (nesting + cgroupv2 enabled)

== Troubleshooting quick-ref ==

{| class="wikitable"
! Problem
! Fix
|-
| <tt>nvidia-smi</tt> shows no devices
| Container lost its GPU mapping—tell a sysadmin
|-
| Driver mismatch error
| Host driver was updated—reinstall matching userspace driver with <tt>--no-kernel-modules</tt>—tell a sysadmin
|-
| Out-of-memory in PyTorch
| If using a VM: request a larger vGPU slice. If using a container: someone else might be using some GPU memory. Check with <tt>nvidia-smi</tt> (gui for that coming up sometime kinda soon).
|}

Questions / ideas → email '''[mailto:ceres-c@muhack.com ceres-c@muhack.com]''', '''[mailto:mrmoddom@muhack.com mrmoddom@muhack.com]''', or '''[mailto:cammo@muhack.com cammo@muhack.com]''' — or just grab us at the Tuesday MUHACK meeting.