While systems based on AMD AI max+ 395 are cheaper (and might in some cases even be faster), the NVIDIA Spark DGX systems have the advantage of providing the CUDA ecosystem. (However, if you just want to run LLMs, then Ollama and LM Studio support the AMD AI processors as well as CUDA.) Both system suffer from the fact, that standard DDR5 RAM is used (or to be more precise LPDDR5, i.e. soldered, instead of upgradable DIMMs).

I did some research on the Spark DGX

• Comes with no 200 GB QSFP Direct Attach Cable (DAC) to cluster two of them without a 200 GB switch in-between, i.e. you need to buy an extra one.
• enP7p1s0 is the Realteak-based 10 GB RJ45 Ethernet port. The others (enp1s0f0np0 enp1s0f1np1 enP2p1s0f0np0 enP2p1s0f1np1) are the two 200 GB QSFP56 ports, i.e. two 100 GB ports each are bundled into a 200 GB port).
• Setup is described here: https://docs.nvidia.com/dgx/dgx-spark/first-boot.html
• While it is setting up, it creates a WiFi AP (SSID shown a booklet that ships with the hardware) so that you can connect and create a user via a Web browser. It uses the Ethernet connection for installing OS updates. Once that setup has been finished, the WiFi AP gets disabled! You should be able to enable the AP again, e.g. using NetworkManager, but this will not NAT the WiFi AP client traffic via the Ethernet connection, i.e. while your WiFi AP client can access the DGX Spark, you cannot browse the Internet.
• From the documentation:
  

  The machine name is your DGX Spark hostname with .local appended, such as spark-xxxx.local. You can find the default hostname on the Quick Start Guide that came in the box. The .local address uses mDNS (multicast DNS) to automatically locate your DGX Spark on the net-work without needing to know its IP address. This is particularly useful if your router periodically reassigns IP addresses.

  For Windows users: mDNS requires Bonjour Print Services from Apple. If you have iTunes or other
  Apple software installed, you likely already have this. Otherwise, you can download it from Ap-
  ple’s website. Alternatively, you can try using just the hostname without .local (such as spark-xxxx),
  though this method is less reliable on modern networks.

  Why .local might not work: .local hostnames may not work in enterprise networks with strict se-
  curity policies, networks that block multicast traffic, or other restricted network environments.

  Using an IP address instead: If .local hostnames do not work, you will need to use the IP address. To
  find the IP address, physically log in to your DGX Spark and click the network icon in the top right
  corner of the Ubuntu desktop. Select Settings from the dropdown menu, then navigate to the
  Network section. The IP address will be displayed under as "Realtek Ethernet". Click on the settings icon to see its IPv4 and IPv6 address.
  Alternatively, you
  can log in to your router’s administration console to view connected devices and their IP addresses.

• AS this OS Ubuntu 2024.04 and uses Netplan for all network configuration.
  To change the hostname, you can either use the DGX Dashboard (see below and then Settings, System, Edit Hostname):

  sudo hostnamectl set-hostname new_hostname

  E.g.: sudo hostnamectl set-hostname spark1

  Restart the device or its services for the change to take effect.

  Further netplan commands: netplan get, netplan status --all

  To generate configuration files for all the involved network tools from the yaml file:
  sudo netplan generate. To try (will revert after 120 s): netplan try To finally apply persistently: netplan apply

• Talking about Ubuntu: Release notes are available, but these refer both to the full-blown X86_64 DGX and the ARM64 DGX Spark, so search there for Spark. recovery media archive file is also available if you need to reinstall.
• As this is standard Ubuntu, you will very likely need to do some variant of the usual hardening.
• The https://build.nvidia.com/spark/connect-to-your-spark/sync NVIDIA Sync tool tunnels from your client machine via SSH to the DGX Spark: at the first usage of NVIDIA Sync, it will ask for your username and password: On Linux systems, it will then create a password-less SSH key (so that in future, NVIDIA Sync can tunnel without needing to ask for a password) and copy the public-key of the SSH key over to the DGX Spark (it will do so even if you already have an SSH key). On Mac or MS Windows system, it seems not to assume that there is support for password-less SSH keys.

  The NVIDIA Sync documentation contains some copy/paste instructions for Ubuntu/Debian of how add it as APT source: I do not like that they append it to sources.list -- I would rather make it file on its own in sources.list.d/. Also the provided deb entry might need to be changed into deb [arch=amd64] .

• There seems to be an issue with the file /opt/nvidia/dgx-dashboard-service/jupyterlab_ports.yaml. But restarting the system after a user has been added, seems to help. The question is whether restarting some service using systemctl restart would do the same job as rebooting.
  Update: I tried systemctl restart dgx-dashboard.service (which will then ask for the sudo-enabled user to be used for that, incl. entering the password) and that did add the missing entry. sudo systemctl restart dgx-dashboard.service did not ask for the user to be used.
• If there is permission issue with docker: sudo usermod -aG docker $USER
newgrp docker
• https://build.nvidia.com/spark Playbooks with first steps to play around.

  Start with activating Jupyter lab viah the DGX Dashboard. There seems to be an issue with the file /opt/nvidia/dgx-dashboard-service/jupyterlab_ports.yaml, though. Also, the sample AI workload will generate a warning concerning GPU NVIDIA GB10 which is of cuda capability 12.1 and Minimum and Maximum cuda capability supported by PyTorch is (8.0) - (12.0): that can be ignored.