title: I need another gpu

the main script #

shorts #

spinning up a brev.dev instance #

I find that the easiest way to run another Linux instance with a GPU is to use Brev.dev. Brev makes it super easy to create the instance and then it just starts. Sometimes there can be an issue, but no where near as often as there is with GCP or any other hosting platform out there. Somehow they just know where the available GPUs are in the world. So we can go to Brev, and click the new instance button. Choose a GPU and give it a name. Choose spot for a cheaper box that could disappear in a bit. Then click deploy. Within less than a minute, you will have a box you can login to with all the gpu drivers all ready to go. Now just run brev shell --host and your instance name, and just like magic you are logged in. It’s using SSH but there is no manual key exchange you need to do. It’s pretty awesome. That’s why I find that the easiest way to run another Linux instance with a GPU is to use Brev.dev.

adding a remote host to your tailscale network #

I think Tailscale is pure magic. I have this Linux instance I just created with brev.dev. It’s actually on GCP, but Brev simplifies everything. Now I want to make it even easier to access the machine like it’s on my home network. To do that I use Tailscale. Tailscale makes all the hard stuff with accessing your remote boxes super brain dead easy. Let’s do a deep dive into how this works…I can’t do that because as far as I can tell, this is the work of pixies or fairies or some other fantasy stuff. I run a simple shell script on the remote box and that let’s me connect to my network. Now the host shows up in my Tailscale dropdown and I can just access an ollama server as if it were in my house. This is going to be very hard for you to recreate on your own because I think Tailscale is pure magic.

Connecting to a remote ollama host from your client #

Running ollama against a remote host is incredibly easy to do. you just have to remember to use ollama_host. On the server that’s the environment variable to set to get the host to know it needs to accept connections from all IPs. We will set it to 0.0.0.0 port 11434. On the client that’s the environment variable to set to get the client to know what server to connect to. We can set that to http://servername port 11434. If your server isn’t actually on your network, consider looking into Tailscale to get it to feel like it is there. Then running ollama against a remote host is incredibly easy to do.