Ollama is an open-source project that enables running of large language models locally with minimal effort. I have a Windows machine with an Nvidia GPU, and I wanted to use it as an Ollama local server for a variety of tasks. One of the use case is to use it as a remote server at my home via the same WiFi network. This will allow me to consume its REST API from my mac. I followed the below-mentioned steps:

On host side

Installation of Ollama

1. Install Ollama on the sytem (Windows machine in my case) using standard installation from here.
2. Once installed, then we can use it via CLI. For the same, open the git-bash or similar CLI tool. Check the version to make sure that its correctly installed: ollama --version.
3. Ollama works (in some way) similar to Dokcer. Therefore, we can assume that each ML model is similar to a docker image. We can check existing model (will be empty as we haven’t pulled any model): ollama list. Similarly, we can check any running model via ollama ps.
4. Now let’s pull a model. For the same, ollama’s model library page is a good start as it highlights all avaialble models. We will try deepseek-r1 model. We will use default DeepSeek-R1-Distill-Qwen-7B model. To pull the model on our machine: ollama pull deepseek-r1:7b. We can double check model after the download via ollama list. We can also check model’s configuration: ollama show deepseek-r1:7b.
5. Now, let’s ask model something: ollama run deepseek-r1:7b "What is the meaning of life?". This will provide the output on the terminal.
6. We can also start a session separately and start the chat: ollama run deepseek-r1:7b. In case, you want to exit from the session then simply type \bye.

Starting the server

1. Quit the ollama completely. And from terminal:

   export OLLAMA_HOST=0.0.0.0
   ollama serve
   

2. Double check if we can query the model:

   curl http://localhost:11434/api/chat -d '{
     "model": "deepseek-r1:7b",
     "stream":true,
     "messages": [
    { "role": "user", "content": "What is the meaning of life?" }
     ]
   }'
   

3. As we want to access the served model(s) from a different machine which is on the same WiFi network, therefore, we will require host’s IPv4 address. From the terminal check out the IP address via ipconfig and grab the IPv4 Address from the section for WiFi (typically in Wireless LAN adapter WLAN). With this IP address of Windows machine, we should be able to access Ollama from another machine. In any case, following steps on client machine doesn’t work then ypu might required to configure the firewall settings. This could include changing the Windows Defender Firewall and creating an inbound rule for the given TCP port (11434 in case of ollama).

On client side

1. Open the terminal and check the access via same command as above (replace the IPv4 address):

    curl http://192.168.178.30:11434/api/chat -d '{
    "model": "deepseek-r1:7b",
    "stream":true,
    "messages": [
        { "role": "user", "content": "What is the meaning of life?" }
    ]
    }'
   

2. You can check out various REST API calls from here. Some of the calls are as follow:

   a. Check out tags: curl http://192.168.178.30:11434/api/tags

   b. Pull a new model: curl -X POST http://192.168.178.30:11434/api/pull -d '{"model": "codellama:34b", "name": "codellama:34b"}'

   c. Chat with streaming True:

    curl http://192.168.178.30:11434/api/chat -d '{
    "model": "llama3.2:1b",
    "stream":true,
    "messages": [
        { "role": "user", What is the meaning of life?" }
    ]
    }'
   

3. Finally, we can use python with ollama or langchain. Follwing example is with langchain:

from langchain.chat_models import ChatOllama

# Define the Ollama server with the correct host
ollama_llm = ChatOllama(
    base_url="http://192.168.178.30:11434",  # Replace with your server's IP
    model="deepseek-r1:7b",  # Ensure this matches your available model
)

# Run a simple query
response = ollama_llm.invoke("What is the meaning of life?")
print(response)

Connect with me on LinkedIn for any questions!