Installing Local LLM for FileMaker – Made Simple!

---

1. Create Ubuntu Server

Below are instructions for creating an Ubuntu 24.04 server using AWS EC2. These instructions should be similar for other cloud services like Microsoft Azure.

Launch a New EC2 Instance

1. In AWS EC2, click Launch Instance
2. Select the following options
   • AMI – Ubuntu 24.04 (x86)
   • Instance Type – t3a.large (or your preferred size)
   • Key Pair – Select or create a new key pair
   • Subnet – Choose an appropriate subnet (e.g., us-east-1c)
   • Security Groups – Ensure SSH (port 22), HTTP (port 80), and port 8080 are open
   • Storage – ~30GB recommended
   • IAM Role – If you have an instance profile with needed permissions, select it here (optional but recommended)

Create and Associate an Elastic IP

1. Reserve a new Elastic IP in AWS
2. Associate it with your new EC2 instance
3. In your DNS settings, create an A record pointing to the Elastic IP

Connect to the EC2 Instance

Use AWS Systems Manager Session Manager or SSH

---

2. Update Ubuntu

After connecting, update your instance

sudo -i

sudo apt update && sudo apt upgrade -y

sudo apt dist-upgrade -y

sudo apt autoremove -y

sudo shutdown -r now

Reconnect after the reboot.

---

3. Download Open Source LLM Files

📌 Important – If you’ve upgraded from FileMaker Server v20, ensure you have the latest FileMaker Server v21 version of the Open Source LLM files. 🔗 Download Open_Source_LLM.zip Download directly to your server

cd ~

sudo apt install zip

sudo wget https://kyologic.com/wp-content/uploads/2025/02/Open_Source_LLM.zip -O /home/ubuntu/Open_Source_LLM.zip

sudo unzip Open_Source_LLM.zip

Set Read & Write Permissions to letsencrypt directory

sudo chown -R ubuntu:ubuntu /home/ubuntu

sudo chmod -R a+rx /home/ubuntu

Remove Unneeded Files

sudo rm Open_Source_LLM.zip

sudo rm -r __MACOSX

OR Transfer the files to your server from your computer

scp -i "/local/path/to/keypairfile" -r /local/path/to/Open_Source_LLM ubuntu@your-server-ip:/home/ubuntu/

---

4. Install Certbot for SSL

Use Certbot to manage SSL certificates

cd ~

sudo snap install core && sudo snap refresh core

sudo snap install --classic certbot

sudo ln -s /snap/bin/certbot /usr/bin/certbot

sudo shutdown -r now

Reconnect after the reboot.

---

5. Configure Firewall

cd ~

sudo ufw allow ssh

sudo ufw allow 80/tcp

sudo ufw allow 8080/tcp

sudo ufw enable

---

6. Obtain an SSL Certificate

Ensure DNS is pointed to your server’s IP before running the following Note – Replace llm.mydomain.com with your domain name.

sudo certbot certonly --standalone --preferred-challenges http -d llm.mydomain.com --dry-run

sudo certbot certonly --standalone --preferred-challenges http -d llm.mydomain.com

Enable auto-renewal

sudo systemctl enable snap.certbot.renew.timer

  Configure Hooks for Auto-Renew Certbot needs to open port 80 temporarily when renewing. Create pre- and post-hook scripts

cd /etc/letsencrypt/renewal-hooks/pre

sudo nano pre-hook.sh

  Contents of pre-hook.sh

#!/bin/bash
# Open port 80
ufw allow 80/tcp

Save (Ctrl+O, Enter) and exit (Ctrl+X). Then make executable

sudo chmod +x pre-hook.sh

cd /etc/letsencrypt/renewal-hooks/post

sudo nano post-hook.sh

  Contents of post-hook.sh

#!/bin/bash
# Close port 80
ufw deny 80/tcp
#Reboot Server
sudo shutdown -r now

Save (Ctrl+O, Enter) and exit (Ctrl+X). Then make executable

sudo chmod +x post-hook.sh

  Test renewal

sudo certbot renew --dry-run

This should reboot your server

---

7. Generate Admin PKI Keypair & JWT for Authentication

1. Download Soliant’s SSH Key & JWT Tool
2. Open in FileMaker
3. Create a new record
4. Enter a Name & # of expiration days
5. Press the Generate Keypair & Generate JWT buttons

Add Admin PKI Public Key file

sudo mkdir /etc/letsencrypt/auth_key

sudo nano /etc/letsencrypt/auth_key/localllm.key.pub

Copy the Public Key from SSH Keys & JWT to your clipboard. Paste the Public Key in the localllm.key.pub file. Save (Ctrl+O, Enter) and exit (Ctrl+X).   Add Admin PKI Private Key file

sudo nano /etc/letsencrypt/auth_key/localllm.key

Copy the Private Key from SSH Keys & JWT to your clipboard. Paste the Private Key in the localllm.key file. Save (Ctrl+O, Enter) and exit (Ctrl+X).   Set Read & Write Permissions to letsencrypt directory again

sudo chown -R ubuntu:ubuntu /etc/letsencrypt

sudo chmod -R a+rx /etc/letsencrypt

---

8. Install Miniconda & Python Environment

Install Miniconda

cd /home/ubuntu

mkdir -p ~/miniconda3

sudo wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh

bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3

sudo rm ~/miniconda3/miniconda.sh

source ~/miniconda3/bin/activate

conda init --all

Create & activate Python environment

conda create --name <name>

conda activate <environment name>

conda install python=3.9.18

pip install jwt && pip uninstall PyJWT && pip install PyJWT

conda install -c conda-forge sentence-transformers

Install Open Source LLM dependencies

pip install -r /home/ubuntu/Open_Source_LLM/requirements.txt

sudo shutdown -r now

---

9. Run & Test the Local LLM Server

Set environment variables

sudo chown ubuntu:ubuntu /home/ubuntu/ -R

sudo chmod -R 755 /home/ubuntu

sudo chmod 700 ~/.ssh

sudo chmod 600 ~/.ssh/authorized_keys

sudo shutdown -r now

export PKI_KEYFILE="/etc/letsencrypt/auth_key/localllm.key.pub"

export CERTFILE="/etc/letsencrypt/live/llm.mydomain.com/fullchain.pem"

export KEYFILE="/etc/letsencrypt/live/llm.mydomain.com/privkey.pem"

conda activate <environment name>

Run python

python3 /home/ubuntu/Open_Source_LLM/server/fm_LLMOS_StartServer.pyc

📌 Test connection using the Claris Academy AI Fundamentals demo file.

• Open Meetings_Solution in FileMaker
• On the initial layout ( Meeting Details), enter Layout Mode and remove the Layout Calculation at the bottom of the page and add the field Details_Embedding to the layout.
• Open the Script Workspace and find the Configure AI Account script
  • Adjust the Configure AI Account script step
    • Model Provider – Custom
    • Endpoint – https://llm.mydomain.com:8080
    • API key – copy and paste the JWT from SSH Keys & JWT file
• Now Navigate to the Embedding Details Data script
  • Adjust the Insert Embedding script step
    • Embedding Model – all-MiniLM-L12-v2
• Clear out the contents of the Details_Embedding field script
• Run the Embedding Details Data script
• Check and confirm that there were no errors and you have a new .fve file in the Details_Embedding field.

Keep this file open, we will use it to run a few more tests in the upcoming steps.

---

10. Automate the LLM Server with PM2

Create a pm2.io account. Enter a Bucket Name You should now see this screen Install PM2 and link to your account If you are still in the conda command, enter ^C to exit.

sudo shutdown -r now

sudo -i

sudo apt install npm -y

npm install -g pm2

pm2 link [[Keys from PM2 Website]]

You should now see a new message on the PM2 web page saying you are linked

shutdown -r now

Start the server process with PM2

conda activate <environment name>

CERTFILE="/etc/letsencrypt/live/llm.mydomain.com/fullchain.pem" KEYFILE="/etc/letsencrypt/live/llm.mydomain.com/privkey.pem" PKI_KEYFILE="/etc/letsencrypt/auth_key/localllm.key.pub" pm2 start python3 --name <environment name> -- /home/ubuntu/Open_Source_LLM/server/fm_LLMOS_StartServer.pyc

  Test Go to FileMaker, clear Details_Embedding field Run the Embedding Details Data script Verify a new .fve file in the Details_Embedding field. Save the process & enable auto-start

pm2 save

pm2 startup

It will return a command looking something like this: sudo env PATH=$PATH:/usr/bin /usr/local/lib/node_modules/pm2/bin/pm2 startup systemd -u ubuntu –hp /home/ubuntu Copy and paste the command and press Enter

sudo shutdown -r now

  Final Test Go to FileMaker, clear Details_Embedding field Run the Embedding Details Data script Verify a new .fve file is created 🎉 Success! Your Local LLM Server is now running on Ubuntu 24.04 with automated startup.

---

🚀 See This in Action! Want a live walkthrough of setting up Local LLM for FileMaker on Ubuntu? Join our Connecticut FileMaker Developers Group (CFDG) meetup where we’ll demo this setup step by step and answer all your questions! 🔗 Register here