Running large language models on your own systems can be a good choice for FileMaker teams that want more control over privacy, infrastructure, and their long-term AI setup. With a local deployment, you do not have to send prompts or business data to outside providers. Instead, you can handle embedding generation, text generation, query generation, and retrieval-augmented generation (RAG) within your own environment.
However, having this control also brings some challenges. Setting up local LLM infrastructure is not a simple add-on for most teams. If you are considering using it with Claris FileMaker, here are some important factors to keep in mind before you begin.
Understand what “local” actually needs to support
A local AI model server isn’t just responsible for chat responses. Depending on your architecture, it may manage several distinct workloads:
- Text generation
- Query generation
- Embedding generation
- Retrieval-augmented generation (RAG)
Embedding generation and RAG add additional tasks for your AI system. Rather than merely creating responses, the system might need to convert source content into vector embeddings, store or search those embeddings, identify the appropriate context, and then deliver a well-supported answer. This requires more computing power and increases the chances of slowdowns or errors.
Therefore, when you move beyond simple prompt-and-response tasks, you are not just running a model on your system: you are managing a full AI service layer.
Separate the AI Server from FileMaker Server
A critical requirement is to keep your AI Server separate from your FileMaker Server.
There are several reasons why this separation is vital. First, LLM and embedding tasks can consume substantial resources and may be unpredictable, especially with multiple users. If these processes compete with FileMaker Server for CPU, memory, or disk space, your main application could slow down or even crash.
Second, separating the AI layer simplifies scaling and troubleshooting. If the model server requires more GPU, memory, or adjustments, you can implement those changes without affecting your primary FileMaker environment. Additionally, if the AI service encounters issues or needs maintenance, it won’t bring down your entire system.
For most real-world deployments, treating the AI layer as an independent service rather than just an add-on to your database server is advisable.
Plan for significantly more infrastructure than expected
Many assume a local LLM setup will operate efficiently on basic hardware, but our testing shows this isn’t true once embedding generation and RAG come into play.
These tasks demand substantial processing power. The smallest server that reliably handled our workload included:
- 4 NVIDIA T4 GPUs
- 48 vCPUs
- 192 Gb of memory
This is considerably more than most FileMaker teams anticipate when thinking about ‘local AI.’ Planning your infrastructure early is crucial, especially before your team begins building features requiring local inference.
If you plan to implement features such as semantic search, knowledge retrieval, internal document Q&A, or other RAG-based tasks, hardware sizing must be considered up front. This decision is essential for assessing project feasibility.
Do not underestimate hosting costs
Hosting your AI locally may reduce reliance on external vendors, but it doesn’t necessarily save money. Based on the server profile above, AWS hosting costs were about $3,000 per month during our tests. This figure alone should prompt serious business discussions.
For some organizations, privacy, control, and compliance benefits justify the expense. For others, a managed model provider might still be the preferred choice.
The key question isn’t whether local hosting is cheaper than API calls; it’s which cost structure aligns best with your usage, risk appetite, and technical capabilities.
Think beyond setup; focus on operations
Establishing a local model server is only the initial step. To be truly ready for operational use, you must also consider:
- Monitoring and alerting
- Model lifecycle management
- Capacity planning
- Security hardening
- Backup and recovery strategies
- Update procedures for embeddings, source documents, and retrieval pipelines
This is particularly critical if your FileMaker users depend on the system for essential business tasks. A setup that works smoothly in testing but is difficult to maintain in production can become more of a hindrance than a help.
The new admin console capabilities significantly simplify deployment, making it easier for teams to experiment and set up initial configurations. However, ease of setup doesn’t equate to reduced complexity overall. While the interface streamlines deployment, infrastructure needs, especially for embeddings and RAG, still require careful planning.
In practice, the admin console enables quicker proof-of-concept development, but careful planning for performance, service separation, and overall cost remains essential.
Conclusion
Local LLMs for Claris FileMaker are an excellent option if privacy, control, or internal knowledge workflows are priorities. They allow you to handle embedding, text, query generation, and retrieval-augmented tasks without transmitting sensitive data externally.
However, operating these systems isn’t straightforward. Once embedding and RAG workflows are involved, more powerful hardware, higher operational costs, and clear separation between the AI Server and FileMaker Server are necessary.
For teams considering this approach, the critical question isn’t just “Can we run local models?” but “Do we have the right technical, financial, and operational setup to manage them effectively?”
