Local AI Server¶
Hardware¶
| Item | Notes |
|---|---|
| Main server | Pop!_OS / Ubuntu |
| GPU 1 | RTX 5070 primary |
| GPU 2 | RTX 4070 |
| RAM | 256 GB |
| Storage | 2 TB SSD |
Services¶
| Service | Purpose | Notes |
|---|---|---|
| Ollama | Local LLMs | Used for conversation and fallback logic |
| Faster Whisper | Speech-to-text | Running on GPU |
| Piper | Text-to-speech | Separate Mary/David voices |
| Qdrant | Memory/vector store | Assistant memory |
| Node-RED | Automation routing | Main voice/action logic |
| Portainer | Docker management | Container visibility |
Things to document¶
- Container names
- Ports
- Compose files
- Model versions
- What is running on which GPU
- Known-good startup commands