Generative AI · Conversational memory
My Best Self (MBS)
The AI pipeline behind a chatbot platform with configurable bots: memory, retrieval, model fallback and overnight jobs.
- Status
- Beta on Google Play
- Where
- Innovizia Technologies · Dec 2025 – present
- My role
- Own the core AI pipeline
- 2
- model providers, with automatic fallback
- 3
- memory layers: summary, database, vector
- Overnight
- batch jobs keep daytime capacity for live chats
Overview
MBS is a generative AI chatbot platform with configurable bots, in beta testing on Google Play ahead of rollout to its intended users. The app is built by a product team; I own the AI pipeline that sits behind every conversation.
- 01 · Input User message Routed to a configured bot
- 02 · Recall Short-term memory Running conversation summary
- 03 · Retrieve Long-term + RAG PostgreSQL history; ChromaDB vectors, only when needed
- 04 · Generate GPT-4o mini Primary model
- 05 · Fallback Gemini On outages, rate limits or other failures
- 01 · Schedule Celery Beat Jobs queued at night
- 02 · Queue RabbitMQ · Redis Broker and results
- 03 · Create Feed content Articles, quotes, poems
- 04 · Consolidate Long-term memory Updated from the day's conversations
What I did
- Integrated GPT-4o mini as the primary model with automatic Gemini fallback.
- Compared models and prompt configurations across conversational use cases.
- Implemented RAG so specific bots get their own dedicated context, using ChromaDB, Nomic embeddings and LangChain.
- Built short- and long-term conversational memory with summarisation, PostgreSQL persistence and vector retrieval; schema migrations with Alembic.
- Scheduled overnight background jobs with Celery Beat, RabbitMQ and Redis; containerised services with Docker and Nginx.
Local inference experiments
I tested self-hosted inference with vLLM and Llama models, and investigated prompt injection, guardrails and where small models fall short for this use case.
Engineering decisions
Fail over, don't fail
When the primary model has an outage, hits a rate limit or errors out, the request moves to Gemini automatically. The user just gets a reply.
Retrieve only when it helps
Embeddings of a user's past issues live in ChromaDB, and the bot pulls them in only when relevant. Context stays small and responses stay fast.
Do the heavy work at night
Generating feed content and consolidating long-term memory run as overnight jobs, so daytime AI capacity is reserved for live chats.
Stack
- GPT-4o mini
- Gemini
- LangChain
- ChromaDB
- Nomic embeddings
- PostgreSQL
- Alembic
- Celery Beat
- RabbitMQ
- Redis
- Docker
- Nginx
- vLLM