Jay Rathod

← Selected work

Generative AI · Conversational memory

My Best Self (MBS)

The AI pipeline behind a chatbot platform with configurable bots: memory, retrieval, model fallback and overnight jobs.

Status
Beta on Google Play
Where
Innovizia Technologies · Dec 2025 – present
My role
Own the core AI pipeline
2
model providers, with automatic fallback
3
memory layers: summary, database, vector
Overnight
batch jobs keep daytime capacity for live chats

Overview

MBS is a generative AI chatbot platform with configurable bots, in beta testing on Google Play ahead of rollout to its intended users. The app is built by a product team; I own the AI pipeline that sits behind every conversation.

Live conversation
  1. 01 · Input User message Routed to a configured bot
  2. 02 · Recall Short-term memory Running conversation summary
  3. 03 · Retrieve Long-term + RAG PostgreSQL history; ChromaDB vectors, only when needed
  4. 04 · Generate GPT-4o mini Primary model
  5. 05 · Fallback Gemini On outages, rate limits or other failures
Overnight
  1. 01 · Schedule Celery Beat Jobs queued at night
  2. 02 · Queue RabbitMQ · Redis Broker and results
  3. 03 · Create Feed content Articles, quotes, poems
  4. 04 · Consolidate Long-term memory Updated from the day's conversations

What I did

  • Integrated GPT-4o mini as the primary model with automatic Gemini fallback.
  • Compared models and prompt configurations across conversational use cases.
  • Implemented RAG so specific bots get their own dedicated context, using ChromaDB, Nomic embeddings and LangChain.
  • Built short- and long-term conversational memory with summarisation, PostgreSQL persistence and vector retrieval; schema migrations with Alembic.
  • Scheduled overnight background jobs with Celery Beat, RabbitMQ and Redis; containerised services with Docker and Nginx.

Local inference experiments

I tested self-hosted inference with vLLM and Llama models, and investigated prompt injection, guardrails and where small models fall short for this use case.

Engineering decisions

  1. Fail over, don't fail

    When the primary model has an outage, hits a rate limit or errors out, the request moves to Gemini automatically. The user just gets a reply.

  2. Retrieve only when it helps

    Embeddings of a user's past issues live in ChromaDB, and the bot pulls them in only when relevant. Context stays small and responses stay fast.

  3. Do the heavy work at night

    Generating feed content and consolidating long-term memory run as overnight jobs, so daytime AI capacity is reserved for live chats.

Stack

  • GPT-4o mini
  • Gemini
  • LangChain
  • ChromaDB
  • Nomic embeddings
  • PostgreSQL
  • Alembic
  • Celery Beat
  • RabbitMQ
  • Redis
  • Docker
  • Nginx
  • vLLM