Let's clear one thing up first: when I say AI tech stack, I'm not talking about Claude Code or Cursor. Those are AI tools for development. I'm talking about the tech stack that runs agents in production! I had to say it because I've run into this confusion when talking to people, and I get why it happens. Now that we're clear on that, let's move on!
Storytime
I have a confession! I actually downplayed AI products for a long time. I went with the narrative that "it's just an AI wrapper!". In the summer of 2024, I created my first hobby chatbot with the DeepSeek API just to flex on my friends. After that, I experimented with AI on and off in my spare time. I was never serious about it back then. And when I saw people come up with AI products, I was never impressed, as I'd already worked out the whole process in my head. If I can visualize how we make things, it shouldn't be hard, am I right?
The indifference
Fast forward to 2025, the company I worked for created an AI division and months later, we had our first product: an internal AI that uses our own data. It was a chatbot you could ask questions and that would answer accordingly.11 well, I'll tread carefully here because I don't know to what extent this is an industry secret! I was indifferent about it. Soon enough, we integrated AI on many parts of our platform. I myself integrated several parts of it. I knew our tech stack but I didn't pay much attention.
The experiment
That is, until 2026, when I had an idea! Since AI had gotten so good, why wasn't I using it yet to manage my life? I began to create my first chatbot: a chatbot to manage my gym sessions. I wanted to create a native app but then scaled back due to time. So I decided to use Telegram instead. After a few prompts, I got Claude to build my chatbot. I used my remaining DeepSeek credits to play with it. The result was well... underwhelming. The texts were robotic despite the tweaks I made to the prompts. That's when I decided to plug in the Grok API alongside DeepSeek, as I had tokens there too. Results got better, then I got error upon error. Bad luck, I guess, but I reasoned that it might be a temporary server crash. Which made me purchase some Claude API tokens. It worked, and after some more prompt tweaking, the results were what I needed.
And now we're all set! I'm a cheap kind of guy, so even though I had my results with Claude, I still put Grok and DeepSeek as fallbacks in the config files.
I then purchased a Hetzner server and put it online.
If it works for a Coach bot, why not also create an Interview bot to train myself?
The issue is that an Interview bot only works best if I can talk to it by voice. So it needs speech recognition. Claude's API doesn't do speech-to-text. OpenAI's does (Whisper). So that's how I bought some OpenAI tokens.
I made a few tweaks to the system prompts and deployed it on the same server as the Coach Bot. I didn't actually set up voice transcription, only text, planning to do voice later.
Then another idea came in: a Finance bot to manage my finances! Yes, all of that in an afternoon!
So I duplicated the initial repo once again and tweaked the prompts for finance. The accountant bot would need me to share invoices and screenshots. This one worked pretty much out of the box. Nothing too crazy.
There it was: in one afternoon, I got myself three chatbots. I tested them. I mean, with zero SDLC conventions. They were vibecoded and they worked!
Then, the pain points manifested progressively!
I was at a networking event. Beyond the Madagascar jokes the homies were throwing at me and asking whether King Julien is still alive, we actually had serious conversations about AI. This is where I came in with the half-baked projects I'd cooked up in one afternoon: the Telegram chatbots.
Stranger: Wow! So you can manage your things with it?
Me: Yes! Look, you can see all my workouts logged here, and if you do this [typing on the keyboard to test]... oh wait....
What happened is that, somehow, it had gone back to the original setup with em dashes and robotic, repetitive answers!
Me: Seems like the model has been downgraded!
Stranger: Which LLM are you using?
Me: Claude, if memory serves, but you can check with /models here and see which one is active.
Stranger: So you can switch models, right?
Me: Uh, yes, but I need to do it in the source code... oh wait, I see a command available here: /models change DeepSeek
Coach Bot: I don't know what that means!
Me: Dude, you're embarrassing me in front of the wizards!22 I knowww, cringey pop-culture reference
It turned out the bot documented a feature it didn't have! I hadn't noticed before. But what do I know? I VIBECODED THE WHOLE THING!!!
Defeated by humiliation, I went home and tried to fix the bots. Changed some prompts. Checked whether the models were working: they were. The real pain point was realizing that I had to push, then log into Hetzner and pull to deploy. Okay, the lack of a CI/CD pipeline is on me! But having to push every time I changed a prompt, though!
And when I tested the prompts, I was messing up my Telegram threads and had to clean them up and start from scratch!
My reasoning through all of this was that I only had to do it once! Make it right once and never again!
So after some tweaks, things seemed to work fine for several weeks. Then the beginning of the next month came! On September 1st, I decided to log an expense. And this happened!


To summarize, Odile the Accountant Bot cleared my balance at the start of the month. Okay, maybe that's one way to manage money, but it's not how I do it! I tried to convince her to change her method, but to no avail! So basically, what should I do? Change it in the prompt, of course. Which meant doing it all again: change prompts, test on Telegram, push the code, open Hetzner and pull (yes, I get it, I need to put CI/CD on that thing!).
And Greg the coach bot didn't do any better. It fell back to em dashes and repetitive sentences for no reason, even though I had explicitly told it not to a month earlier! And it asks for summaries on weird dates.

My moment of realization
Okay, so what did I learn?
A proper CI/CD pipeline is still beneficial, even for hobby projects (spoiler: I still won't set it up for the next two months).33 Actually, I'll have to set it up myself because it's an API of some sort. For web apps, and for my website, I use Vercel, which has it baked in from day one, so you don't have much to do yourself.
Tweaks will always be needed eventually. Rules change. Demands evolve. I need a proper place to run evals on the tweaks without affecting prod.
The classic development lifecycle still matters (dev -> staging -> prod, plus a testing strategy), even for hobby projects. You can one-shot everything, but you should be familiar enough (emphasis on "enough") with the code as a whole to understand when things break, even without looking at it much. At the very least, you won't be surprised by features you didn't request. It's not so much about owning the code as about owning the product.
Also, switching models should be easy! And checking my token consumption shouldn't require visiting each LLM provider.
I could have vibecoded the whole setup, but there are great open source tools out there for this. I know how my company does things, and I dug further with the team lead on the parts that weren't clear to me. So I got the whole thing figured out. I won't go into how the company does things or its full tech stack, since I don't know how much I'm legally allowed to share. But I can set things up with what my project actually needs. And since I'm cheap anyway, I can't afford what a multi-million dollar company can, so I need alternatives.
My answer to those problems
So I went with this very simple setup:
- Telegram as UI: I've used it since the start, and this one won't change
- LiteLLM: A must-have I didn't even know I needed! It centralizes LLM APIs behind a single virtual key that can access any model you plug into it. It can route from one model to another, and you can set budgets per key or team so you don't wake up to a €80,000 token bill because of a hack!
- Langfuse44 no, it's not LangChain! The similarity in the name is only a coincidence. I went to a meetup with LangChain and the guys seemed annoyed when people confused them with Langfuse lmao: It handles further cost tracking, as well as tracing multi-step agent workflows if you need it. Prompt management and evals are the key features I use it for.
At the time of writing, I've already installed them but haven't wired them up properly yet, for a specific reason:
I actually have a better idea now! Instead of using 3 different projects, I'm now creating one big app where we can create as many bots as we need. Some will need a voice API, others won't. Same with images and all. So this is my next step! We'll try to optimize costs by giving each bot only the models it needs.
I'm planning to post about the progress here! We should get an alpha version by the end of this year if I'm not too busy. For now, stay tuned!
Notes
well, I'll tread carefully here because I don't know to what extent this is an industry secret!
↩I knowww, cringey pop-culture reference
↩Actually, I'll have to set it up myself because it's an API of some sort. For web apps, and for my website, I use Vercel, which has it baked in from day one, so you don't have much to do yourself.
↩no, it's not LangChain! The similarity in the name is only a coincidence. I went to a meetup with LangChain and the guys seemed annoyed when people confused them with Langfuse lmao
↩
Did this post speak to you?
