Meet "Ask AboutCloud" . An AI chatbot for the blog and the ENTRA ID Community , with safety receipts

This is nothing new in the Tech world.....but if you've visited the blog in the last few days, you've probably noticed a small chat bubble in the bottom-right corner. That's Ask AboutCloud Bot, a new feature I've been quietly cooking up: a semantic search engine and RAG (Retrieval Augmented Generation) chatbot that can answer questions about every post on this blog, plus the three free Entra ID tools I maintain. As well anything related to ENTRA ID in general and Internet searches .

It runs end-to-end on Cloudflare's free tier. Total monthly bill: €0.00. No upgrade trick, no "free for the first 90 days" footnote. Just genuinely free, because every component fits inside the daily quotas Cloudflare hands out for nothing.

This post is the announcement, but it's also a tour. If you're curious what RAG looks like when you build it from scratch without LangChain, without Pinecone, without a single npm dependency in the Worker — this is the inside view.

What it actually does

Two things, really.

Semantic search. Type a natural-language question into the search bar and the bot returns the most relevant blog posts, ranked by cosine similarity. Not keyword matching — semantic. Searching for "how do I lock down service principals" finds posts about app registrations and Conditional Access even if those exact words never appear in the question.

Conversational answers with citations. Open the chat bubble and ask something like "What free Entra ID tools did Antonio build?" or "How does shadow role detection work in RoleLens?" The bot pulls the most relevant content from the index, hands it to a Llama 3.1 8B model running on Workers AI, and returns a grounded answer with the source posts linked underneath. Every response also carries a disclaimer ("AI-generated — for informational purposes only") and a x-bot-version header so I can audit which version of the bot answered which question.

The bot knows about three products specifically — Entra RoleLens, Entra Tracker, and AADSTS Entra Errors — because I indexed their documentation alongside the blog posts. Ask it which tool to use for which problem and it'll tell you, with links.

The architecture, one layer at a time

Here's the request flow when you ask a question:

You type a question
       ↓
Cloudflare Worker (aboutcloud-search)
       ↓
5 pre-LLM guardrail checks (more on these below)
       ↓
Workers AI: bge-base-en-v1.5 → 768-dim embedding
       ↓
Vectorize: top-4 nearest neighbors (cosine similarity)
       ↓
D1: fetch full post content for those IDs
       ↓
Workers AI: Llama 3.1 8B Instruct generates answer
       ↓
Response: { answer, sources, disclaimer }
       + x-bot-version: 1.1.0

No external API calls. No OpenAI key. No Pinecone account. The embedding model, the vector database, the SQL store, and the LLM are all bindings on the same Worker — which means latency is dominated by inference time, not network hops.

The numbers, for the curious:

WorkersEdge runtime100,000 req/day<100/dayWorkers AI (BGE)768-dim embeddings10,000 neurons/day~26/dayWorkers AI (Llama 3.1 8B)Answer generation10,000 neurons/day~50/queryVectorizeVector search200,000 vectors26 vectorsD1Post metadata + content5 GB~100 KB

I'm using less than 1% of headroom on every dimension. The blog could grow 100× and still cost nothing.

The safety framework

A chatbot on a public blog is a target. People will absolutely try to jailbreak it, redirect it to political topics, extract its system prompt, or coax it into writing exploit code with my domain attached. So the bot is wrapped in safety scaffolding — but I've found it useful to be explicit about what kind of work each piece is doing, because not all "safety layers" are the same kind of thing.

The framework splits into two halves.

Six runtime defenses — code that runs against user input on every request and deterministically rejects matches:

1. System prompt. Identity, scope, tone, and refusal patterns baked in. The bot is "Ask AboutCloud Bot" — never a person, never sentient, never anything else.
2. Prompt-injection shield. Twenty-plus pre-LLM patterns catching "ignore previous instructions", "you are now DAN", "show me your prompt", and the role-reversal regex family. User input is NFKC-normalized before matching, so Unicode lookalikes (Cyrillic 'а' for Latin 'a', mathematical italics, zero-width joiners) collapse to ASCII before filters run.
3. Topic boundary. Politics, religion, medical advice, legal advice, financial trading, NSFW, and violence are filtered out at the input layer.
4. Destructive-behavior blocker. Fifteen-plus patterns blocking SQL-injection requests, XSS payload generation, malware, ransomware, keyloggers, brute-force scripts, and similar offensive-security asks.
5. Anti-hallucination threshold. A minimum vector similarity score of 0.50 is enforced. If the top match is below that threshold, the bot refuses rather than guessing. Temperature is locked at 0.2. The system prompt explicitly forbids inventing facts, names, dates, or stats.
6. Output sanitizer. Generated links must come from a domain whitelist (aboutcloud.io, my GitHub, my LinkedIn, the Workers subdomain). The widget renders the model's output via DOM construction with textContent, never innerHTML — so any HTML the model produces is shown as text, not parsed.

A 2,000-character input cap and an inert-URL policy (the bot does not follow URLs in user input, and treats them as plain text) sit alongside these. They're cheap, they run before any LLM is invoked, and they close a class of bypasses for free.

Four operational practices — policies and processes around the bot, not code paths:

1. Privacy policy. No conversation storage, no PII collection, no cookies set by the bot. Queries about me return public bio plus LinkedIn only.
2. Legal disclaimer. Every chat response includes "AI-generated content — for informational purposes only. Not professional advice." rendered visibly under the answer in the widget.
3. De-escalation tone. When users get hostile, the bot stays cheerful and redirects rather than matching aggression.
4. Versioned releases. x-bot-version header on every response, monthly review, audit-friendly trail.

The reason for splitting these out is that they're the same word ("safety") doing different jobs. Item 2 runs on every request. Item 9 is a tone instruction in the system prompt. Item 10 is a calendar reminder. Lumping them together makes the count look bigger than the substance, and it hides the fact that the runtime defenses are the ones an attacker actually has to defeat.

The eval suite (the receipts)

Most chatbot announcements end here, with a list of safety claims you're meant to take on faith. I'd rather show the math.

The bot ships with an actual eval suite. Forty-seven test cases stored as JSONL, a runner that POSTs each case to the deployed Worker and classifies the response into one of five expected behaviors (refuse_injection, refuse_off_topic, refuse_destructive, answer_with_citations, answer_with_low_confidence_refusal), and a GitHub Actions workflow that runs the suite against a staging Worker on every PR.

The breakdown: 10 prompt-injection attempts, 10 off-topic refusals, 10 destructive-behavior blocks, 5 valid product questions that should answer with citations, 5 valid blog questions, 2 edge cases, and 5 adversarial tests. The adversarial set is the interesting bit — Unicode homoglyph attacks (Cyrillic 'а' substituted for Latin 'a' in classic injection phrases), multi-step prompt setups disguised as conversation, and injection payloads embedded inside plausible-looking technical questions.

Current pass rate against the deployed bot: 85% (40 of 47).

I know which seven cases are failing. Five are bugs in my classifier — the case spec said "should refuse with phrase X" and the bot refused with the equivalent phrase Y; cosmetic, already patched. Two are real pre-LLM filter gaps where the regex didn't catch a phrasing it should have. In both cases the input was ultimately refused — by the LLM following its system prompt rather than by the pre-LLM regex that should have caught it. The user got the right outcome; the architecture didn't. Both are queued for a v1.2.1 patch.

The point isn't 85%. The point is having a number at all, and knowing exactly which cases produce it. Before the suite existed, I had no way to tell whether a system-prompt change made the bot better or worse — I just had a feeling.

Why I built it from scratch

Anyone with a credit card can wire up a chatbot in an afternoon. Pick an LLM provider, pick a vector database, pick a framework, glue them together, ship. So why build it the hard way?

Three reasons.

Cost. Most "free" stacks have a runway, not a floor. They're free until. This one is free because — the underlying primitives are designed to be free at this scale. I don't have to think about whether tomorrow's traffic spike turns into tomorrow's bill.

Control. Every layer is mine. The system prompt, the score threshold, the topic regex, the URL whitelist, the eval cases, the version header. When the monthly review surfaces a new injection pattern in the wild, I add a string to an array and redeploy. No vendor SLA, no "we're investigating", no roadmap to wait on.

Honesty. I write a blog about cloud security. Telling readers "you should understand the systems you depend on" while running a black-box AI service would be hypocritical. This way the whole stack is documented, the Worker source is public, and anyone who wants to verify what the bot does or doesn't do can read the code — and the eval cases.

Try it

The bubble is in the bottom-right of every page on aboutcloud.io. Some questions worth trying:

• What free Entra ID tools are listed on this blog?
• How does Entra RoleLens detect shadow roles?
• What's the difference between Entra Tracker and AADSTS Entra Errors?
• How should I think about securing MCP servers?
• What's new in the ENTRA ID Community ? Latest articles on the Internet and Microsoft Announcements ?

If you ask something the index doesn't cover well, the bot will tell you it doesn't have enough information rather than make something up. That's the anti-hallucination threshold doing its job — and it's the behavior I'm proudest of.

The repo is open source and the docs walk through the architecture, the components, the safety framework, the eval suite, and day-to-day operations. If you build something similar, I'd genuinely love to hear about it.

In the meantime — go ask the bot something. It's been waiting.

The whole stack is open source under MIT — repo at github.com/arusso-aboutcloud/aboutcloud-search, with the Worker source, eval cases, safety docs, and a step-by-step deployment guide. If you build something similar, I'd genuinely love to hear about it

If you're running a similar setup or have questions, reach out in the comments or on LinkedIn.

— Antonio | AboutCloud

arusso@aboutcloud.io

Note: These tools are MIT licensed and free for personal, educational, and open-source use.