A founder asked us last month for "a couple of client references, and maybe some code you're proud of." We sent him one link: app.ekmire.com. Sign up, run a real scan against a real repo, see what it flags. No case study PDF, no curated screenshot, no NDA to sign first. If the product breaks or the output looks fake, he'd know in ten minutes — which is exactly the point.

Every agency site looks roughly the same: a grid of client logos, three glowing testimonials, a portfolio of screenshots that prove someone can make a page look good in Figma. None of it tells you whether the team behind it can build something that has to keep working after the invoice is paid. We think there's a better question to ask, and we think most agencies would rather you didn't ask it.

Why Portfolios Don't Tell You What You Need to Know

A portfolio answers "can this team make something look finished." It doesn't answer "can this team make something that survives contact with real users, real load, and eighteen months of feature requests." Those are different skills, and the gap between them is exactly where most agency relationships go wrong — the site launches, looks great in the handoff call, and then nobody on the agency side has ever had to fix it at 11pm because it's actually down.

Client work also has a structural blind spot: the agency doesn't own the consequences of its own architecture decisions. If a client's site gets slow under load, or a chosen database can't handle the client's actual write pattern, the agency delivers the postmortem and moves to the next project. When you run your own product, you live with every decision you made a year ago, on a Tuesday when it breaks in production. That accountability is the thing a portfolio screenshot can never show you.

The One Question That Filters Real Engineering Depth

Ask any agency this directly: "Do you operate anything yourselves, live, with real users, right now?" Not "have you built things" — everyone has built things. Operate. Present tense. Something that has to stay up, get patched, and handle a support ticket from someone who isn't a client.

Most agencies go quiet here, or point to an old side project that's been offline for a year. That's not a character flaw — running a product is a genuinely different commitment than shipping one and handing it off. But it means the team has never felt the specific pain of a bad architecture decision compounding over time, because someone else always inherited that pain.

What We Point To When Asked This

We don't have one flagship product to hide behind — we have several, at different scales, and we'll hand you the login to any of them:

• ekmire: our developer security SaaS, live at ekmire.com. Paying customers, a free tier we actually support, and a proxy component running in front of real traffic. We eat our own architecture decisions here — badly chosen ones show up as our own outage, not a client's.
• Minyut: our embeddable RAG chatbot platform, live at minyut.com. Real documents, real embeddings, real query volume from businesses that are not us.
• Flux8Shield: a free passive security scanner that's run 65,000+ real scans in production — no signup, no gate, just a live tool anyone can hit right now.
• GoPDF and GoVid: browser-based tools processing real files for strangers on the internet every day, with zero server-side handling of the file — if the client-side code is wrong, it fails visibly and immediately, in public.

This isn't a claim that in-house products make us better at client work by default. It's that every one of these products forces us to live with our own technical debt, our own scaling mistakes, and our own on-call pages. That's a different kind of pressure than a client project ever applies, and it's the pressure that actually improves engineering judgment over time.

Other Real Signals Worth Checking

If an agency can't point you to something they operate, these are the next-best signals — and they're still stronger than a portfolio page:

• Ask for a live demo, not a case study PDF: a PDF is curated after the fact. A live walkthrough of a real client project (with permission) shows you decisions being explained in real time, not narrated after the win.
• Ask them to describe a failure, specifically: not "challenges we overcame" marketing language — an actual thing that broke, why, and what changed afterward. Teams that can't name one either haven't shipped enough to fail yet, or aren't willing to be honest with you before you've signed anything.
• Ask why they chose their stack, not just what it is: "we use Next.js and Postgres" is a fact. "we use Postgres because the client's data has relational integrity requirements that a document store would fight us on" is a decision. You want to be billed by people making decisions, not reciting a tech list.

The takeaway isn't "only hire agencies with their own SaaS products" — plenty of excellent teams have never built one and are still worth hiring. The takeaway is that a glossy portfolio is the lowest-effort thing an agency can produce, and it's usually the only thing you're shown. Ask for the thing that's harder to fake: something running right now, in public, that they can't quietly take down before you look. If you want to see what we're willing to show you, our work is at case studies and our products are at flux8labs.com/products — or just reach out and we'll walk you through whichever one you're skeptical of.

---

Frequently Asked Questions

• Q: Is it a red flag if an agency has never built its own product? Not by itself. Plenty of strong agencies focus entirely on client delivery and are excellent at it. It becomes relevant only when you're specifically trying to assess technical depth and long-term accountability — in that case, an agency operating its own live product has had to live with its own mistakes in a way pure client work doesn't force.

• Q: What's the difference between a web design agency and a web development company? A design agency is primarily concerned with how a site looks and converts — brand, layout, visual polish. A development company is concerned with how it works — backend logic, integrations, data handling, and functionality that goes beyond a page rendering correctly. Many teams, including ours, do both, but the questions you ask should differ depending on which problem you actually have.

• Q: What questions should I ask before hiring a web development agency? Beyond the standard timeline and budget questions: ask what they operate themselves in production, ask them to describe a specific failure and what changed afterward, and ask why they chose their stack rather than just what it is. Answers that are specific and slightly uncomfortable are a better signal than answers that are polished.