A/B Testing Without the Overhead

By Joel Dare - Written May 5th, 2025

When most people think of A/B testing, they imagine sophisticated tools like Optimizely—complete with dashboards, metrics, and multivariate experiments. Those tools are great, but they come at a cost: subscription fees, extra code, third-party cookies, and a hit to user privacy.

Right now, I’m not looking to add that complexity or expense to my workflow. But I do need to A/B test—at least in a lightweight way.

Recently, I’ve been selling a niche product on Etsy: customized levels designed to help with laser engraving on tumblers. Early on, sales were slow, but with some tweaks—photos, titles, and descriptions—they picked up. Then, unexpectedly, sales flatlined.

I had no idea what had changed.

That’s when I realized I needed a simple versioning system to track my own changes. Etsy doesn’t keep a history of your titles or descriptions, so if your changes tank performance, there’s no built-in way to trace it back.

My solution: use Git.

I keep a local Markdown file (joel/sandbox/abtest/tumblertrick.md) where I log the exact titles and descriptions I post. Each version is committed to Git. That way, I can run git log to see exactly when I made changes and what those changes were.

Here’s the structure I use:

# High-Visibility Level for Tumblers - Perfect for Laser Engraving

Place this level on your tumbler and rotate until it's level. Helps ensure precise laser engraving. Available in Standard and MAX versions.

I don’t commit notes or observations—just real, published changes. That keeps the Git history clean and useful for correlation. If sales drop again, I’ll know whether it was a title change, a photo tweak, or something else entirely.

This isn’t fancy. But it works—and it respects my budget, my privacy, and my customers.