The Corner Datastore

Vibe Coding: How I found a data disclosure error by checking more than just API endpoints

Sat, 18 Apr 2026 00:00:00 -0400

There is promise in what modern-style AI code tools can do to make life easier for engineers and other technically-minded people as a companion to their knowledge and experience.

The Takeaway

It also can work well for vibe coding a proof of concept to explore ideas. If you’re non-technical and vibe coding some software, please ask at least one technical friend to take a look, even casually, at what code was generated or the final website/app.

Disclaimers

I won’t share who the founder or the startup is; it isn’t relevant.
No live end-consumer data was disclosed.
This is meant as an illustrative example, not the one specific thing you need to watch out for. Don’t just tell ChatGPT to check for this vulnerability and think you’re 100% safe.

Context

Knowing the promise of modern AI agents in making it easier to write code, I’ve been curious to see how people are using it. Engineers on my team have liked using it as a companion, filling skills gaps to help keep the work moving. I’ve been exploring it for certain development work, too. But what are startup founders doing? In particular, what are non-technical founders doing?

Non-technical founders are doing more vibe coding, and I wanted to know what kind of code is getting deployed, potentially into production, and potentially in more secure data contexts.

The Target

The target is a B2B2C web application that processes and stores PII, and also provides a front-end for authenticated business accounts to view a scoped set of that PII. It includes one demo account which has access to demo data for one business. Other than the demo business account, I did not get access to the codebase, application db, or app backend. Only the knowledge that it was vibecoded and the interface that a regular user would use. I’d call it “informal black box testing” to explore how LLM-generated systems work, not explicitly to find any problems.

The Method

Keeping this brief, rather than telling a whole story.

I loaded the public-facing website, and found that it makes a series of requests to load and then display user data for the demo business account. The request sequence looks like:

GET index
GET companies?select=*&id=eq.demo_guid
GET users?select=*,user_contact:user_contacts(email,phone_number, zip_code,birth_year,name) &company_id=eq.demo_guid
GET companies?select=id,demo_enabled&slug=eq.company_demo_name
GET companies?select=*&id=eq.company_demo_guid
GET users?select=*,user_contact:user_contacts(email,phone_number, zip_code,birth_year,name) &company_id=eq.company_demo_guid
GET users?select=*,user_contact:user_contacts(email,phone_number, zip_code,birth_year,name) &company_id=eq.company_demo_guid
GET companies?select=*&id=eq.company_demo_guid

Some funny notes:

The page fetches the data for a ‘demo’ and immediately discards it. This was probably part of some early demo/proof of concept work, before support was added for multiple companies, with each company have a demo_enabled state.
The page fetches information about the demo company multiple times (5, 8), and also fetches information about all users multiple times (6, 7). It is not clear why.

More important notes:

Requests all use PostgREST with the full query being made by the web front-end itself.
- The table name is the URL base (users, companies, etc.).
- The where clauses are each URL parameter after the select (e.g. company_id=eq.company_demo_guid).
All the data being returned is highly-structured JSON

There’s a DB-like interface exposed on to front end, and the API endpoints seemed to have a little security I had to workaround. This is the easiest and first thing to check. Initially I tried using Firefox’s ‘Edit and Resend’ feature to alter the requests, to change the parameters but making requests out of order was yielding strange responses: Firefox would say 3.34kB transferred (0B in size). Only requests made in the expected order would provide results.

Rather than trying to modify the javascript powering the front-end, I used the free and open source Requestly extension to rewrite the requests on the fly. Set up multiple rules to intercept those calls to change the parameters.

A little back-end validation to work around: limit 1. There was some back-end validation to limit certain queries to only return one result, and the server returns an HTTP error with information about that limit. This meant I couldn’t get a full list of every company in one query. I just worked around this by adding an offset=1 parameter to the PostgREST query, and incremented the offset with every call. If there were more than dozens of customers, I would have automated or found a faster way to pull this information.

This method allowed me to modify queries to return arbitrary data (with a different/no where clause), and to return the count on any table in the db, and retrieve rows, either in bulk or one at a time.

The Findings

This simple method in this case was enough, without ever having seen a line of the app code, to:

Get a full list of all companies, whether they had a demo mode or were in production.
Get a listing of all individual users that shared their information with this app, including their names, emails, locations, and highly-personal details.
Get data from any other table in the database available to the db user – I didn’t find anything too interesting though. At least not as interesting as the two previous bullet points.

Other than that I also found:

A semi-functional admin user signup/account creation page
A password reset function
A number of redundant calls and data processing functions, probably abandoned by the LLM while iteratively building the codebase.

The Follow-ups and My Recommendations

I did connect with my friend to let them know what I found, and how easy it was. No code-level exploits involved. Minimal technical knowledge. No knowledge of how the back-end was implemented. And thanked them for the opportunity to learn a lot in the process. Thankfully they hadn’t gone live with real people’s data yet, and I offered to take a look before they do, as a friendly courtesy. They’ll take me up on that offer :-)

What should you do? At minimum, get a friend with technical knowledge, someone you trust, to take a look before you launch/publish/go to production. Ideally leverage the LLM as a way to learn more yourself about what your codebase is and what it does. If you’re staking your reputation and your company on it, someone, at some level should understand what it is and what it does.

— Varun

Who’s Varun? I’ve been in Product at companies like at CLEAR and Noom, Hotel Tech with ALICE (acquired by Actabl), and I was previously founder of an HR tech startup Disqovery. I have worn many hats, and I like making things. Advising startups and early stage founders. I also like talking business. You can reach me at me@varunmehta.com, Mastodon, Github, and LinkedIn.

How to repair a broken Force Sensitive Resistor (FSR) in a GalaxiPad DDR Dance Pad

Sat, 14 Feb 2026 00:00:00 -0500

I still enjoy Dance Dance Revolution (DDR), but (good) hardware is still hard to find. Ordered a Galaxi Dance Pad. Unfortunately, it broke after a few days of use – and the guy that runs the company has not been responsive. One of the arrows stopped responding at all.

There was no obvious disconnect with the wiring inside. On a recommendation, I purchased this replacement FSR strip, placed it between the aluminum bar and rubber strip, and then had to figure out how to securely connect the metal pins to the rest of the wiring for the system.

It originally seems like they soldered the wires directly to the FSR pins, and covered the whole thing in hot glue. Instead, I found that a double wipe 0.1” (2.54mm) IC socket fits the pins from this strip perfectly. I soldered my own wires to the pins on the socket, plugged the sensor pins into the socket, and then covered the whole thing in tape. Ultimately the tape didn’t hold, and to make it a lasting repair I still covered the whole thing in hot glue. It’s been working great since.

Notice: Modern PC BIOSes will ignore small EFI System Partitions (ESP)

Sat, 14 Feb 2026 00:00:00 -0500

Small note, but important:

If you are reformatting or repartitioning disks, and using EFI/UEFI, your EFI System Partition (ESP) must be probably at least 100MB or 1GB. If it’s smaller than that, your BIOS will silently ignore the partition and refuse to boot.

No disk partitioning utility, boot repair utility, or bios screen warns you about this requirement. It doesn’t matter if your ESP is easily large enough to fit all of the files you need to fit.

Posting this on the personal blog so hopefully it won’t disappear off the internet.

Could the Sochting Oxydator Actually Work? I think so.

Sat, 14 Jun 2025 00:00:00 -0400

This is mostly copied from a YouTube comment I posted a few weeks ago. Brought onto the blog because Google can be capricious, and to improve discoverability. There are a bunch of claims out there about the Söchting Oxydator, about how it can boost oxygen levels, clarify aquarium water, and keep your fish healthier.

But is it even possible for it to live up to the hype? Based on my quick back-of-the-envelope calculations, I think so.

To be clear, I am not a chemist, just a general nerd. Someone more knowledgeable could and should think through this.

What is The Söchting Oxydator?

The Sochting Oxydator is still a mystery to me; there are a lot of pseudoscience-y claims out there. Basically, it’s a ceramic vessel that you fill with hydrogen peroxide solution, and it slowly reacts or releases the peroxide into the aquarium. There are a bunch of claims that slow-dosing hydrogen peroxide into your aquarium can help with specific problems aquarists have. With the oxydator though, I’ve read some dubious claims about being the “freshest oxygen” 🙄 that I don’t quite udnerstand. To borrow a few marketing phrases from a few retailers’ listings:

Söchting Oxydators breakdown hydrogen peroxide into pure oxygen and water (2 H2O2—->2 H20 + O2) using a special catalyst. (The Shrimp Tank)
The Söchting Oxydator adds oxygen continuously and slowly to your aquarium. (TST)
The additional oxygen helps provide oxygen for aquatic critters and the beneficial bacteria in your tank. The oxygen produced is almost immediately dissolved into the water so there are little to no bubbles produced. (TST)
Due to the Oxydator’s oxidizing capacity, the water quality will be improved and harmful contaminants in the water will be neutralised and In the summer, it is effective at fighting algae. (Amazon)

It does make tiny bubbles, which folks on YouTube have documented in their videos, and it’s cute how the shrimp like to hang around it. (Happy Little Whatever on YT)

Could it work?

It would be great to cut through all the marketing claims and get an objective breakdown of how/when it’s useful! Primarily it seems like a good way to slow-dose H₂O₂ into the tank to keep it healthy.

Again, I’m not a chemist, but by my quick calculations, maybe it does work to keep oxygen levels up?

The Oxydator A (for up to 400L aquariums) has a 150mL capacity
Which is 9mL of hydrogen peroxide (at 6% concentration)
That’s 13g of H₂O₂.
0.4 mol, since peroxide weighs 32g per mol
$2H_2O \longrightarrow 2H_2O + O_2$, assuming all of the peroxide breaks down, instead of reacting with organic compounds in the water.
That’s 0.4 mol water, and 0.2 mol oxygen
It will turn into ~7g of water (H₂O = 18g/mol), and 6g of oxygen (O₂ = 32g/mol).
Fish like 5-8ppm of dissolved O₂. Atlas Scientific
In a 50 gallon tank (10.5mol H₂O), we’d want to have 0.05mol to 0.08mol of dissolved oxygen.
Which is 1.6g to 2.5g of O₂ continuously available in the water.
That sounds reasonable for the Sochting Oxydator to supply for at least a few weeks!

Depending on how quickly fish consume it, and the amount of oxygen absorbed through the water’s surface, the oxydator is providing around 4x the oxygen concentration that we would want to sustain in the tank for our fish friends.

If my math/chemistry calculations hold out, I could see how it’s great for adding O₂ without losing all your dissolved CO₂ to surface agitation.

Next step: try it and see

I want to get one of these for myself. If I do, I’ll run some tests, as methodically as I can, and share my findings here on how well the Sochting Oxydator does (or doesn’t!) work. I wouldn’t mind removing the air pump from my indoor pond, if I can get away with it.

— Varun

Who’s Varun? Currently Product at CLEAR. I’ve been in Product at Noom, Hotel Tech with ALICE, now Actabl, and I was previously founder of an HR tech startup Disqovery. I have worn many hats, and I like making things. I also like talking business. You can reach me at me@varunmehta.com, Mastodon, Github, and LinkedIn.

IPv6 Learning Followups: Google and Ubiquiti (Unifi UXG) Gateways

Mon, 07 Oct 2024 00:00:00 -0400

The advice in this post is for more advanced computer/network users. One month later, I’ve had to adjust a few things for our IPv6 setup. And rather than posting it exclusively on corporate social media, I’m writing it up here and sharing on the fediverse (Mastodon).

Recap

Previously, I went through the certifications to get /64 and /48 reserved IPv6 spaces from Hurricane Electric, and set up a dual stack configuration for our whole home using HE’s free tunnelbroker.net service.

Overall, things have been working well. I was skeptical of what speeds to expect, but given that HE runs a huge part of the internet backbone, and the tunnel entry point is local to us in New York, it’s no surprise that latency and throughput are just as good as our ISP’s direct IPv4 service.

Solving the Google IPv6 Tunnel Problem with my Pi Hole

As mentioned in the last post, Google seems to treat any IPv6 tunnel user as a malicious actor/VPN/shady person, and the whole family has been struggling with increasingly impossible CAPTCHAs if they want to run a Google web search. YouTube, Gmail, and other Google services work without issue.

They’ve mostly migrated over the DuckDuckGo as their default search engines, but I still wanted to provide some relief for when Google search was necessary. Thankfully, since I run a Pi Hole locally, I can control how any domain name resolves on our home LAN.

Set up a Pi Hole rule to specifically block IPv6 (AAAA record) DNS resolution for all *.google.com domains.

Domain/RegEx: (\.|^)google\.com;querytype=AAAA
Type: Regex blacklist
Status: Enabled
Comment: “No Google IPv6” or something else that will help you remember

Ubiquiti EdgeRouter X IPv6 Firewall Rules are Awkward

Since every IPv6 address is publicly routable, it is vital that you keep good firewall rules in place. We cannot rely on NAT+Port Forwarding for protection the way we do for IPv4 LANs.

I guess since the EdgeRouter X/Max series is older, it only has a GUI for creating IPv4 firewall rules. If you want IPv6 firewall rules, then you need to add them using the console or the config tree. This is okay, but I found it awkward to set up and validate that the rules were working as expected. Coupled with the fact that the aging EdgeRouter X was under high CPU load with all of our LAN traffic, and I used this as an excuse to upgrade to Ubiquiti’s newest line of Unifi-compatible gateways. Specificaly the 5-port UXG Max. It would be easier to manage it together with our Unifi wireless access point, ande it has a nice-looking and functional GUI for setting up IPv4 and IPv6 firewall rules. However…

Uniquiti Unifi UXG Gateways don’t natively support IPv6 Tunnels!

My bad for assuming that a newer product would at least be as capable as the older (cheaper) product, I guess. There are no GUI tools for creating a tunnel, and there is no config tree create whatever you need, either. Ubiquiti Support confirmed to me that IPv6 tunnels are not supported on the UXG series :-(

Add an IPv6 Tunnel to the UXG Max anyway

We can still SSH into the gateway, so there has to be a way. Turns out there is, and it works better than I expected, once you jump through some setup hurdles. Basically:

Modify and set up the UDM on_boot.d daemon, which will run whatever scripts you want on boot.
Add the UDMP-ipv6 startup scripts to on_boot.d, to set up the IPv6 tunnel on every boot.
Set up your IPv6 firewall rules on the Unifi GUI as Internet v6 In like a normal person would.
Add the UDMP-ipv6 cron jobs. That will keep copying Internet v6 firewall rules over to the correct interface.

1. Modify and install on_boot.d on the UXG

This is the most technically complex part. Do this right and the rest is smooth sailing.

Set up SSH console access to your UXG device.

Follow the instructions in the UDM Boot Script Readme for a remote installation.

NOTE: If you are using a UXG device, as of this date, the maintainer has not added support for it in the installation script. You can check out my open pull request to see how to edit the script for your own device. Run ubnt-device-info model on your device to get the correct model string to use on line 78. You will have to copy a modified version of the remote_install.sh script over to your UXG and run it from there.

2. Add IPv6 tunnel startup scripts

Add the UDMP-ipv6 startup scripts to your boot scripts. Follow the instructions in that README. No modifications are necessary for UXG.

3. Set up IPv6 firewall rules

Do this as normal using the Unifi GUI to set them up as Internet v6 In advanced rules or else simple rules. Both will work.

4. Add the UDMP-ipv6 cronjobs

This is also documented in the README inside UDMP-ipv6 startup scripts. This wil make sure that your Internet v6 In rules will apply to the IPv6 traffic that’s coming into your tunnel.

What’s Next

Nothing is left here! I might contribute to the community-developed Unifi OS Utilities project to help get it working for UXG devices, too.

— Varun

Who’s Varun? I’ve been in Product at Noom, and I was previously founder of an HR tech startup Disqovery. I have worn many hats, and I like making things. I also like talking business. You can reach me at me@varunmehta.com, Mastodon, Github, and LinkedIn.

Getting Caught Up On IPv6

Tue, 27 Aug 2024 00:00:00 -0400

A couple of things came up recently that got me thinking about IPv6 and how little I understood beyond the absolute basics. I figured that now is as good a time as any to learn and see what I can do to get involved. Here’s what I accomplished and what I still have to do.

Why now?

IPv6 was drafted 1998, and ratified by the IETF in 2017. Why am I looking at this now? A few things came up around the same time that brought IPv6 back to the forefront for me:

Home automation. I got a new smart home fan which supports Matter, because of course I’m a nerd. The fan comes with a note that in order to control it using a Matter platform (like Apple Home), it the fan must be connected to the network over IPv6. I’ve never encountered this requirement for any device before.
My email server supports IPv6. As I wrote recently, I’ve set up my own personal email server using Mail-in-a-Box. During the setup process I noticed that Contabo (the VPS host) and the full Mail-in-a-Box stack support and provide IPv6 connectivity. When resolving the server’s issues it got me to set up all of the DNS records, reverse DNS records, etc. for both the IPv4 address and the IPv6 address. That’s pretty cool.

An Opportunity, a Roadblock, and an Opportunity

To learn more about IPv6, the legendary Hurricane Electric runs a fun and free IPv6 certificate program. It’s a mixture of practical tests and quizzes that get progressively more difficult for each of the 7 levels, starting at being an IPv6 Newbie all the way up to being an IPv6 Sage. There are video tutorials and online resources to help you learn what you need to know to pass each level.

Unfortunately, I hit an obstacle while getting up and running with IPv6. Early in the certificate program you need to have both a server that runs on IPv6 and also a desktop that can reach IPv6 locations. I had the server covered thanks to my Contabo VPS. However, the desktop was a challenge. You see, Spectrum, my ISP for my NYC apartment, supports IPv6, but our ISP for our Long Island home, Altice/Optimum, does not, and has no definite plans to support IPv6 any time soon.

Thankfully, Hurricane Electric also offers a free IPv6 tunnel service where you can get your own /64 block of routable IPv6 addresses for free. That’s 18 pentillion usable, public IPv6 addresses, just for me :-)

Tunnel Notes

Getting the tunnel set up on my Ubiquiti EdgeMAX was easy enough once I noticed they had specific instructions to follow to get it operational. They have example configurations for many common routers and operating systems. And a little bit of poking around on the internet and you’re bound to find someone who has your particular configuration. After all, they now provide 56,000 IPv4 to 6 tunnels!

Certification Success!

Once the tunnel was set up, most of the certificate course was easy enough to progress through. There were a few tricky questions I had to look up, and a few specific steps to take to get my IPv6 desktop and server environments set up and totally ship-shape. The learning was fun, and some of the questions/snark in the certificate course was fun, too.

What’s Next

Free t-shirt! Apparently Hurricane Electric sends a nerdy-looking t-shirt to folks who finish their course to the Sage level. I didn’t know this until I finished the certificate program. I look forward to receiving it and showing it off.

Unexpected IP issues with major servers, to be resolved. Unfortunately, now that our family home is on IPv6, it’s triggering some security protections on websites like Google. While I don’t mind much, as I’ve been minimizing how much I use Google (including Gmail), the rest of the family has noticed and is being irked by the fact they need to solve a Google CAPTCHA a couple of times a day for each device they use. Some HE tunnel users also noticed this issue, and I have to get caught up on the discussion and figure out if any of these will help us. It might be as simple as just waiting for Google to realize that we’re legit, I don’t know.

— Varun

Serving My Own Email and More

Mon, 12 Aug 2024 00:00:00 -0400

It feels like in the middle of the corporate-influenced internet’s heydey, we’re seeing a re-decentralized internet’s nascent emergence. And for me, I think it’s time to figure out and free one of the oldest parts of the internet I still use regularly: e-mail. Thankfully, it’s easier now than it’s ever been.

Why?
Requirements
The Approach
- Services
- Software
Guides
Tips & Tricks
How’s it going?
Now What?

Why?

Why not? Seriously though, my reasons are manyfold:

No limits. I’m tired of the regular reminders from Google to upgrade my storage, while getting casual warnings that I might stop receiving email unless I clear up my storage or start paying them for more room in my inbox. If I run my own email, I can provide my own storage, and upgrade it however/whenever I see fit.
No ads. I don’t like seeing Google-inserted ads in my inbox.
Privacy, please. Google/Alphabet/Microsoft/Apple already know enough about me without also having access to all of my email. The less data floating out there on the corporate-controlled internet, the happier I’ll be.
Ownership feels good. More philosophically, I own the domains. If I own the domain, the email should be mine, too.
Bonuses and network effects, abound. Depending on the exact approach, there could be more benefits to running my own email, that would apply to other aspects of my online/tech footprint.

Requirements

A good approach has to satisfy a few key requirements for me:

Keep costs “reasonable”. For now I’m looking to pay around the same price as a storage upgrade from any of the major consumer cloud providers (Apple/Google/Microsoft). I think $5-10 a month sounds about right.
Occasional maintenance at most. I want an email system that is mostly “set and forget” for its core features. I’m okay with occasionally needing to log in to change settings, upgrade, or perform other maintenance tasks – let’s say maybe every two weeks or so. A good system will notify me when maintenance is required instead of making me poke aorund and check for myself.
Won’t get marked as spam. I need to be able to set up all of the proofs, validations, and security so that a receiving email server (especially gmail) won’t mark my email or domain as spam.
Reliable enough that I can use it for important email. Servers go down and I want to be prepared for that. In particular, I want to make sure incoming emails don’t get lost/bounce if my server goes offline for a little bit due to maintenance or an outage.
Data integrity so I don’t lose what I already have. In particular, I need some kind of automated backup system in place so I don’t lose my whole inbox because of a storage failure.
Tinkerability, please. Good off-the-shelf solutions are cool. I still want something with some bells and whistles and knobs to play with to keep things interesting.

The Approach

tl;dr: The core of my approach will be using Mail in a Box on a Virtual Private Server. Set up backup storage and backup DNS, too.

Services

Contabo Virtual Private Server (VPS) to run the server
- $6.85/mo.
- US East Location
- 4 vCPUs, 6GB RAM, 130GB NVME Storage
- 32TB monthly outbound traffic, unlimited inbound traffic
- I thought about running the server on one of my own computers at home that’s connected 24/7. However I decided against it
Contabo Object Storage for offsite backup
- $2.99/mo.
- US Central Location
- 250GB, S3-compatible API
- This is enough storage to last a long time for backups, and maybe also to store other stuff if/when I need to.
Gandi DNS for Domain Name registrar and hosting
- $24/yr.
- Supports Glue Nameservers
- Supports DNSSEC (for extra email deliverability)
- Supports Advanced DNS configuration
Backups DNSes in case the main server goes down
- Free
- afraid.org
- nether.net
- As long as the backup DNS provides the MX records, any incoming email will be held by the sending server and retried until my email server comes back online.

Software

Mail in a Box v69b is giving us the bulk of what we need. It comes with a bunch of components already set up. Just take a look at the architecture diagram!

Requires minimal configuration out of the box
Runs on top of a fresh Ubuntu install
Sends and receives email
Provides good webmail (Roundcube)
Serves its own DNS for all of the domains you want to serve
Hosts static web pages on any domains
Preconfigured NextCloud for contacts, andcalendar
Automatically manages backups locally or in an S3 bucket
Uses Z-Push for push email/calendar support on iOS
Generates activity and status emails so I know about any errors or if maintenance is required

Guides

I’m not going to write up a full guide on MiaB, both because a good one exists, and because as soon as I write it, it will be out of date. They have a setup guide that you should follow. Without the issues I mentioned in “Tips & Tricks” it would have taken me only a couple of hours to finish getting my email server totally up and running.

Tips & Tricks

I ran into a few snags along the way. I document them here in case you’re looking for these terms on your web searches.

Name Server Daemon (NSD) v4 won’t start

If you get this error, and NSD won’t start, be on the alert! Your DNS won’t be served because the daemon isn’t running! You’ll see errors on the console or journalctl like:

nsd can't bind tcp socket: Cannot assign requested address
nsd.service: Start request repeated too quickly.

This happened to me because IPv6 is off. The Name Server Daemon (NSD) that comes with Mail in a Box requires IPv6 is working on your server, and it will entirely fail to launch if it can’t. My VPS build came with IPv6 off. Rather than disabling IPv6 for NSD, I figured it’s smarter to just enable IPv6. It’s the future, after all.

Turning on IPv6 was simple. I just edited /etc/sysctl.conf to include the line below and restarted:

net.ipv6.conf.all.disable_ipv6 = 0

UI and validation errors for cloud/S3 email backup

Cloud backups in Mail in a Box have some UI quirks when want to back up in S3. Thankfully this was well documented on the MiaB forums by syslarper. Here are a few paraphrased notes from them. Refer to image below to see the fields I’m referring to.

Omit the protocol (http/https) and port numbers on the S3 Host/Endpoint.
Omit the region name.
Only port 443 is supported.
You need to have already created the S3 bucket.

After validating and saving your settings, most of the configuration fields will be empty when you open the backups management page. Just trust that the backup is going to work, and confirm it by checking the logs and peeking in your S3 bucket the next day :-)

Follow the built-in advice

Use the status checks in Mail in a Box to see what else you need to do to get everything working properly. Also use Mail-Tester to validate that your emails are getting sent with all the right security bits set to improve deliverability!

How’s it going?

Having been over a week, it’s so far so good! Backups are getting generated. When the server is down, incoming emails are just delayed for a few minutes while the sending server waits to retry. Everything costs me about $10/month.

Now What?

Now we get to have some fun! Send me an email if you want to help me test things out. Also, since MiaB is set up to serve static pages, I was able to migrate my Jekyll git repos and hooks over to the server. Which means that even this blog you’re reading (right now at varunmehta.com!) is published and hosted on the very same server. I might even cancel my account with Dreamhost, my discount shared hosting provider!

— Varun

FPGA Game Graphics Part 0: Motivation

Fri, 22 May 2020 00:00:00 -0400

FPGA hardware has been getting a lot more affordable, and I have a few ideas for projects. The new low-cost boards that combine FPGA-style programmable logic with a fast processor are especially interesting. But before I can get to the more advanced stuff, I need to re-learn the basics. I need a plan.

Why?
Goals
Hardware
Requirements
Challenges
Follow along!

Why?

Why not? My day jobs keeps me far away from low-level technology, and that’s fine with me. However, as a hobby, it’s an awful lot of fun. I find I learn best by picking a goal and learning as I go along to make it happen.

Goals

I actually just want to make a clock. The thing is, I like making strange clocks. Some examples:

An Arduino PWM-based clock that uses analog VU-style meters to show hours/minutes/seconds.
A Raspberry Pi running QEMU, emulating a PPC running Mac OS 9. The Mac OS 9 desktop is displayed on an e-ink display. The Mac OS desktop naturally displays the time in the menu bar. However, within the virtual machine, Internet Explorer 5 auto-refreshes a webpage hosted by the Raspberry Pi – a simple webpage that displays current and upcoming weather conditions. This might be worth a post of its own one day.

My strange clock game has been ramping up. People are building some tetris-style clocks which are pretty cool, and that got me thinking about making my own video game-styled clock. But I’m a visual nerd, and smooth graphics are important to me. There are plenty of interesting systems out there like the Odroid Go that even include a display, buttons, and a battery, but pretty much all of these systems have limitations that I don’t like. A big one is the quality of the visuals: these devices include Serial/SPI based displays, and SoCs like the ESP32 don’t include the kind of graphics hardware necessary to provide smooth (60 frames per second) motion at resolutions higher than 240x160.

Hardware

I also like playing in the boundaries between software and hardware, so taking a system like the Odroid Go and building my own graphics capabilities is exciting. As luck would have it, the folks at Seeed Studio released an “Arduino Shield” that they call the Spartan Edge Accelerator. Seeed releases many dev kits, boards, and peripherals, but this one caught my attention.

At a high level, it’s a board meant to be attached to an Arduino, or potentially to run standalone. It includes:

Xilinx Spartan 7 xc7s15 (12.8k logic cells, 360 kilobits block ram)
Espressif ESP32 “for wireless”
HDMI port (raw)
CSI/MIPI interface (raw)
MicroSD slot
ADC chip
Accelerometer and gyroscope
A few LEDs and push buttons

Requirements

Hard requirements:

No additional processors or PCBs
Teach myself verilog
480p video support
60fps video
Single screen-style NES background
NES-level sprite support
16-bit color

Stretch Requirements:

720p video support
Super Mario Bros 3-level background scrolling
Super Mario World-level layer support
SNES-level sprite support
24-bit “true” color

Challenges

A few challenges already stand out to me on making any of this possible.

Inter-IC Communication: The ESP32 and FPGA on the Spartan Edge Accelerator weren’t really meant to continuously talk to each other: they’re both supposed to communicate with an Arduino. In fact, the ESP32 is only supposed to be using i2c most of the time! To have the type of dynamic display that I want, all data updates need to be copied from the ESP32 to the FPGA during the VBLANK period. This is similar to how systems like the NES worked, where there is a CPU and separate PPU (Picture Processing Unit) and the systems do not share memory. Looking at the schematic, my best bet is likely the QSPI connection that the ESP32 can use to flash the bitstream to the FPGA upon startup. Even with that, I’ll need to find a way for the FPGA/GPU to signal to the ESP32/CPU that VBLANK has begun and data can now be copied from CPU memory into GPU memory. And then there’s also the challenge of just how much data can be copied during the VBLANK period over a connection like that.
Memory Size: The ESP32 will have plenty of RAM for our purposes, at least 512KB. However, the FPGA has no discrete memory chip, and so will depend on its block RAM, which is limited to 45KB. This is much more memory than available to the NES, but it’s not quite up to the 64 KB that the SNES could take advantage of. I don’t yet know how far I’ll be able to get towards 16-bit graphics, especially given that we’re targeting much higher screen resolutions than the NES or SNES.

Follow along!

This should be a fun time! I’ll add more posts as I work out the rest of the systems. If you want livestreaming updates, you can check out my YouTube Channel, where I livestream most of my work on this project. Also, I’ll be checking in all of my progress on GitHub.

— Varun

Who’s Varun? I work in Product at ALICE, and I was previously founder of an HR tech startup Disqovery. I have worn many hats, and I like making things. I also like talking business. You can reach me at me@varunmehta.com, Mastodon, Github, and LinkedIn.

Opinionated Behavior-Driven Development

Wed, 11 Oct 2017 00:00:00 -0400

After having a few conversations on this topic recently, I’ve decided to put down my thoughts on behavior-driven development (BDD): what it’s for, how to use it, and how not to use it.

Behavior-driven development
My context
BDD flow
BDD tips

Behavior-driven development

There is a good overview of BDD on Toptal’s blog, especially as it compares with test-driven development. My perspective: behavioral testing uses 100% implementation-neutral descriptions of what a user does with your software. BDD is writing those tests before any code is written, and using those tests to gauge the implementation’s progress. When all of the behavioral tests pass, the feature is complete.

My context

Working on Disqovery is what exposed me to BDD and behavioral testing in the first place. Because of that, my opinions and experiences are based on:

Web and mobile web users
Ruby on Rails backend, angular.js frontend
Existing unit testing framework
Agile-style project management
Using cucumber with phantomJS set up with capybara via poltergeist. Yes, these project names are all very amusing.

I won’t explain how to get cucumber or similar systems working. There are plenty of great tutorials and videos that will explain it better than I could. As you could imagine, some setup is required to get all of these pieces working together, but the benefits are numerous:

Every test is written in plain English, so every team member and stakeholder can understand what is being tested.
Tests are run automatically while developers are working on their code. We can catch regressions before any changes even submitted for code review.
The headless browser system sees exactly what the end user sees, including all javascript and css. This is end-to-end testing in its purest form.
Not only are any test failures reported, but screenshots are automatically generated to aid in debugging and development.

BDD flow

I incorporated behavioral testing and BDD into a pretty standard PM workflow. This works easily with cucumber since it breaks tests down into “Features” which consist of multiple “Scenarios.”

Perform customer research, including an understanding of customer challenges and needs.
Synthesize feature descriptions, use cases, requirements statements, and priorities. This is the start of your functional spec. Get it signed off by stakeholders.
Convert spec features to cucumber features, use cases to cucumber scenarios, and requirements to cucumber steps, as appropriate. Tag scenarios according to milestones.
Work with development team to get features created. Behavioral tests will help track progress, as well as catch regressions.
Reprioritize as needed during development. Moving a failing test to another milestone makes it easy to identify exactly which features and use cases are affected.
Go back to step 1 once you’ve shipped a milestone to the customer.

BDD tips

Developers don’t write behavioral tests. You do. Let developers write the code-centric tests. The level of customer understanding for behavioral testing requires they be written by the PM and UX designers. However developers are stakeholders, and should be part of the conversation.
Involve your stakeholders early and often. Cucumber’s tests are written in plain English; use that to your advantage. Involve your stakeholders in the feedback process early, to make sure every sentence of the test agrees with their assessment of what a user should experience.
Think of scenario steps at the most basic levels. Steps in cucumber should include verbs, and I focus on them being the simple possible verbs. The user ‘sees’ something, ‘clicks’ or ‘taps’ a button, ‘drags’ a picture. In some cases maybe they even ‘hear’ a sound or ‘select’ a file from their computer.
Cucumber is not a way to validate 100s of different devices/browsers. Cucumber, along with the tools I described above, is a way of validating behaviors during development, but it is limited to a few browser frameworks. Tools like Xamarin’s Test Cloud may provide the diversity of devices you’re looking for, at the expense of test complexity. There may be a way of connecting a behavioral testing framework with Xamarin Test Cloud; I’m not sure.
Close development feedback loops as tightly as possible. Automate test runs. I set up our Rails test environment so that every time a developer saved a source file, the relevant tests would automatically run and report their results onscreen. Within seconds, developers would have feedback on whether they fixed and issue, or made it worse. Tighter feedback loops can speed development.

If you’ve never heard of BDD before, this post may have piqued your interest. If you’re considering it, I hope my experience and implementation will prove helpful. Tools like cucumber are so incredibly flexible, that having some opinion on the “right way” to use them is better than none. Completely disagreeing with me is just as valid as completely agreeing.

— Varun

Who’s Varun? I most recently was the founder of an HR tech startup, Disqovery. I have worn many hats, and I like making things. I also like talking business. You can reach me at me@varunmehta.com, Mastodon, Github, and LinkedIn.

Reverse Engineering: Logitech Z680 LCD Replacement, Part 2

Sun, 24 Sep 2017 00:00:00 -0400

Timing is everything, especially in serial communications. I’ve run into two interesting timing challenges on this project that need to be solved, but they couldn’t be more different. Check out part 1 first if you have no idea what’s going on.

Hooking up a real LCD
Gibberish problem
- Gibberish solution
Custom characters problem
Next steps & code
Who’s Varun?

Hooking up a real LCD

In part 1, we got the SPI data from the control pod, and used ANSI escape codes to recreate the display in a terminal window on my Mac. This is very cool, but not useful. I need to turn the Arduino into a translator that converts control pod serial data into parallel data for a replacement LCD, likely using the HD44780 protocol.

I ordered a slick, slim 2x20 character OLED for a replacement, but in the meanwhile, I have an old 4x20 CrystalFontz character LCD I can use to test my code and hookups.

It’s a parallel LCD, so we have 4 bits of data [D4:7], enable [E], and instruction/data register select [RS]. Since we’re starting with an HD44780-compatible display, I can simply use the Arduino LiquidCrystal library. Most of the commands from the control pod map directly to hd44780 commands, so I can either parse the commands, or just send the raw payload directly to the display. Here’s an example from the actual code.

    //payload is the decoded byte from the 3-byte serial sequences the control pod sends
    switch (payload) {
      case 0x01:
        Serial.print("\e[2J");
        lcd->clear(); //parse the command and send via library
        break;
      case 0x02:
        Serial.print("\e[1;1f");
        lcd->home(); //parse the command and send via library
        break;
      case 0x40:
        Serial.print("\e[2J");
        lcd->command(payload); //send the raw command
        break;
      case 0x0c: //turns on LCD, disables cursor
        lcd->command(payload); //send the raw command
        break;
    }

So basically the code works like this:

The Arduino SPI interrupt adds any received bytes to the buffer.
The main code loop processes the buffer whenever there are a multiple of 3 bytes in the buffer. (Remember that all data from the control pod comes in 3-byte sequences).
For each 3-byte sequence, the main code loop performs the appropriate commands on the LCD.
Once finished, the main code loop clears the buffer and waits for more data.

This is a simple solution that should work really well, right?

Gibberish problem

No, the solution was not so simple.

It’s supposed to say “Input: Optical” and “Effect: Detecting”. You can kind of see that.

This is a silly issue, but it drove me crazy for a little while. I did not really think about how the SPI interrupt and my main code loop interact. The interrupt can be called at any time, including when the main code loop is performing commands on the LCD. If I’m not careful, the main code loop can clear out the buffer without processing all of the data, because its state can overwrite more recent information coming from the interrupt function. This results in missing data. This could be missing commands or missing letters, and results in the gibberish you see above. This is a classic producer-consumer situation, and I can probably use a classic solution to get around it.

Gibberish solution

Ring buffers are the best solution I can think of. I used Steve Karg’s FIFO library for another project in the past, and fortunately he has an interrupt-friendly ring buffer available, as well. This library allows for objects of any arbitrary size, include our 3-byte sequences. As an added bonus, it’s lightweight, and it will use minimal space on our Arduino board.

The ring buffer changes the code like this:

The Arduino SPI interrupt adds any received 3-byte sequences to the ring buffer.
The main code loop pops an item off of the ring buffer and processes it.
When the main code loop is finished it goes back to step 2.

In this way, the main loop is not responsible for clearing the buffer. This is enough to make sure no data is clobbered, and the SPI interrupt can safely add more data to the ring buffer any time it needs to.

Success! If only for some screens, we can now get data sent successfully from the control pod all the way to our own replacement display.

Custom characters problem

If the Z680 control pod just used standard letters, numbers, and symbols we would be done. Unfortunately, this is not so. The display uses custom characters for volume control and a few other functions, and those aren’t working with our new display yet.

How it works

When the control pod is first powered up:

It enters standby mode. Speakers are off. LCD backlight is off. LCD power is on.
Control pod initializes the LCD.
It sends all of the data necessary to define the custom characters.
It clears the LCD contents, and keeps the backlight off.

Only when the power button is pressed is control pod is turned on. Then the speakers turn on, the backlight turns on, and content is sent to the LCD for display. The LCD uses a 5x8 font, and supports up to 8 custom characters. Each custom character is defined using the lower 5-bits of an 8-byte sequence. There’s a nice breakdown of this here if you would like more information. I captured all of the data with my logic analyzer, so this breakdown might be helpful:

Why it doesn’t work

Like the gibberish problem, getting custom characters working also comes down to timing. The Arduino and OLED are both powered by the control pod, and the pod sends custom characters into CGRAM very quickly after it is plugged in and goes into standby mode. This is the only time it sends custom characters. We need to catch it quickly if we want to read that data, and unfortunately Arduino systems can take a long time to start running code after power on.

My oscilloscope shows we have about 125ms after power before the first bye of data is sent by the control pod. To see how long it takes Arduino to start up, I wrote a simple app that just sets a pin high. The RBBB Arduino starts after 1500ms!

Why am I using an oscilloscope instead of logic analyzer? Since I have to measure the 5V voltage rail, and Arduino expects at least 4.5V before it turns on, I need to see the analog values. If I used a digital logic analyzer, it might report every value above 3.0V as “logical 1”, which would make understanding the power behavior harder. Additionally, this way I can check for voltage dips during boot.

Solutions

The most obvious solution would be to hardcode the custom character information into my Arduino. But that would not be a ‘clean’ solution. I would never expect the control pod to send different custom characters, but I don’t care; if I’m solving this problem, I’m going to solve it right.

In that case, the only other easy solutions are:

Replace the default Arduino bootloader with a faster one, like adaboot or optiboot.
Reprogram the fuses on the ATmega328p on the board to start execution faster. The default is 65ms after power is applied, and it can be dangerous to execute sooner.
Bypass the power stabilization components on the RBBB so that we can feed the ATmega328p with 5V sooner.

The bootloader seems like the safest option, and the one with the most potential gains. Lucky me, I still own an Atmel STK500. It is the AVR programmer that all other AVR programmers are based on, and it is exceptionally powerful, if a little dated.

Getting my STK500 to flash the bootloader was unpleasant. I’ll spare the details of each step. Contact me if you want to discuss any of them:

Realize STK500 is so old it only supports v1 of the protocol, and can only be upgraded in Windows.
Install Atmel Studio on Windows. Every time you try to upgrade STK500’s firmware, Atmel Studio crashes.
Identify missing VC90 dll version and install.
Upgrade STK500 firmware.
Realize RS232 USB adapter kernel panics macOS 10.12.
Install Arduino tools on Ubuntu.
Edit Arduino system config so it supports STK500 high-voltage parallel programming.
Install optiboot in Arduino app.
Prepare STK500 for high-voltage parallel programming. Insert ATmega328p into STK500 slot 3.
Burn optiboot bootloader onto chip, and put chip back into Arduino board.

Amazing, just flashing optiboot reduced startup time from 1.5s to 65 milliseconds! Before trying anything else, I put my normal code back on the Arduino, and yes! The system starts up quickly enough to receive all of the CGRAM data from the control pod! Everything works! That’s how I took the yellow OLED pictures you saw earlier in the blog post.

Next steps & code

Code for this project is available as a GitHub repository. I created my own version of LiquidCrystal that is designed for the NHD Character OLEDs instead. It’s very messy and not ready for primetime. When it is, I’ll spin it off into a proper Arduino library with its own github repo.

The next step is figuring out exactly how to cram all of these electronics into the speaker control pod, in a way that works reliably and looks good. Maybe we can also get some bonus functionality. The character OLED datasheet has some neat features.

Who’s Varun?

I most recently was the founder of an HR tech startup, Disqovery. I have worn many hats, and I like making things. I also like talking business. You can reach me at me@varunmehta.com, Mastodon, Github, and LinkedIn.