Forage development updates

Some backstory is necessary.

My wife has collected recipes for a long time. They’re hand-down family recipes; recipes torn from countless cooking magazines (Southern Living was a gold mine of spectacularly delicious, artery-clogging recipes); and more recently, print-outs from web sites. In addition to a large number of actual cooking books, we have several binders full of these old recipes.

Those books are a fantastic source, but as you’d expect, they’re impossible to work with. I had to rely on my wife’s memory about where to begin looking for recipes – was it in a cook book, or in one of the binders; was it a print-out or a tear-out or some recipe that came with some cooking thing we bought? Was it something saved from the back of a box of food? Was it a recipe embedded in an instruction manual for some appliance? Look, if you’re older than 30, you probably are familiar with this, and it only gets worse the older you get.

A few years ago, I started digitizing everything. I tried a couple of self-hosted solutions, and eventually settled on Mealie¹. The biggest issue I had was that the UI was entirely web based, and trying to use Mealie shopping lists while actually shopping was a painful experience, and building shopping lists was one of the main things I wanted to use this for.

So, I built a mobile app, which brings us to today. Or, at least, up to a couple of months ago, when a culmination of issues prompted me to re-write Forage.

Forage was built using Fyne, a fancy cross-platform platform. Fyne is more than a GUI toolkit, and that’s part of its problem. It provides not only a GUI, but a data layer, and a bunch of other features that make it a platform. I have pretty strongly negative opinions about plaforms, starting with vendor lock-in, but one of the other main issues with Fyne is that binaries built with it are huge. Go executables aren’t small to begin with, but Fyne is a massive extra load. Forage is over 100MB – for a glorified to-do list, that’s absurd. But the necessity of tightly coupling the data to view was worrisome, as was the fact that using Fyne means the exit cost is extreme. The tight coupling between the model and view meant writing unit tests was difficult, which made debugging hard. I needed to change the UI toolkit, but the pervasiveness of the platform meant a total rewrite. This is the problem with platforms: you don’t use them to write applications; you write applications on them, and once you do, there’s almost nothing you can re-use when you want to migrate to something else.

Another issue is that the Mealie API I built Forage against was pre-1.0, and that version of Mealie is static. All current work is going into v1.x, and it’s a drastic API change. Everything is different, such that I couldn’t even use the light storage abstraction layer I’d built. Mealie v1 adds paging, more separation between lists and their contents, units of measure, and a dozen other disruptive changes.

Finally, Forage has issues. Syncing and diffing worked surprisingly well, but update triggers took an unaccountably long time. Some of this was due to the fact that the Mealie v0 API provided no way of detecting changes other than simply downloading everything and checking for changes (there was no useful HTTP HEAD call). Forage was also missing functionality I found difficult to implement in Fyne, like context menus on checkmark widgets; and Fyne itself was awkward when it came to doing some things on mobile interfaces. Forage worked, but it was kind of kludgey, and at 100MB, it was just embarassing.

Forage needed a rewrite. The scale of the Mealie API changes, and the pervasiveness of the Fyne platform meant an utter rewrite. The fact that Forage worked impeded change; I have many irons in the fire at any given time, and the effort needed to re-implement Forage meant it was pretty low-priority. However, as Mealie started showing its age (as the size of our recipe DB grew through constant migration from paper and the addition of new recipes from the web), the new version started looking better and better. Around February 2024, I started serious efforts in the rewrite. This time, I took a more structured approach that wasn’t feasible with Fyne. First, I wrote foragelib, a pure library wrapper around the Mealie APIs (both versions). This allowed me to write foragecli, a terminal (but not TUI) Mealie interface. One of the main purposes of foragecli was testing foragelib; the Mealie API documentation is generated and missing a lot of key information, so much of the “how” it works I’ve found out through trial and error. Eventually, forageserver will be updated to use an enhanced API that will very likely differ from Mealie, as the Mealie API is almost certainly a DOM generated from a DB schema, wrapped in a generated OpenAPI spec. So much of the database shows through the schema, and I prefer an API-first design.

And so, here we are at today’s post. After a number of tests with various GUI toolkits, I ended up using gioui. I was almost certain this was what I’d be using, since the number of cross-platform GUI toolkits for Go is rather limited. Working with gioui has been interesting; like switching to a functional programming language, it requires some grinding of mental gears, but as I work with it I’m finding myself getting happier with it, when at first it was just frustrating. Working with Fyne was the opposite: easy to get into, more frustrating as the work progressed. For one thing, gioui by design encourages the developer to separate the model from the view, something which is turning out to be fantastic in terms of optimization. For another, gioui Android APKs are about 1/10th the size of Fyne APKs; I expect the rewrite to end up somewhere in the 20MB range (as of today, the APK stands at 18MB, and the application already does far more than Forage v1 did). As it stands today, it’s mostly feature-complete, and I’m tracking down bugs and adding some nice-to-haves – these are turning out to be far easier to add in gioui. I just today hit a major roadblock when I discovered that units of measure have their own API in Mealie v1, because of course they do, and to use UoM I have to add a bunch of code to foragelib to ensure units are synced before syncing shopping list items.

I’m sure I’m further from the end than it seems, but I’m really close to installing the app on my phone and testing it there, which is a pretty good indicator of how close I really am.

If you’re interested in this, I have a Matrix channel called #forage:matrix.org, where I sometimes post status updates. I’m thinking of calling the new app “Forager”, because I am not going to replace the old app in the fdroid DB. If the new version is ever backwards compatible – something that is very possible – then I’ll deprecate Forage and point people to the new entry.