Liquid Gotchas and the Dynamic Schema

The diary API has nine endpoints — entry save, delete, and list; product create and search; profile read and write; submitting and listing weigh-ins. Not one of them is written in a server-side language. Each is assembled from flows: graphs of condition and graphql nodes where Liquid stitched variables and response bodies together. We put the whole backend together in one day, and even that single day produced enough gotchas. One clarification for new flows: values are computed by expr today (compute.expr, body_expr, variables_expr, predicate_expr), and most gotchas in this list belong to Liquid assembly. Since v1.4.16 that territory is smaller: the Liquid path remains in templates, emails, variables and “HTTP POST”, but it is removed from the engine for the graphql query text and the “JSON response” “Body (template)”.

The osteele/liquid engine

The Liquid documentation describes Shopify’s parser, while the platform runs on the osteele/liquid library. The syntax is compatible, but a number of filters and tags behave differently, and those differences are what we tripped over the most.

The first difference is the blank check. Shopify’s engine understands blank as a special value. osteele/liquid knows nothing about it, so the “is the string non-empty” check has to be assembled by hand:

{%- assign nm = json.name | strip -%}{% if nm != "" %},name: {{ nm | json }}{% endif %}

A filter inside {% if %} is a syntax error, so {% if x | size > 0 %} won’t compile: the value goes into a variable through assign first, and the comparison happens after that. Emptiness is checked with size > 0, and it’s easy to get the direction wrong — == 0 gives you the inverted result. The familiar Shopify trim goes by strip here. None of this trap family exists in the expr fields: a missing field is nil, and the non-empty check is one comparison — form.lastName != nil && form.lastName != "".

Arithmetic is the plus, minus, times, and divided_by filters, and two of them behave in ways you want to know about in advance. plus: 0 silently turns any garbage into zero: handy for summing, but in field validation it masks a typo, and the value quietly zeroes out. divided_by falls over on nil, so check the divisor before you divide. In new flows the math belongs to a compute node in expr: literal numbers are ints there, and a bad operand fails visibly in the run journal instead of silently becoming zero. The filters remain for the Liquid fields — texts, emails, page templates.

JSON from Liquid: a comma inside the if

The JSON response body is assembled by hand, and every value goes through | json, which escapes the string and turns nil into null. The comma for an optional key is placed inside its {% if %}, and lists get assembled by a loop with {% unless forloop.last %},{% endunless %}:

{"ok":true,"items":[{% for i in results.lst.foods %}
{"id":{{ i.id | json }},"name":{{ i.name | json }},
"kcal_100":{{ i.kcal_100 | json }}}
{% unless forloop.last %},{% endunless %}{% endfor %}]}

This is a relic: since v1.4.16 Liquid in the “JSON response” body_template is removed from the engine — the key became a static string, and the recipe with substitutions no longer applies to it. A new body needs one expr line in the respond_json node’s body_expr:

{ "ok": true, "items": results.lst.foods }

The value is encoded as data — no escaping, no comma choreography, no loop: arrays stay arrays, numbers stay numbers, and invalid JSON is impossible by construction.

Another gotcha surfaced in an automated test. An entity field in a graphql query with no sub-selection gets expanded by the platform into the whole object, and food_id in the entries list once came back as a nested product object with no id field and a pile of service fields. Two tokens fixed it: food { id } in the query, i.food.id | json in the template.

The project’s templates use no template-insertion tags, although the site engine knows both: {% include %} inserts a template with shared variables, {% render %} — with isolated ones. The flow engine has no such tags at all, which is why our reusable graphql queries got names in the query library and are pulled in by a node via library_name.

There’s also a small thing we tripped over exactly once. A regular GraphQL client wraps every list in a nodes object, and a template reads the data as posts.nodes[0].title. Flow templates work without the wrapper: the graphql node’s response is available under its result key (lst), the array with the plural type name sits right inside, and the full path looks like results.lst.foods[0].name. A template ported from a regular client won’t show an error — it simply finds no data.

The flow engine enforces its own discipline around graphql nodes. Every write is preceded by condition nodes doing validation: no client_uuid, or a date that doesn’t look like YYYY-MM-DD, and the answer is a 400 with the machine code INVALID_REQUEST — graphql never even runs. Every graphql node carries on_error: stop. The API routes live at TTL 0: caching a response for a specific user is out of the question, and the platform never serves cache to logged-in users anyway. Passwords in the member table are hashed on write. On read they come back masked as "***" — the login flow gets the real hash for its bcrypt comparison, every other client gets stars.

The schema that changes under your feet

We describe the data schema in the admin UI, and the GraphQL schema rebuilds from it on its own. The rebuild is asynchronous, though, and can take up to a couple of seconds. So after a schema edit, the mutation and the query against the new type go out as separate HTTP requests, each preceded by an introspection poll. The catalog seeding scripts are built exactly that way: create the type, wait, confirm the type is visible, and only then pour in the data.

orderBy is an ENUM, so it’s orderBy: [{field: DATE, direction: ASC}], not a string "date". Numeric variables in mutations are declared Float, but JSON hands them over as strings. The update branches do their own conversion with the number filter before passing them on (in new flows number() inside variables_expr does the same); the create branch ships the strings as-is, and GraphQL’s type conversion turns them into Float on its side. Both paths work — the explicit filter is just easier to trust. And the big one for catalogs: limit/offset are capped at 100 entries per page. A catalog of 131 products exports in two requests, offset 0 and offset 100: the client keeps pulling pages while exactly 100 come back, with a safety stop at 10,000 entries.

The query library didn’t accept Liquid

We keep the diary’s queries in the query library. A graphql node stores its text next to a library_name label, the editor makes sure the two never drift apart, and at execution time the platform substitutes the text from the library entry live. This discipline had a boundary: the library validator accepted pure GraphQL only. Our update mutations assemble their JSON input with Liquid — a comma inside {% if %}, number filters on numbers — and the validator rejected such text outright.

So the Liquid-assembled mutations were back then deliberately kept in the flow nodes themselves, without a library_name: readable selects went into the library, and the assembled bodies stayed where they were. The knot was cut from the other side: since v1.4.16 Liquid is removed from the query text — the validator accepts pure GraphQL only again, the mutations became static and returned to the library alongside the selects, and variables_expr assembles the variables (partial assembly included — the recipe in the JSON-API article). The deletion rule survived unchanged: a library entry can’t be deleted while a live flow references it, so when you clean up the query catalog, you unbind the nodes first.

What the haul cost

The costliest items in the list were Shopify habits: the blank check, the strip filter, and the string enums all behave differently in osteele/liquid, and every such spot now gets checked with a small example first. The schema rebuild and the route cache update with a delay, and a flow that writes and immediately reads the data has to account for that delay from the start.

What’s next. The backend and the client are done — what remains is the math and the wrapper. The final part of the series, “The norms calculator, themes, and polish”, covers Mifflin-St Jeor, a ring that kept cutting itself, and the project wrap-up.