FLAVOR LINES — HOW THIS MAP WAS BUILT A reproducible blueprint for an "isogloss map from restaurant names" ===================================================================== WHAT THIS DOCUMENT IS A step-by-step recipe for building a project like Flavor Lines (https://isoplov.vova.today): an interactive world map that shows where one dish name gives way to another — where "fried rice" becomes "plov" becomes "paella" — drawn from the names on restaurant signs in OpenStreetMap. It is written for a capable coding assistant (or a person) to follow and produce a similar project from scratch. It describes WHAT the data is, WHERE to get it, HOW to process it, HOW to turn points into coloured territories, and HOW to draw the map — not the exact source code. Line counts, thresholds and library choices are given as the values that actually worked, so you can start there and tune. The method generalises. The unit of analysis is "points of interest with a name, classified into categories, then coloured by nearest neighbour." Swap restaurants for shops, place-names, dialect words, brands, or anything with a location and a label, and the same pipeline draws its isoglosses. ----------------------------------------------------------------------------- 0. THE IDEA (read this first — it constrains every later choice) ----------------------------------------------------------------------------- An "isogloss" is a line on a map separating two ways of saying the same thing. Linguists draw them for words. This project draws them for dishes. The core, non-obvious framing — keep it visible in the UI or the whole thing is misread: We map dishes by the NAME ON THE SIGN, not by what the kitchen serves. A place is counted for "plov" only if the word for plov (in any language, any spelling) appears in its OSM name. That makes this a map of where a dish became an IDENTITY MARKER — the thing a venue names itself after — which is a diaspora-and-borderland phenomenon, not a map of where the food is eaten. "Paella" restaurants cluster where Spanish identity is being signalled, which is not only Spain. Everything downstream (classification, colouring, the hover "ray to the nearest venue") exists to make that one idea legible. ----------------------------------------------------------------------------- 1. DATA SOURCES (where everything comes from) ----------------------------------------------------------------------------- All of it is free and openly licensed. Credit OSM and Natural Earth. a) POINTS OF INTEREST — OpenStreetMap planet dump. File: planet-latest.osm.pbf (~80–90 GB) from planet.openstreetmap.org (or a mirror). This is the whole planet's OSM data. You download it ONCE. We keep food venues (amenity = restaurant / fast_food / cafe / food_court) mapped as BOTH points (nodes) and building outlines (ways). That is ~2.9 million points worldwide — outlines are ~18% of them (fast food and food courts skew higher), so nodes-only silently drops ~half a million places. Alternative for a smaller/faster start: the Overpass API can fetch the same venues by tag without the full dump, but it rate-limits hard at global scale. The planet dump + a local query engine is what makes the whole thing run in minutes instead of days. Prefer it. b) LAND / COASTLINE — Natural Earth via the "world-atlas" npm package (land-10m.json, TopoJSON). Used to mask the sea (only land cells get coloured) and to draw the coastline. c) TERRAIN RELIEF — a global elevation model (DEM). We used ETOPO 2022 at 60-arcsec resolution (EPSG:4326, ~1 GB GeoTIFF, free from NOAA NCEI). Any global DEM works; you only need coarse large-scale form because the shader synthesises fine ridge detail at runtime. d) RIVERS — Natural Earth rivers (optional, baked into the terrain image). e) DISH / CATEGORY PHOTOS — Wikimedia Commons (openly licensed, with attribution). One photo per dish for the info cards. f) MULTILINGUAL NAME LISTS — Wikidata (optional but high-value; see §3). ----------------------------------------------------------------------------- 2. STEP ONE — BUILD A LOCAL FOOD-POI DATABASE (do this once) ----------------------------------------------------------------------------- Goal: turn the 87 GB planet blob into one small, fast file you can query instantly for the rest of the project. Tool: DuckDB (single binary, has a `spatial` extension with `ST_ReadOSM`, which reads .osm.pbf directly). Query, in words: - Read the planet .pbf. Keep elements whose `amenity` tag is one of the four food values, as BOTH nodes and ways. - A node's coordinate is the node itself. A way (building outline) carries no coordinate, only node refs — so take its FIRST node (refs[1]): a corner of the outline, cheaper than a true centroid and close enough at map scale. - Namespace the id ('n'||id / 'w'||id): OSM node and way id-spaces overlap, so a bare id would collide downstream. - Project out the columns you'll need: id, lat, lon, and the tag values for amenity, cuisine, and EVERY name variant OSM carries — name, name:en, int_name, alt_name, official_name, loc_name. (You want all name tags: a place's dish word may live in name:en or alt_name, not the default name.) - Write the result to Parquet with zstd compression. Output: food_nodes.parquet — ~80 MB, ~2.9M rows (2.3M nodes + ~0.5M ways). Every later step queries THIS file; the planet dump is never touched again. Gotchas that cost time: - DuckDB's OSM tags are a MAP: `tags['name']` returns the value string directly. Do NOT index it like a list (`tags['name'][1]` string-indexes the value — a classic silent bug). - `ST_ReadOSM` gives NODES a coordinate directly; a WAY only carries its node refs. Resolve a way's location in a second pass that joins refs[1] to that node's lat/lon. Skip only multipolygon RELATIONS — rare, and each needs full geometry assembly. Note there is NO bare `name` column: the name lives in tags['name'] (columns are kind, id, tags, refs, lat, lon, roles). - Point DuckDB's temp_directory at a disk with space — a planet scan spills tens of GB of temp. - Resolving way coordinates joins ~0.5M way-node ids against BILLIONS of planet nodes. Force the tiny id set to be the hash BUILD side or DuckDB buffers the whole planet and fills the disk: materialize the ids into a temp table and disable the build/probe swap (PRAGMA disabled_optimizers='build_side_probe_side') so the node stream stays the probe. Cap max_temp_directory_size as a safety valve. ----------------------------------------------------------------------------- 3. STEP TWO — CLASSIFY EACH VENUE BY THE NAME ON ITS SIGN ----------------------------------------------------------------------------- This is the heart of the project. For a "family" of related dishes (say the rice family: fried rice, plov, pulao, biryani, paella, risotto, jollof, com tam…), decide which dish — if any — each venue's name declares. Design the classifier as TWO layers per dish: 1. A BROAD RECALL NET — a big substring alternation of every spelling and transliteration of the dish, across languages and scripts. Purpose: catch everything, over-match on purpose. Example for fried rice includes "fried rice", "nasi goreng", "chao fan", "khao pad", "com rang", "yakimeshi", "chahan", "bokkeumbap", plus the CJK/Thai/Arabic forms 炒饭 / チャーハン / 볶음밥 / ข้าวผัด, and so on. This is what you run against the database (fast, cheap, forgiving). 2. PRECISE, WORD-BOUNDED REGEXES — for each hit the net returned, re-test with tight rules to (a) confirm the exact spelling and (b) DROP false positives the broad net let in. "\bfried[\s-]*rice\b" won't fire on "Winifred Rice Co." The precise layer is where accuracy lives. Run it as: DuckDB regexp pre-filter on the union of all nets (turn 2.9M rows into a few thousand candidates) → dump to JSON → classify each candidate in your own code with the precise regexes → assign it a dish (or reject it). Keep an AUDIT bucket: names the broad net caught but no precise regex matched. That bucket is your to-do list for missed spellings (recall) vs. noise (precision). Never trust the keyword list blindly; read what OSM actually returns. Hard, interesting classification problems you WILL hit (these are the project's real content — document them for users): - Same word, different dish by country. "kebab" is a döner WRAP in Britain/Australia/Europe but skewered meat ("kabab"/کباب) in Iran/Pakistan/India. Map each spelling only where it's unambiguous, and leave genuinely ambiguous forms (the Turkish/diaspora "kebap") off the map — the sign honestly can't tell them apart. - Same characters, different reading. 餃子 is "gyoza" in Japan but "jiaozi" in China/Taiwan; 点心 is "dim sum" in the Cantonese south but "a snack" in Mandarin. Disambiguate by other cues ON the sign: kana vs. Han, simplified vs. traditional, regional words (北方, 山東). - Generic fallbacks. A vague "noodle"/"kebab"/"dumpling" catch-all should still be classified (so it absorbs generic venues away from the specific dishes) but usually should NOT form its own territory — its sheer count would drown the interesting signals (pad thai in Thailand, etc.). BOOSTING COVERAGE WITH WIKIDATA (optional, high yield): Your hand-written spelling list is biased toward languages you know. To fix that systematically, pull each dish's Wikidata item and harvest its labels and aliases in ALL languages, then fold those into the recall net. This measurably surfaced venues the hand list missed (e.g. large gains for soba and satay). Do it as a non-destructive expansion you can A/B against the hand-only baseline, so you can see exactly what each addition changed. ----------------------------------------------------------------------------- 4. STEP TWO-B — THE "WORLD CUISINES" MAP (a different signal) ----------------------------------------------------------------------------- Alongside the dish maps, build one map from the OSM `cuisine=*` tag instead of the name. This answers "which national cuisine dominates a region's signage." - Fold raw cuisine values into ~29 national cuisines: sushi→japanese, kebab→turkish, tapas→spanish, etc. - EXCLUDE pan-global fast food (pizza, burger, coffee) and vague tags (regional, asian, mediterranean) — they carry no national signal and, left in, they drown the real one. - Because a region can be a genuine patchwork (Germany's restaurants are a near three-way tie of Italian, German and Turkish, ~a fifth each — no majority), the cuisine map legitimately looks noisier than the dish maps. That's real data, not a bug. Default the legend to the top few cuisines and let users toggle. ----------------------------------------------------------------------------- 5. STEP THREE — TURN POINTS INTO COLOURED TERRITORIES (isoglosses) ----------------------------------------------------------------------------- You now have, per map, a cloud of points each tagged with a dish. Colour the whole land surface so the boundaries between colours ARE the isoglosses. The method that survived a long fight (see §9 for what we rejected): NEAREST-CLUSTER on a grid. 1. Aggregate venues into cells of ~0.5° (this is the main knob; the cuisine patchwork map uses a larger ~2.5° cell). Each non-empty cell becomes ONE cluster = its centroid + the MAJORITY dish of the venues in it. 2. Lay a fine grid over the land (~0.1°, about 11 km). Mask to land first (only land cells, using the Natural Earth coastline). 3. Colour each land cell by its NEAREST cluster, using great-circle distance. Why cluster-then-nearest and not plain nearest-venue or a majority window: - Plain nearest-venue gives a lone outlier its own island of colour inside a sea of another dish. Aggregating first absorbs that outlier into the local majority. - But a far-away big cluster must never outvote a genuinely nearer one, so the FINAL colouring is still nearest-among-fixed-clusters. Fixed clusters + nearest gives clean regions without a distant dish bleeding across 1500 km. - The cell size is THE dial: bigger = more aggregation, cleaner, larger zones; smaller = more local detail. Two ways to render those territories: A) BAKE POLYGONS (offline). Trace region boundaries along EXACT integer cell edges (so neighbouring dishes share identical segments), build a TopoJSON topology (shared arcs, zero gaps/holes), then smooth. Smoothing that worked: Douglas–Peucker with a small epsilon (drops the ~half-cell grid "staircase" ripple but keeps real corners exactly, so venue points stay inside their own zone) + a light Chaikin pass to round corners, output at ~0.001° precision. Do NOT use aggressive Visvalingam simplification — it amputates real corners and pushes borders across the points they contain. Ship polygons.geojson per map. B) COMPUTE ZONES AT RUNTIME on the GPU (what the live site does now). Ship only the point cloud; the shader builds the nearest-cluster partition as a planar Voronoi in normalized-Mercator every time the data changes. This drops payload a lot (no polygon files) and lets you recolour instantly as the user toggles categories. It's more work in shaders but far more flexible. Either approach is valid; pick by whether you value a static file or live interactivity. ANTIMERIDIAN / longitude wrap — the invariant that bites everyone: Longitude is periodic. Any "nearest point/seed" search MUST wrap the longitude delta (dLon -= 360 when > 180), or a hard colour seam appears down the 180° meridian. For the Voronoi, replicate every seed at lon−360, lon, lon+360, build over all three copies, clip to one period. The hover ray, the grid search, and the density glow all wrap the same way. QUALITY METRIC: measure "points-out-of-own-zone" (what fraction of venues land in a zone coloured for a different dish). For clean families it's ~5%; for the cuisine patchwork it's ~55% — and that high number is CORRECT (every local minority venue legitimately sits in the majority's zone). It's an aggregation floor, not a bug. The hover ray still points to the nearest venue OF THE ZONE'S dish, so users never see a contradiction. ----------------------------------------------------------------------------- 6. STEP FOUR — THE BASE MAP AND TERRAIN ----------------------------------------------------------------------------- The look is a dark "intel/defcon" map: glowing coastlines and borders, density-lit territories, subtle relief. No third-party map tiles — the base is rendered from your own assets, which keeps it seamless, offline, and on-brand. - Bake ONE terrain image (a WebP, ~0.5 MB) from the DEM: R channel = normalized elevation (kept LOSSLESS — a lossy height shows block artifacts once you derive slope from it), A channel = land coverage (the coastline, also reused as the hover "is this land?" gate). Reproject the equirectangular DEM to Web-Mercator once, in the bake step, so the runtime shader never reprojects (longitude is linear in both projections; only the vertical axis is remapped row by mercator-y). - At runtime the fragment shader height-shades that base per pixel and synthesises fine ridge detail (warped ridged-multifractal + Voronoi) so the relief never blurs on zoom — the stored base only needs coarse form, which is why it can be small. - Rivers are carved into the same terrain image (a small size/quality tradeoff vs. crisp vector rivers). - The very same terrain image doubles as the boot splash `` (with a CSS brightness lift), so the browser fetches ONE asset for both the loading screen and the live canvas. ----------------------------------------------------------------------------- 7. STEP FIVE — THE RENDER ENGINE (drawing it fast) ----------------------------------------------------------------------------- Everything is drawn in ONE shared WebGL2 context with a single canvas, in layer order: terrain → zones → points, then a bloom post-pass over the whole scene so glow spills past the coasts into the ocean. Key pieces (each earned by fixing a real performance wall): - CUSTOM MAP ENGINE, no Leaflet. A small module does Web-Mercator project/unproject, camera (center/zoom/size), mouse pan / wheel-zoom / zoom-to-cursor, infinite longitude wrap, and fires the pan/zoom/resize events the layers listen to. Removing Leaflet + the Pixi overlay it needed cut ~460 KB of JS. Pan = move the camera and re-render; zoom = tween. - POINTS as instanced quads in a raw WebGL2 shader (anti-aliased discs with FIXED-pixel AA, so small dots don't vanish). Decimate to one sprite per ~6 screen-pixel cell so on-screen density is constant at every zoom (bounded sprite count, no mush). Pre-project once per map into typed arrays so the per-frame cull is pure arithmetic. - ZONE FILL GLOW = a world-space distance field via Jump Flooding (JFA). Splat active points into a 1024² texture, jump-flood to get each pixel's nearest point, resolve to a linear falloff clamp(1 − dist/R). The fill samples it by WORLD coordinates, so pan/zoom just re-sample a fixed field — zoom-independent, rebuilt only when categories toggle. (We tried additive Gaussian KDE and additive cones; both looked wrong or overdrew badly. The linear distance field is the one that reads as "evenly lit territory with a soft halo around each point.") - ZONE OUTLINES via edge-detection on the zone buffer, normalized by local contrast and supersampled, so borders have uniform brightness and width regardless of the two colours' contrast. Fills are toned down; outlines keep the vivid saturated colour, so borders stay crisp as fills sink back. - CATEGORICAL COLOURS from a hand-tuned, maximally-distinct pool, allocated lazily when a category is enabled and freed when disabled (so toggling one cuisine never recolours the others). A fixed by-index hue ramp fails at 29 categories — the steps get too small to tell Korean from Thai from Vietnamese. PAYLOAD: ship the point cloud as a compact binary, not GeoJSON. Format: a tiny header, then zig-zag varint DELTAS of the rounded coordinates (rows spatially sorted so deltas are tiny), then one code byte per point packing (family, dish). Gzip on top. This took ~1.8 MB of JSON down to ~280 KB on the wire. - One subtlety: the file is spatially SORTED for compression, but the render decimation keeps the first point per screen cell — on a sorted array that produces an ugly regular grid of dots. So the decoder SHUFFLES the points with a fixed seed after decoding. File order = compression; array order = natural scatter. Keep them separate. ----------------------------------------------------------------------------- 8. STEP SIX — THE APP SHELL ----------------------------------------------------------------------------- - Framework: Nuxt 3 (Vue), SSR on, statically generated. - Routing: /[lang]/[mode] — one page, e.g. /en/rice, /ru/cuisines. The MODES are the dish families; two LANGS (English, Russian). Switching family or language REUSES the same map instance (stable page-key) and swaps only the zones + points, keeping the camera where the user left it — remounting reset the pan/zoom and felt broken. - i18n is a tiny custom composable reading /locales/{en,ru}.json (UI strings, dish labels, and a short description per dish). - HOVER interaction: a dashed ray from the cursor to the nearest venue OF THE HOVERED ZONE'S dish, plus the distance in km. The "which dish is under the cursor" lookup is O(1) against a rasterized dish-index grid, not a per-move point-in-polygon scan (that scan was the last CPU cost on the dense cuisine map). - Panels: a legend (click a category → detail card with photo + description; hover → highlight all same-dish regions), a "Method" explainer, layer toggles (Territories / Points), and an image-export tool that renders a shareable framed screenshot. ----------------------------------------------------------------------------- 9. THINGS THAT DID NOT WORK (skip these; we already paid for them) ----------------------------------------------------------------------------- - Plain nearest-VENUE colouring → lone outliers get their own colour islands. Aggregate into clusters first. - Majority-in-a-window / k-nearest / distance-cap colouring → a far cluster could outvote a near venue, or the colour flickered as the cursor moved. Nearest-among-fixed-clusters avoids both. - Matching a global Voronoi to a per-pixel flat-earth "nearest" metric → 95–98% agreement, never 100%. If you want exactness, bake polygons from the same grid the field uses. - Aggressive polygon simplification (Visvalingam) → borders shift inward and swallow the very points they should contain. Use corner-preserving Douglas–Peucker instead. - Additive Gaussian KDE / additive cones for the territory glow → energy conservation fades wide radii to nothing, plus banding and overdraw. Use a Jump-Flood distance field. - Fixed by-index hue ramp for many categories → indistinguishable colours. Use a hand-tuned distinct pool with lazy allocation. - Forgetting to wrap longitude anywhere in a nearest-search → a colour seam down the antimeridian. Wrap it EVERYWHERE. - Third-party map tiles → seams, external dependency, and the wrong look. Render your own terrain. ----------------------------------------------------------------------------- 10. DEPLOY ----------------------------------------------------------------------------- - Static site (Nuxt generate) → any static host. We use Cloudflare Pages, deployed by git push; only the web/ folder is the deployed root. - Social share images (Open Graph): pre-render a full-bleed screenshot of the real WebGL map per mode and language (points off, UI hidden), then overlay the brand chrome. One PNG per mode × language. ----------------------------------------------------------------------------- 11. AT A GLANCE — the whole pipeline in one breath ----------------------------------------------------------------------------- planet.osm.pbf (~87 GB, once) → DuckDB ST_ReadOSM, keep food nodes + ways (way → first-node coordinate) → food_nodes.parquet (~2.9M points: 2.3M nodes + ~0.5M outlines) → per family: broad name-net pre-filter → precise regex classify (+ Wikidata multilingual expansion; + a cuisine-tag map) → nearest-CLUSTER field on a 0.1° land grid (great-circle, wrap lon) → EITHER bake smoothed TopoJSON polygons OR compute zones at runtime → ship point cloud as gzipped binary + dish metadata JSON → WebGL2 scene: baked-terrain (from a DEM) → Voronoi zones → point cloud → bloom + JFA density glow, custom map engine, no tiles → Nuxt app: /[lang]/[mode], hover ray to nearest venue, legend cards → static build → Cloudflare Pages Data: OpenStreetMap (venues), Natural Earth (coastline, rivers), ETOPO 2022 (relief), Wikidata (multilingual names), Wikimedia Commons (photos). All open. Credit them. Live map: https://isoplov.vova.today