One Name, Two Languages: Giving Siunertaq a Canonical Namespace.

This is the third post in a series on Siunertaq, a small Scala 3 project that
models computation as a stack machine and cross-validates it across languages.
Previously: One Stack Machine, Two Runtimes: Cross-Validating Scala and Perl with a Shared JSON IR
introduced the batch layer; the JSON IR post covered how PerlBridge and
Siunertaq::StackMachine.pm cross-validate the same Program in Scala and Perl
via a shared JSON format.
This post is about a smaller, quieter problem that shared JSON doesn’t solve on
its own: once two languages agree on *what ran, how do you name the thing that
ran the same way in both of them?*

🎯 The Problem Shared JSON Doesn’t Solve

The last post left things in a good place. ClassASTBridge reads a .class file
and emits StackInstr JSON. Program.toJson does the same from the Scala side.
Siunertaq::StackMachine->execute_json consumes that JSON in Perl. Three
producers/consumers, one wire format, no code generation.

But once both sides are pushing rows into ClickHouse — bytecode_instructions
for JVM extractions, forth_words for compiled Forth definitions — a new
question shows up that JSON doesn’t answer: how do you find the row that
corresponds to the same logical thing?

Concretely: io.siunertaq.expr.Program‘s toJson method, on the JVM side, is
identified by class_name = "io.siunertaq.expr.Program",
method_name = "toJson". The Perl mirror of that same logic, once it exists,
would live in a package like Siunertaq::Expr::Program, in a sub named
to_json. Same concept. Completely different strings. WHERE class_name =
method_name was never going to work, and neither was anything close to it.

We needed a name that both sides could compute independently and land on
the same value — a join key that doesn’t require either side to know
anything about the other’s naming convention, only about a shared,
neutral one.

❌ The Tempting Wrong Answer: A Reversible Mapping

The instinctive fix is to write a bijection: given a JVM name, derive the exact
Perl name it corresponds to, and vice versa. It’s tempting because it feels more
complete — you’d get a real two-way translator, not just a comparison key.

It falls apart on the return trip. Going JVM → Perl is mostly fine:
toJson → to_json is an unambiguous, mechanical camelCase-to-snake_case
conversion. Going Perl → JVM is not:

to_json  →  toJson   ? toJSON   ? ToJson   ?

Enter fullscreen mode Exit fullscreen mode

All three are plausible JVM spellings of the same snake_case name, and Perl
code has no way of telling you which one the original author picked. A
“reversible” mapping that silently guesses wrong for some fraction of names is
worse than no mapping — it would produce canonical_name collisions and
splits that look authoritative but aren’t.

So we gave up on reversibility as a goal entirely.

💡 The Actual Design: Fold, Don’t Translate

Instead of mapping JVM ↔ Perl, NamespaceCanon maps both down to a single,
neutral, lossy key. Neither side is treated as authoritative over the other;
both are folded toward the same normal form:

object NamespaceCanon:

  private val JvmRootPrefix  = "io.siunertaq."
  private val PerlRootPrefix = "Siunertaq::"

  def fromJvm(className: String, methodName: String): String =
    val stripped = className.stripPrefix(JvmRootPrefix).stripSuffix("$")
    val segments = stripped.split('.').filter(_.nonEmpty).map(_.toLowerCase)
    (segments :+ camelToSnake(methodName)).mkString(".")

  def fromPerl(perlPackage: String, subName: String): String =
    val stripped = perlPackage.stripPrefix(PerlRootPrefix)
    val segments = stripped.split("::").filter(_.nonEmpty).map(_.toLowerCase)
    (segments :+ subName.toLowerCase).mkString(".")

  private def camelToSnake(s: String): String =
    s.replaceAll("([a-z0-9])([A-Z])", "$1_$2").toLowerCase

Enter fullscreen mode Exit fullscreen mode

Both calls below converge on the same string, which is the only property
that actually matters:

fromJvm("io.siunertaq.expr.Program", "toJson")
  == fromPerl("Siunertaq::Expr::Program", "to_json")
  == "expr.program.to_json"

Enter fullscreen mode Exit fullscreen mode

Nobody has to guess to_json‘s JVM spelling, because nobody is trying to
recover it. fromPerl doesn’t reconstruct toJson; it just lowercases
to_json and leaves it alone, betting — correctly, for this codebase’s
naming convention — that the JVM side would fold to the same string via its
own camelCase→snake_case rule. The two functions don’t need to agree on an
inverse. They only need to agree on a fixed point.

There’s a second, explicitly non-reversible pair of helpers,
suggestPerlPackage / suggestPerlSub, for the opposite situation: scaffolding
a new .pm file from an existing Scala class. Those are best-effort code
generation helpers, not part of the join-key contract, and the naming makes
that distinction obvious at the call site.

📦 Where It Actually Lands: ClickHouse

canonical_name had to exist somewhere queryable, not just as a Scala
function. ClickHouseSync computes it once, at the JSON-serialization
boundary, and never touches the underlying row types to do it:

private def withCanonicalName(row: MecrispCompiler.BytecodeRow): Json =
  row.toJson.mapObject(
    _.add("canonical_name", NamespaceCanon.fromJvm(row.className, row.methodName).asJson)
  )

Enter fullscreen mode Exit fullscreen mode

MecrispCompiler.BytecodeRow and MecrispWordDef stay exactly as they were;
canonical_name is injected into the JSON on the way out, which meant this
shipped without touching either of those types.

On the ClickHouse side, a migration adds the column to both tables and
backfills existing rows with a SQL expression that mirrors the Scala logic
sentence-for-sentence — replaceRegexpAll for the camelCase split, lower()
for the fold:

ALTER TABLE bytecode_instructions
    ADD COLUMN IF NOT EXISTS canonical_name LowCardinality(String) DEFAULT '';

ALTER TABLE bytecode_instructions
    UPDATE canonical_name =
        concat(
            lower(replaceAll(class_name, 'io.siunertaq.', '')),
            '.',
            lower(replaceRegexpAll(method_name, '([a-z0-9])([A-Z])', '\\1_\\2'))
        )
    WHERE canonical_name = '';

Enter fullscreen mode Exit fullscreen mode

Keeping the Scala version as the one authoritative implementation and the SQL
as “a faithful translation of that logic for in-database use” (it says so
right in the migration’s header comment) was a deliberate choice — the two
were never going to be enforced identical by the type system the way
Program.toJson and Siunertaq::StackMachine->execute_json are, so the
comment does the job a compiler can’t.

The payoff is a view:

CREATE OR REPLACE VIEW cross_language_equivalences AS
SELECT
    canonical_name,
    countIf(language = 'jvm')  AS jvm_count,
    countIf(language = 'perl') AS perl_count,
    ...
FROM ( ... UNION ALL of bytecode_instructions and forth_words ... )
GROUP BY canonical_name
HAVING jvm_count > 0 AND perl_count > 0;

Enter fullscreen mode Exit fullscreen mode

🚧 The Honest Status: One Hand Clapping

Here’s the part that’s easy to gloss over and shouldn’t be: this view is
empty of anything interesting right now. HAVING jvm_count > 0 AND
perl_count > 0 means a canonical name only shows up once both sides have
written a row under it — and only the JVM side writes rows today.
forth_words even grew a language column with a default of 'jvm' in
anticipation of this, but nothing populates language = 'perl' yet.

That’s not a bug, it’s a sequencing decision. Building the fold function and
the schema first, before the second producer exists, meant the join key’s
design could be tested against real JVM data and a NamespaceCanonSpec before
committing to it. The alternative — building the Perl-side writer first —
would have meant designing the canonical format against a single example
instead of the actual shape of two independent naming conventions.

🔧 A Related, Less Glamorous Fix: ClassASTBridge Had to Stop Guessing

None of the above is worth much if the JVM-side rows it’s keying aren’t
trustworthy. While wiring up canonical_name, we found ClassASTBridge had
two quiet correctness gaps: an unrecognized opcode was silently dropped from
the output instead of failing, and a target method name that happened to be
overloaded had its bytecode silently merged across every overload. Both are
“confidently wrong” — the caller gets back a JSON array that looks complete.

Since the whole point of canonical_name is “this row is a faithful record
of what the JVM ran,” it made sense to close both gaps at the same time:

final case class UnsupportedBytecodeException(
  className: String, methodName: String, methodDescriptor: String, opcode: Int
) extends RuntimeException(...)

final case class OverloadedMethodException(
  className: String, methodName: String, descriptors: List[String]
) extends RuntimeException(...)

Enter fullscreen mode Exit fullscreen mode

An unsupported opcode now aborts extraction instead of vanishing from the
array; an ambiguous overload now aborts instead of conflating two methods’
bytecode into one instruction stream (pass targetDescriptor, e.g. "(I)I",
to disambiguate). A ClassASTBridgeSpec exercises both failure paths against
.class files compiled in-memory by the JDK’s own javac, rather than
hand-crafted byte arrays.

🔮 What’s Next: PmASTBridge

The obvious next step is the thing cross_language_equivalences is actually
waiting on: a Perl-side bridge that reads a .pm file’s AST, computes
NamespaceCanon.fromPerl for each sub, and writes forth_words rows tagged
language = 'perl'. Once that exists, the view stops being a one-sided
placeholder and starts being the thing it was built to be — a live diff
surface between what the JVM runs and what Perl runs, keyed on a name neither
side had to learn from the other.

Code is at github.com/Yoshyhyrro/Siunertaq.
NamespaceCanon lives in modules/core; the ClickHouse migration is
modules/postgres-bridge/extension/clickhouse_schema_v2_canonical_name.sql.
As always, issues and PRs welcome — particularly if you’ve solved the
Perl→JVM name-guessing problem in a way that isn’t “don’t.” 🙏