Go에서 GitHub 검색창 구축 시도

Hello, I’m Maneshwar. I’m building git-lrc, a Micro AI code reviewer that runs on every commit. It is free and source-available on Github. Star git-lrc to help devs discover the project. Do give it a try and share your feedback.

You know that little search box on GitHub issues? The one where you type is:open label:bug author:me created:>2024-01-01 and it just gets it? I always assumed there was some gnarly hand-rolled parser behind it, held together with regex and prayers.

So I decided to build my own version in Go, using a neat little library called participle (by Alec Thomas) that lets you define a parser as a plain Go struct.

Turns out the whole thing is surprisingly little code, and most of it reads like the grammar you would sketch on a napkin.

By the end of this post we will have a tiny tool that takes a search string, turns it into a typed syntax tree, and compiles that tree into safe, parameterized SQL.

Here is the money shot, so you know where we are headed:

$ make run q='is:open label:bug'
query: is:open label:bug
6 issue(s)

#   STATE  AUTHOR   CREATED     COMMENTS  LABELS         TITLE
1   open   octocat  2024-02-10  12        bug,urgent     Login button unresponsive on mobile
6   open   bob      2024-04-18  15        bug,p0,urgent  Memory leak in worker pool
8   open   octocat  2024-09-03  9         bug            Flaky test in CI
12  open   bob      2024-07-19  6         bug,p0         Rate limit not enforced
15  open   alice    2024-11-08  11        bug,urgent     Search returns stale results
18  open   bob      2024-10-22  10        bug,p0         Slow query on dashboard

Enter fullscreen mode Exit fullscreen mode

Let’s build it.

The filter string nobody wants to parse

Here is the version we all write first, be honest:

// the "I'll clean it up later" special
parts := strings.Fields(query)
for _, p := range parts {
    if strings.HasPrefix(p, "label:") {
        labels = append(labels, strings.TrimPrefix(p, "label:"))
    } else if strings.HasPrefix(p, "is:") {
        state = strings.TrimPrefix(p, "is:")
    }
    // ... 40 more else-ifs, each one a tiny betrayal
}

Enter fullscreen mode Exit fullscreen mode

This works right up until someone types label:"help wanted" with a space in it. Or -author:bot to exclude a user. Or created:>2024-01-01 with a comparison.

Every one of those pushes you deeper into regex territory, and hand-rolled regex for this kind of thing has a way of solving one edge case and quietly creating two more.

Once you stop treating this as string munging and start treating it as an actual parsing problem, the solution gets a lot friendlier.

Participle: a grammar is just a struct

participle has a lovely core idea: you describe your grammar as a struct, annotate the fields with tags, and it builds the parser for you.

No separate .y grammar file, no code generation step.

The struct is the grammar.

Here is a taste. If you wanted to parse a single key:value pair, you would write:

type Pair struct {
    Key   string `parser:"@Ident"`
    Colon string `parser:"@Colon"`
    Value string `parser:"@Ident"`
}

Enter fullscreen mode Exit fullscreen mode

The @ means “capture this into the field.”

That’s basically the whole trick.

Now let’s scale it up to a real search language.

The AST: three little structs

Our search syntax is what I call “GitHub minimal”: qualifiers like is:open, comparisons like comments:>=5, free text like login, negation with a leading -, and implicit AND between terms.

No OR, no parentheses (we will talk about those at the end).

The entire grammar fits in three structs:

// Query is the whole thing: a flat list of terms, all ANDed together.
type Query struct {
    Terms []*Term `parser:"@@*"`
}

// Term is one search unit: a key:value qualifier OR a free-text word,
// with an optional leading "-" to negate it.
type Term struct {
    Negated   bool       `parser:"@Dash?"`
    Qualifier *Qualifier `parser:"( @@"`
    Text      *Value     `parser:"| @(String | Ident | Number) )"`
}

// Qualifier is "key:value", e.g. is:open or created:>2024-01-01.
type Qualifier struct {
    Key string `parser:"@Ident Colon"`
    Op  string `parser:"@Op?"`
    Val *Value `parser:"@(String | Date | Number | Ident)"`
}

Enter fullscreen mode Exit fullscreen mode

Read those tags out loud and they say what they mean.

@@* is “zero or more sub-structs.” The ? after @Dash makes the negation optional.

The ( @@ | @... ) in Term is an alternation: a term is either a qualifier or a bare word.

What I like about this approach is that the AST doubles as documentation.

When a teammate asks what the search syntax supports, I point them at these three structs instead of a README that’s probably out of date, because the structs are the parser and can’t drift from what actually runs.

There is one subtle line doing a lot of work when we build the parser:

var parser = participle.MustBuild[Query](
    participle.Lexer(searchLexer),
    participle.Elide("whitespace"),
    participle.UseLookahead(2),
)

Enter fullscreen mode Exit fullscreen mode

That UseLookahead(2) matters.

Both login (free text) and label:bug (a qualifier) start with an identifier.

The parser needs to peek ahead and check for a : before it decides which branch to take.

Lookahead of 2 lets it see the colon coming.

How the lexer handles the dash

The tokenizer has one detail worth slowing down for, because the dash character does three different jobs depending on where it shows up:

1. -author:bot -> the leading dash is negation
2. help-wanted -> the dash is part of a label name
3. 2024-01-01 -> the dashes are date separators

If your lexer treats every - as a negation token, help-wanted becomes “help, NOT wanted,” and your date turns into subtraction.

Neither of those is what you want.

The fix is worth understanding on its own, because it’s a technique you’ll reuse elsewhere.

participle’s simple lexer is an ordered list of regex rules, and it takes the first one that matches. Order decides everything:

var searchLexer = lexer.MustSimple([]lexer.SimpleRule{
    {Name: "whitespace", Pattern: `\s+`},
    {Name: "String", Pattern: `"(\\.|[^"])*"`},
    {Name: "Date", Pattern: `\d{4}-\d{2}-\d{2}`},
    {Name: "Number", Pattern: `\d+`},
    {Name: "Op", Pattern: `>=|<=|>|<`},
    {Name: "Colon", Pattern: `:`},
    {Name: "Ident", Pattern: `[a-zA-Z_][a-zA-Z0-9_./-]*`},
    {Name: "Dash", Pattern: `-`},
})

Enter fullscreen mode Exit fullscreen mode

Notice that Date, Number, and Ident all come before the bare Dash rule, and every one of them must start with a digit or a letter.

Walk through -author:bot one character at a time.

The lexer is sitting at the -. It tries Date, which needs a digit first, so no. It tries Number, same problem. It tries Ident, which needs a letter or underscore, and - is neither. Nothing longer can start here, so the plain Dash rule finally gets its turn, and that becomes our negation token.

Now walk through help-wanted.

The lexer starts at h, tries Ident, and the Ident pattern swallows the internal dash because I included - in its character class. One token, help-wanted, done.

The date 2024-01-01 gets grabbed whole by the Date rule before Number or Dash ever get a look.

So the dash only becomes a negation token when it literally cannot be anything else.

No special cases in the parser, no lookbehind, no state machine — just careful ordering of regex rules.

Here is the whole pipeline, from raw string to rows:

Compiling the tree into SQL

A parsed tree is nice, but nobody can query a Go struct directly.

We need SQL.

This is where the AST pays off, because walking a typed tree and emitting strings is fairly mechanical work, which is exactly what you want near a database.

The heart of the compiler is a registry that maps each qualifier key to a column and a comparison style:

var registry = map[string]fieldDef{
    "is":        {column: "state", kind: kindString},
    "state":     {column: "state", kind: kindString},
    "author":    {column: "author", kind: kindString, userAware: true},
    "assignee":  {column: "assignee", kind: kindString, userAware: true},
    "milestone": {column: "milestone", kind: kindString},
    "label":     {column: "", kind: kindLabel},
    "created":   {column: "created_at", kind: kindDate},
    "updated":   {column: "updated_at", kind: kindDate},
    "comments":  {column: "comment_count", kind: kindNumber},
}

Enter fullscreen mode Exit fullscreen mode

Adding a new searchable field means adding one line here.

The grammar doesn’t change, and neither does the lexer.

Each term compiles to a small SQL fragment plus its arguments:

switch def.kind {
case kindString:
    // is:open -> state = ?
    return def.column + " = ?", []any{value}, nil

case kindDate, kindNumber:
    // created:>2024-01-01 -> created_at > ?
    op := qf.Op
    if op == "" {
        op = "="
    }
    return def.column + " " + op + " ?", []any{value}, nil

case kindLabel:
    // labels live in a join table, so we ask "does a matching row exist?"
    sub := "EXISTS (SELECT 1 FROM issue_labels il " +
        "JOIN labels l ON l.id = il.label_id " +
        "WHERE il.issue_id = issues.id AND l.name = ?)"
    return sub, []any{value}, nil
}

Enter fullscreen mode Exit fullscreen mode

Free text (a bare login) becomes a LIKE against the title and body.

Negation wraps the whole fragment in NOT (...). Implicit AND joins the fragments with AND. That’s the entire compiler.

The part where I stop you from getting hacked

Look carefully at those returns.

The column names come from my registry, which is fixed code.

The user’s value never gets glued into the SQL string. It goes into a separate []any of arguments, and the SQL only ever contains a ? placeholder.

Keeping user input and SQL structure in two separate places is what actually prevents injection here — it’s not a cosmetic choice.

There is a --sql flag in the tool that prints exactly what gets sent to the database, so you can see the safety with your own eyes.

Watch what happens to a spicy query full of user input:

And a bigger one with negation, a date, a label join, and free text all at once:

Every value the user typed ends up in args.

The SQL itself is all structure and question marks.

If someone types author:'; DROP TABLE issues; -- it just becomes a string that matches zero authors, and your tables are unaffected.

By the way, author:me quietly became octocat in that first example.

The compiler resolves me to the current user, the same way GitHub does.

What the parsed tree actually looks like

If you are a visual thinker, here is the AST for is:open label:bug -author:bot drawn out. This is what those three structs produce in memory:

Three terms, hanging off one query, the third carrying a Negated: true flag.

The compiler walks this top to bottom and stitches the fragments together.

Making it real with SQLite

I wired this to a real database so it isn’t just a toy that prints ASTs.

I used modernc.org/sqlite, a pure Go SQLite driver, which means no CGO, no C compiler, and go test just works on any machine.

The schema is three tables: issues, labels, and an issue_labels join table, which is exactly the thing that makes label: interesting to compile.

There is a seeded sample dataset with twenty issues so every example in this post is reproducible.

Run make demo and it fires a whole battery of queries at the database. A few highlights:

That second query, -author:bot is:closed, taught me a separate lesson. When I first ran it through the CLI, it errored out, because a query that starts with - looks like a command line flag to Go’s flag package.

The fix is the standard POSIX -- separator (go run . -- "-author:bot is:closed") — the same “what does this dash mean” ambiguity from the lexer, showing up again one layer down at the command line.

Where this goes next

Right now there is no OR and no parentheses, on purpose, to keep the first version teachable.

Adding them is a satisfying next step: you introduce an OrExpr and AndExpr layer above the terms, and you get operator precedence essentially for free from how the structs nest.

Same AST idea, one more level deep.

Because the AST sits in the middle as a clean seam, SQL isn’t the only possible backend.

You could compile the same tree into an in-memory func(Issue) bool for filtering a slice, or into an Elasticsearch query, or a Bleve query.

Parse once, target anything.

The whole thing is about 200 lines of Go: three structs for the grammar, one ordered lexer, one registry for the compiler.

That’s the entire GitHub-style search box, minus the parts that would get me sued.

A big shoutout to Alec Thomas for participle.

The whole “your grammar is just a struct” idea is what made this fun instead of a chore, and his library quietly did all the heavy lifting while I got to take credit for it in a blog post.

All the code, the seeded database, and the tests are here: github.com/lovestaco/gh-search-dsl.

Clone it, run make demo, and try adding a new qualifier, it should only take one line.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs — without telling you. You often find out in production.

git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It’s online, source-available, and ready for anyone to use.

⭐ Star it on GitHub:

GenAI today is a race car without brakes. It accelerates fast — you describe something, and large blocks of code appear instantly. But AI agents silently break things: they remove logic, relax constraints, introduce expensive cloud calls, leak credentials, and change behavior — without telling you. You often find out in production.

git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.

In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen

At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…