Early accesslast update deployed 20 Aug 2026, 11:49 am ACST

How we identify and rank sources

When you ask a question, CouncilTracer doesn't rely on a single search. It runs two independent searches over City of Mitcham's whole public record — one that matches your words, one that matches your meaning — and trusts a source in proportion to how much the two agree. Two very different methods pointing at the same document is hard to fake, and it's what lets an answer stand behind every source it cites.

Sketchnote of Reciprocal Rank Fusion in hybrid search: a keyword result list and a semantic result list are combined by scoring each document 1/(k + its rank) in every list and summing, so documents ranked highly by both searches rise to the top.
Reciprocal Rank Fusion, sketched by Guillaume Laforge — “Understanding Reciprocal Rank Fusion in hybrid search”. © Guillaume Laforge, shown with credit.

Here's the same idea on City of Mitcham's own record, with a real question: “What are the rules for dogs off-leash?”

Two searches, each its own way

One matches the words you used. The other matches what you meant. Each hands back its own ranked shortlist.

The Words search

Matches your exact words— “dogs”, “off-leash”. Fast and literal, but blind to meaning: it can't tell a rule from a passing mention.

  • 1Minutes — Item 10.4, dog off-leash areas
  • 2Community News — “Hot dog stall at the fair” · the word, wrong topic
  • 3Animal Management By-law
  • 4Dog registration fees schedule

The Meaning search

Matches the idea, even in different words — it knows “animal management” and “dog exercise areas” are the same subject, and that a hot-dog stall isn't.

  • 1Animal Management By-law · where the rule actually lives
  • 2Minutes — Item 10.4, dog off-leash areas
  • 3Parks & Reserves Policy
  • 4Dog registration fees schedule

How the two rankings become a confidence score

A source both searches rank highly rises to the top; one only a single search found sits lower. The method is Reciprocal Rank Fusion: a source scores for every list it appears in, and more for appearing nearer the top — 1 / (k + words rank) + 1 / (k + meaning rank). No cleverness, just arithmetic — which makes it easy to explain and hard to game.

  • Animal Management By-law

    The rule itself — filed as a by-law, not a “policy”

    words #3meaning #1

    highest

  • Minutes — Item 10.4

    What council actually decided

    words #1meaning #2

    high

  • Parks & Reserves Policy

    Related context

    meaning #3

    medium

  • Dog registration fees

    Mentions dogs, not the question

    words #4meaning #4

    low

the relevance floor — nothing below this reaches the answer

“Hot dog stall at the fair”

The Words search's false match — dropped, because the Meaning search never rated it

words #2

below bar

Agreement is confidence

The sources that reach an answer are the ones two independent methods both vouched for — not one lucky keyword hit.

Meaning catches what words miss

A rule filed as a “by-law” instead of a “policy” still surfaces — and a hot-dog stall doesn't.

A floor keeps weak matches out

Anything that clears neither bar is left out, so an answer is built only from sources it can stand behind — each one cited.

Reciprocal Rank Fusion is a standard, well-documented information-retrieval technique — we're glad to be open about it. What's ours is the council-specific judgement layered on top: knowing a by-law isn't a policy, that a briefing carries no votes, that “this term” has exact dates. ← Back to how CouncilTracer works