Skip to content
TrustList
Blog

How we now decide who belongs in a ranking, and who leads it

Editorial

By TrustList Editorial

Place tags put firms from London, Ontario, on our London rankings. Now place takes evidence, no business leads more than five rankings, and reviews count only from accountable reviewers.

About How we now decide who belongs in a ranking, and who leads it

How we now decide who belongs in a ranking, and who leads it

A ranking can be wrong in two ways. It can put the wrong businesses on the page, or it can put the right businesses in the wrong order. Our old rankings did both, and the reasons were the same in each case: we trusted labels where we should have looked for evidence. We also let a small number of signals carry far more weight than they had earned.

The Q4 2026 edition changes four rules. A business now belongs on a place ranking only if we can place it there. A business can lead at most five rankings. A client review counts only when the reviewer is accountable for it. And every ranked entry has to say why it is ranked. This article explains each rule, the problem it fixes, and the cases that convinced us.

Tags told us where a business said a word, not where it was

Most of the businesses in our catalogue arrived years ago with tags attached, and many of those tags were scraped from text. A firm whose website mentioned a client in Toronto might carry a "toronto" tag; a firm that wrote "we serve the US and Europe" might carry "us", and so might a firm whose text contained the word "us" in any sentence at all. When a ranking for a city is built from tags, it collects every business that ever used that city's name.

Our page for accountants in London showed how far this went. On 5 October it listed 45 firms, and our own profiles placed about 15 of them in London. The rest included firms in London, Ontario, and others in Winnipeg, Queensland, Milan and Mumbai. Its first place went to a firm based in Abu Dhabi. A page for accountants in York had a quieter version of the same fault, because a "york" tag also matched firms whose text only mentioned New York.

Tags also confused places with ordinary words. Nice is a city on the Riviera and an English adjective, and a tag cannot tell which one a business meant.

Place evidence: what puts a business on a place ranking

Since the first edition, a business is on a city, region or country ranking only when we have evidence that it is there. The evidence comes in three kinds, and the ranking page names the kind on each entry.

The first is the business's headquarters as its own website states it. Our rewritten profiles record this in a "Based in" line, read from the business's site, such as "Based in Clapham and Elephant & Castle, London". The second is an office: a profile that names offices in Brighton and London puts the firm on both rankings and says "Office in Brighton" on the first. The third is a postal address held on the listing.

Each place is given a country from the evidence of its own members. Because nearly all the businesses with evidence of being in London are in England, the London ranking belongs to the United Kingdom, and a firm whose only London is in Ontario no longer qualifies. We keep a list of names that need this care, places that share a name with another place or with an ordinary word, and the matcher checks it before it accepts a place.

On 6 October our place data held 141,698 memberships over 4,299 place terms. It is rebuilt every night, so a business whose profile is rewritten today is placed correctly tomorrow.

A few exceptions are deliberate. A business that has claimed its listing keeps the place tags it set itself, because it can correct them and is answerable for them. Investors and recruiters use their own place rules, built from the cities and regions they say they invest in or hire for. Web hosts keep their data-centre regions, which were checked by hand. Broad regions such as Europe or EMEA are still tags, because a firm that serves Europe is not "in" Europe in the way a firm is in Leeds.

When a page has too few businesses left

Applying the rule had a cost. Some place rankings lost nearly all their members, because none of them could be placed there. A ranking with no evidenced members cannot be published honestly.

We did not want those pages to become dead ends, because some of them were pages people found in search. So a place ranking that falls below the bar now sends visitors to the closest live ranking for the same service. Usually that is the country page: a search for design agencies in Newcastle lands on our ranking of design agencies in the United Kingdom, which has 244 members and 78 ranked. When there is no country page for that service, the visitor goes to the category hub.

In October we checked every ranking address that appeared in our Search Console data for the rankings section. Each one now loads a ranking or moves to the closest live one. Where a town ranking we removed comes back with enough evidenced firms, it returns at the same address.

Five leads at most

The second rule came from a single business. One app developer in our catalogue was first on all 27 of the rankings it appeared in, from app developers in Karachi to apps for sport, science, blockchain and real estate. It had tagged itself into every subject it could, and on each of those pages it had the strongest score.

It was not alone. Before the change, one company was first on 46 rankings, and about a dozen led 16 or more. A ranking where the same name is first on every page tells a reader very little, and it shuts every other business out of the place that people notice most.

From the Q4 2026 edition, no business is first on more than five rankings. It keeps first place on the five where its lead over the next business is largest, which are the places where its first is most clearly earned. On every other page the next ranked business is first, and the entry that would have led says why it does not: "Placed below first here: a listing leads at most 5 rankings". The cap also applies to the SME view of every page.

The cap moved 4,744 first places in the edition.

There is one known exception. In a few small software categories, the only ranked products are the same three or four across many subject pages, so first place cannot be shared out without promoting a product we cannot rank. In those categories a product can still lead up to eight pages. The fix is more evidence for the other products, not a looser rule.

Reviews that count, and reviews that show

The third rule is about reviews. A client review is the strongest evidence a ranking can have, and so it is the one most worth faking.

When we looked at the reviews behind the rankings, the pattern was clear. Of the eleven published client reviews that counted towards a ranking, eight came from accounts on free webmail services. The leading app developer above had two of them, each from an account created minutes before its review appeared (three minutes in one case, ten in the other), and neither account ever posted anything else.

From 6 October, a client review counts towards a ranking only when the reviewer is accountable for it. Either the reviewer signed in with LinkedIn, or the reviewer's account is on a company email domain. Accounts on public mailboxes such as Gmail, Outlook, Yahoo or Proton do not count. Nor does an account on the reviewed business's own domain, because that is the business reviewing itself.

We chose those two tests because each ties a review to a person who can be found. A company email address means the reviewer works somewhere that controls the domain, and a LinkedIn sign-in links the review to a public professional profile. Neither proves the review is fair, but both make a fake one costly in a way that a fresh webmail account does not.

These reviews are not deleted or hidden. They still appear on the business's profile, where a reader can weigh them. They just earn no score and no rank number.

The rule removed a lot of the review evidence we held. On 6 October only two businesses had reviews that met it. We think that is the right starting point: a small amount of evidence we can stand behind is worth more than a large amount we cannot. Businesses that want their reviews to count can invite clients through their profile, and clients who sign in with a work address or LinkedIn will count from the next edition.

What counts instead

If tags and webmail reviews no longer carry a ranking, something has to. Mostly it is facts the business states on its own website, which we read when we write its profile and date when we read them.

For a firm, that means its founding year or its years in business, its team size, its client and project counts, and its certifications: ISO 9001 and ISO 27001, Cyber Essentials, professional bodies such as ICAEW and ACCA, and partner status with AWS, Google, Microsoft, Shopify, HubSpot or Salesforce. For a software product, it means its launch year, SOC 2 Type II and ISO 27001 certification, GDPR and HIPAA compliance, round-the-clock support, an uptime commitment, and the number of major incidents on its public status page over the last 90 days, which we count ourselves.

We treat these facts with some care. A fact a business states about itself is worth less than one we have verified, and the methodology gives it 0.8 of the weight. A fact older than twelve months counts as unconfirmed until the business confirms it again. A plan is not a fact: "working towards ISO 27001" earns nothing. A certificate held by a vendor's hosting provider is not the vendor's certificate. A product that helps its customers become GDPR compliant is not thereby GDPR compliant itself. And when a firm says it has "more than 20 years" of experience, we print "In business 20+ years" rather than converting it into a founding year it never gave us.

On 6 October, 20,165 companies and 1,735 software products had at least one fact of this kind.

The reason on every entry

The last rule ties the others together. Every ranked entry on an edition page carries a short reason built from the evidence that put it there. On our London accountants ranking, the first-placed firm's reason reads "Based in London · Founded 1888 · Independent firm". A software product's might read "ISO 27001" or "No major incidents in 90 days (status page)".

The reason does two jobs. It lets a reader check us: if the founding year is wrong, the reader can see it and tell us. And it shows every other business on the page what it would take to move up. Below the ranked entries, each unranked business is listed with what it is missing, usually client reviews and vetting, sometimes any checkable fact at all.

None of these rules is aimed at a particular business. The app developer whose 27 first places started the cap is still listed on all of its pages. It lost its rank numbers because its only evidence was two reviews from webmail accounts, and it can earn them back the same way every other business can: one checkable fact on its own website, or a review from a client who will stand behind it.

Sources

TrustList Weekly

The week in software and IT, in one email

The news that matters to buyers, new rankings and our own research. Every Thursday, free, and easy to leave.

We will email you to confirm. Unsubscribe with one click in any issue. Privacy policy