Exchange Ratings…It’s a Number

Coingecko and Kaiko (plus others) have developed exchange ratings, originally branded as Trust Scores, to help investors choose where to trade. But, without doubting their good intent, these scores are essentially made up. So it’s not clear what they actually measure and, by extension, how they should be used by investors. We argue that they measure different things and that neither of them is default risk, a crucial investor concern.

‍

‍

Kaiko and Coingecko’s ratings are intended to rank exchanges from most to least investor friendly. Accordingly, both scores include measures of desirable exchange features such as liquidity, regulatory compliance, and usability. But the R-Sq for the subset of exchanges that have both ratings is just 28%. Either 72% of these ratings are noise or they simply aren’t measuring the same thing, whatever the stated original intent.

Beyond the above features, both ratings do indeed diverge a little. Coingecko includes “scale” and Kaiko adds “maturity”. The chart quantifies this divergence by showing the standardised betas* of the scores against the same input variables. If you increase your number of employees from the 50th to 70th percentile of available exchanges, your Coingecko and Kaiko scores go up +1.34 and +0.90 respectively.

The chart shows how Coingecko puts more emphasis on size, whether that’s measured using employees, Twitter followers or reported daily volumes. Both scores take a similar position on exchange domicile and AML risk. Kaiko gives higher ratings to older, more proven, exchanges, a feature entirely missing from Coingecko’s analysis. So these two ranking systems are at odds. Which one should investors use?

We would argue neither because the entire endeavour necessarily inheres trade-offs that individual investors need to make for themselves. An investor’s exchange choice should be driven by a unique set of considerations such as what markets they want to access, idiosyncratic UX preferences, price sensitivity, and so forth. The idea that this can be summarised into a single number is flawed.

However, there’s one consideration that investors should include in their decision-making that isn’t currently available. As investors have discovered, exchanges can become insolvent, freezing accounts and losing client money. There is currently no system for measuring this risk. It isn’t Kaiko or Coingecko’s rating because these include non-credit variables (e.g. market depth) and are missing others (e.g. exchange leverage). Crypto-markets need this risk rating. There’s a reason why TradFi has produced Moody’s and S&P.

_{* This is the beta of a univariate regression using the given independent variable multiplied by the standard deviation of that varaible. This shows how much an exchange score changes when you shift it by one standard deviation in that variable. The unfilled bars are not statistically significant. For the purposes of this analysis the Kaiko score, which is out of 100, has been divided by 10 so it matches the Coingecko score, which is out of 10.}