S-Tier is the best. For dungeons, that means that dungeon is easier compared to others. For specs, that means that spec is stronger. S-Tier is better than A-Tier which is better than B, which is better than C, and so on.
lb_ci stands for the lower bound of a 95% confidence interval. Here's the site calculates it:
For specs: The site looks at the top 100 keys per dungeon per affix across all region and examine their raider.io scores. From these, for each dungeon, for each spec, the site takes the lower bound of a 95% confidence interval. The site then average these lower bounds, one per dungeon, to get an overall average lower bound for each spec, which the site then uses to create the tier list.
For dungeons: We take the top 100 keys per dungeon for each region for a given affix. For each dungeon, the site calculates the lower bound of a 95% confidence interval and uses that to create the tier list.
For affixes: We take the top 100 keys for each dungeon for each region per affix. For each affix set, the site calculates the lower bound of a 95% confidence interval and uses that to create the tier list.
This approach was inspired by pair of articles by Evan Miller:
It isn't perfect (no model is), but I've found it gives a good sense of what the meta looks like at the top end and generally "feels correct" to folks. Very open to suggestions for how to make it better, if you have ideas.
The site takes the lower bounds of the 95% confidence interval calculated as per the question above and then clusters them into 6 buckets using the Ckmeans algorithm.
Note that because we're using the lower bound of the 95% confidence interval of the raider.io score, Tanks, Healers, Melee, and Ranged are all on the same scale. They're just shown separatey for ease of comparison across roles.
The site didn't originally, but this resulted in specs that were used for a pull or two at the start of the dungeon showing up in S-Tier. For example, in Battle for Azeroth, mages would often play Frost for the first pull of Tol Dagor and then swap to Fire for the rest of the dungeon. But Blizzard's API only considered the spec for each class at the start of each dungeon, not the end, which resulted in Frost Mages, otherwise unplayed at the top end at the time, showing up as S-Tier. Taking the average across dungeons solved this issue.
No, not in the tier lists.
That said, untimed runs are included in the Top Builds section, since the Warcraft Logs API doesn't easily distinguish between the two.
No. Keys that are not completed do not show up in the API and thus the site doesn't have visibility into their existence.
Yes. Because we're only looking at the top 100 keys per dungeon per affix for each region (since that's what the raider.io provides), we are necessarily biased toward the top end. We also don't look at keys below +10, and only consider timed keys.
For a more comprehensive look at the entire set of keys, please check out benched.me.
For interesting statistics on timed vs. untimed keys, please check out bestkeystone.com.
The site looks at the top 100 Mythic logs for each spec for each boss. For tanks, melee, and ranged, it considers the top dps logs. For healers, it looks at top hps logs. I am aware that dps for tanks and hps for healers is non-ideal, but don't have anything better at this time. Open to ideas here.
For these top 100 Mythic logs, the site takes the lower bound of a 95% confidence interval again, and then, for the overall raid tier list, averages it across all bosses.
Melee and Ranged DPS are on the same tier list, though displayed separatey for ease of viewing. Healers are on their own tier list (since they use hps). Tanks are also on their own tier list (since tank dps is typically much lower).
Note that some specs may have very few or no Mythic logs on certain bosses. These specs effectively get a "0" for those bosses, potentially resulting in their overall raid tier list ranking dropping.
The site is also open source, so feel free to take a look at the source. Pull requests welcome.