Accurately Simulating Opening an Overwatch Loot Box

One way to try and understand loot boxes is to open many, many loot boxes and then do a statistical analysis on the items that come out. While it would be expensive to open many, many loot boxes, it is very cheap to simulate opening loot boxes. In the past, some people have written programs or used analytic methods to find out how many loot boxes it would take to get all items (1, 2, 3, 4). This is useful, but most casual gamers do not want to collect all of the items. For instance, some people have a preferred hero and would like to collect the cosmetics for that particular hero. Or some gamers like the best looking skins, and want to collect all the rarest items.

It would be useful to have a more general loot box opening simulator. In order to simulate a loot box, I need to know the probability of getting each of the possible drops. There are two ways to find the probability of each drop. The first would be to use what the developers say the drop rate is, and the second would be to open many loot boxes and count the different drop types.

Probability disclosure from Blizzard

Due to a law passed in China, Blizzard had to disclose their loot box drop probabilities. They released the following:

  • Each loot box will have at least one item of “rare” quality or better
  • On average, 1 in every 5.5 loot boxes has an epic item
  • On average, 1 in every 13.5 loot boxes has a legendary item

This information is useful, but leaves some questions unanswered. Do these chances bundle epic coin drops and epic item drops in the same group? What is the chance of getting credit drops? There are seven types of drops: common item, rare item, epic item, legendary item, rare credits, epic credits, and legendary credits. The occurrence of each of these is not clear based on only the disclosure of drop rates by Blizzard. This is insufficient information to simulate a loot box.

Instead of using these drop rates, the probability of each of the seven drops can be calculated by counting the items that come out of loot boxes. Many people have posted "loot box opening" videos, and robust statistics can be calculated from these.


Probability from opening many loot boxes

Some users on Reddit have posted statistics from opening a large number of loot boxes. One can calculate the drop rate of each of the seven possible drop types by simply counting how many of each quality are opened. For instance, a Reddit user named The_Tastiest_Tuna (TTT) posted this where they counted the drops for 1024 loot boxes here. These 1024 loot boxes came from 14 different streamers, each opening between 150 and 50 loot boxes.

I have written a simple script to analyze the drops in 669 loot boxes from this Reddit post where Twitch streamer marcothemarco (MTM) opened 669 normal loot boxes. The script can be found here to count the items. The probability from these sets are below:

DropProbability From TTTProbability From MTMWeighted AverageBlizzard probability
Common item58.66%59.04%58.81%-
Rare item27.41%26.57%27.08%-
Epic item4.67%4.90%4.76%4.55%
Legendary item1.89%2.09%1.97%1.92%
Rare credits4.28%4.30%4.29%-
Epic credits2.55%2.65%2.59%-
Legendary Credits0.54%0.45%0.50%-
Table 1 - The fraction of each item or credit type that were dropped from a large number of loot boxes. The weighted average column uses the number of loot boxes opened for weighting.

The drops from TTT are from many different players, and the drops from MTM are from a single player. These percentages match closely, and the probability used to simulate a single drop in Overwatch is the mean of these two weighted by number of loot boxes opened. To be clear, a loot box has four drops in it, and the drops can be either items or credits.

Simulating one drop at a time

Rather than simulating opening an entire loot box, the tool simulates one drop at a time. This is easier than trying to figure out exactly how the entire loot box system works. As long as you open a large number of loot boxes, the distribution of items will match the drop rates in Table 1. This tool does not simulate opening one loot box. It simulates opening many, many loot boxes. Any one loot box may look unrealistic (e.g. 4 commons) but the statistics that come out should be accurate.

Simulating drops one by one does not take into consideration that a loot box must have at least one rare or the possibility of a pity timer. This is because the percentages in Table 1 already went through whatever system Overwatch uses to give out drops. To implement a pity timer or the "at leasts one rare in a loot box" rule would double-count that rule. Essentially the percentages in Table 1 already take into account whatever rules that the loot boxes follow. This point is explained further here.

The main assumption here is that the actual item given is truly randomly selected. The items in a loot box aren't favored towards the heroes that a player plays often. There are some posts on the Overwatch forum to this end but none showed robust statistical evidence.

Running Simulations

The way the loot box simulation tool works is that the user chooses which items they have and which items they want from a list of items available. Once they do that, the program simulates someone opening loot boxes until they either have all the items they want or can purchase all the items they want with the credit they have. Running one simulation looks like this:

  1. User marks the items they have, items they want, and how many credits they have and then begins the simulation
  2. For one iteration, create a temporary list of items the user has, a temporary list of items the user wants, and a temporary credit total
  3. Choose a drop, based on the percentages in Table 1
  4. If the drop is credits, add the credits to their credit total
  5. If the drop is an item they do not have, add it to the list of items the user currently has
  6. If the drop is an item they have, add the appropriate amount of credits to their credit total for a duplicate of that rarity
  7. Check if the player has all of the items they want or if they have enough credits to buy the remaining items. If so, then the simulation ends, and if not, the simulation loops back to step 3.
  8. If max iterations or max time have not been reached go back to step 2
  9. Display the results.

The amount of loot boxes needed is recorded and the simulation begins again. This repeats until the max number of iterations is reached (default 1,500) or the time has exceeded (default 1 second).

Features

Filters: You can filter by item type, item quality, and event items. This makes it easier to select certain groups of items, such as all of the legendaries or all of the icons.

Simulation options: You can change the number of iterations, maximum time, maximum number of loot boxes to open for a single iteration, the number of histogram bins on the output plot, and a refund multiplier.

Spending strategy: By default, the tool assumes that the user will save their credits until they have enough to buy everything. This is the optimal spending strategy because buying an item increases the chances that you will get a duplicate.

Crash prevention: Since running excessively long Javascript code can crash most browsers, the computation is hard-coded to not last more than 20 seconds.

The tool is available here.



Previous page - Next page