Author Topic: Star Citizen Analytics Project  (Read 4723 times)

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Star Citizen Analytics Project
« on: January 16, 2017, 07:22:11 AM »
We have a Goon, boviscopophobic, who has done an amazing job these past months running analytics on Star Citizen using data mined from the game's public facing data (the same place where the funding numbers come from). If you see a chart in any of my blogs or missives (e.g. my latest), chances are, he did it. Below is his latest masterpiece.



JAN 2017 RSI DEMOGRAPHICS UPDATE

About 6 months and $25M have elapsed since the previous demographic snapshot of the RSI forum population. Updated funding-related statistics are summarized below. The methodology is mostly the same as in my previous post, so you can refer to that for details as well as an explanation of the meaning of each graph.

First off, the forum account vs RSI account rate discrepancy has stabilized at about 3.5, meaning that 3.5 RSI accounts are currently being created for every forum account. This ratio obviously spikes during free fly events.




For the funding-related graphs, the basic funding assumptions remain the same as last time, but there are three updates to the methodology. The first relates to closed accounts. Thanks to the wave of refunds post-Streetroller, I learned that is possible to determine with reasonable confidence if an RSI account has been closed, which generally indicates revocation due to refunds or possibly other misbehavior such as hacking. This latest set of summary graphs includes only accounts that were "alive" at the time of the snapshot. The previous set of graphs included a certain number of "dead" accounts, which affected the accuracy of the title counts.

The second change is that I've aggregated all titles not associated with a funding level into an "OTHER" title, except for a small set of user titles that I deemed to be CIG-related.  These titles, namely "Staff", "Developer", "Creator", "QA", and "Game Master", are assigned the aggregate title of "CIG".  Some other user titles that are arguably CIG-related, but which I did NOT include in the CIG set, are "Bug Moderator" and "Moderator". Note that some developer accounts may mark themselves as such as such only by their account name; these would not be included in the CIG count. 

Finally, to counteract title churn from people changing their title, I look at each account's titles over a number of forum snapshots and use the one that implies the highest funding level. Since user titles can be "understated" but not "overstated", so to speak, this should be a reasonable procedure if user funding levels are non-decreasing. Thankfully, since CIG almost never grants partial refunds, an assumption of non-decreasing funding levels should not introduce too much additional error. (Note: if no funding-related titles are available, then CIG-related titles are prioritized over "OTHER" titles.)








The contribution of concierge-level backers (High Admiral and up) has slightly increased -- 56.4% under the mid funding scenario, versus 53.6% last time. If we compare the graph of estimated average user spending by quarter of enlistment with the previous version, we get the following average per-user increases:



This indicates that accounts of all "ages" are continuing to put money into Star Citizen, possibly more so for pre-2014 accounts. However, note that an increase of $10-15 or so is a small fraction of the likely average transaction amount -- recall that this period includes Gamescom, Citizencon, the anniversary livestream, and the holiday sale, which featured pricy concept ships, cash-only sales, capital ship sales, etc. Depending on what you think the average transaction amount is (which I have not attempted to estimate), you could translate this into an estimate of the size of the current paying backer population.

Another longitudinal view of the backer population can be obtained by constructing contingency tables at various time snapshots. For example, the following is a comparison of highest user titles achieved through early August 2016, versus early January 2017. Because of how highest titles are computed, this table contains some unknown fraction of users "leveling up" through spending, and some users simply adjusting previously understated titles upward. I believe that the dominant contribution is leveling up, especially when looking at movement between the higher tiers, but I have not attempted to quantify this. 



Note that we have two new pseudo-titles: "DEAD", indicating that the account died off (refunds etc.) between Aug 2016 and Jan 2017, and "UNBORN", indicating that the account was made between Aug 2016 and Jan 2017. So for instance, we can see that of 92 completionists as of Aug 2016, 1 of those accounts got a refund. Of 193 wing commanders as of Aug 2016, 31 were promoted to completionist and 2 got a refund, etc. Notably, 22 CIG accounts "got a refund", which most likely means they left the company.

As a rough measure of the propensity of backer subpopulations to level up, we can construct a matrix of outflow percentages. In this table, the number in a particular row/column indicates the percentage of the population with that row's title that advanced to get the corresponding column's title.  So for instance, 16.06% of all Wing Commanders in August became Completionists by January. Similarly, 0.37% of Civilians became Freelancers/Colonels, etc. The hottest cells consist of concierge backers (High Admiral and up) moving up one or two levels, and CIG accounts moving to the exits.



If we are interested in inferring refunds specifically, then we need to look at pairs of snapshots that are closer together in time. Otherwise we can miss salient developments -- for instance, if a Civilian in August became a Wing Commander in November then got a refund in December, it would only show up as a Civilian refund in the above table. Using a set of several snapshots I derived the following counts for account deaths per highest title. I also noticed a large number of newly established Civilian accounts showing up as dead. To exclude possible low-effort banhammered trolls from the refund counts, I only counted Civilian accounts if they were confirmed as being alive for at least 45 days in at least one historical snapshot.

  • Completionist: 5
  • Wing Commander: 4
  • Space Marshal/Lieutenant Commander: 19
  • Grand Admiral: 27
  • High Admiral: 70
  • Vice Admiral: 47
  • Rear Admiral: 58
  • Freelancer/Colonel: 116
  • Bounty Hunter: 67
  • Mercenary: 61
  • Scout: 32
  • Civilian: 448
  • CIG: 26
  • OTHER: 74

Since this is a small and very much non-random sample, the likely accuracy of the funding scenario assumptions (already not that good) is probably far worse for refunded accounts. On the one hand, Civilians are assumed to have a low average contribution partly due to the proliferation of free accounts; however, a refunded account would obviously not be a free account. On the other, high-value accounts may not be refunded for anywhere near their nominal value, due to grey market transactions.

If we go ahead and apply the min/mid/max funding assumptions anyway, we get refund totals of $407,420, $674,587.50, and $941,755, respectively. For another estimate, also problematic, we can consider the self-reported refund amounts from /r/starcitizen_refunds. From reading through the posts that stated actual refund amounts, I arrived at an average per-user refund of $1366.10.  Applying this to the 1028 non-CIG refundees, we would get a total of $1,404,350.80. These estimates are of course only for the refunded forum population. The multiplier to get the total amount of refunds in the entire RSI population would likely be well less than 2.5, which is the ratio of all RSI accounts to all forum accounts.


CONCLUSIONS

All previous caveats about the accuracy of these estimates still apply. In addition, there are particular problems with trying to estimate refund amounts. Nevertheless, I think we can conclude that the refund outflows, while CIG certainly would find them annoying, are probably small enough in total that they can be easily compensated for with an extra concept sale (if we don't account for increases in engineering debt).

There are indications that funding is leaning even more heavily on concierge-level backers; this might be a good topic for follow-up analyses. Account age does not appear to play a large role in incremental spending. 

Previously I speculated about a soft per-user average spending ceiling around $200. This now seems to be more of an artifact of the bounded time window the backer populations have had to spend their money in. As that time window lengthens, fresh spending continues apace and it remains to be seen when there will be a large-scale change in backer purchasing behavior.
Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #1 on: January 16, 2017, 12:35:14 PM »
Quote
The longitudinal comparison showed an increment in average spending for accounts of all ages, so I now think the correct interpretation of that graph is a combination of a large initial outlay from the earliest purchasers (2013 and prior), combined with a steady increase in spending for accounts of all ages as more time/sales go by. So newer accounts have lower values mostly because they haven't had as much time to spend money.

As for new whales, the "UNBORN" line in a contingency table provides an imperfect estimate of new backer dollars:



So for instance, out of 103,589 accounts established between June 2016 and January 2017, 4 of them are now Wing Commanders, 19 are Space Marshals, etc. This works out to something like $1-5 million worth of funding from new accounts, depending on your assumptions. The new accounts are about 90% Civilians whereas the overall population is about 75% Civilians, so that may indicate something about the number of free accounts.

The question of how many whales there actually are, and just how much they are in for, is still not satisfactorily answered, but it was my main motivation for starting to look at this stuff in the first place.
Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #2 on: February 10, 2017, 02:32:07 PM »
New analytics, courtesy of our numbers Goon, boviscopophobic

Quote
Funding so far in 2017 is lackluster compared to the previous two years, but the comparison is arguably unfair because we have yet to see any major sales events this year.



Quote
However, new account signups are slowing even compared to early 2015, which may not be a good sign. The huge influx in 2016 was due to the announcement of the SQ42 package split.




Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #3 on: February 14, 2017, 06:31:19 PM »
Have some new Valentines Day sales metrics too. As I wrote in my missive earlier, once lies catch up, it's only downhill from here.






« Last Edit: February 15, 2017, 05:00:55 AM by dsmart »
Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #4 on: February 15, 2017, 04:39:18 AM »
Our numbers Goon, boviscopophobic, has his latest analysis from the funding chart numbers.



The Valentine's Day sale is pretty sad so far. However, I doubt CIG was counting on it to bring in all that much cash compared to a typical concept sale.





The total take so far is about $113K.

Thanks to the "stock limits" on some of the items (Aurora starter package, Mustang starter package, Starfarer Gemini, Starfarer Gemini two-pack), we can see how well individual SKUs are selling as well as get an upper bound on the amount of fresh cash spent on those items. The starter packages ($35 each) are warbond editions and thus cash-only; the Starfarer packages can be bought with store credit if I understand correctly. I only started checking stocks a while after the sale began, so the time axis in this plot does not start at 0.



The trends are fairly linear over the sampled period, with the Aurora selling at about 10.7 units/hour, Mustang at 13.4 units/hour, Starfarer Gemini at 2.9 units/hour, and the Starfarer Gemini two-pack at 0 units/hour. The enticing 8.8% discount on a two-pack (which I believe has limitations placed on it with regard to individual melting and such) hasn't brought backer wallets out, and so the number of two-packs sold has been firmly stuck at... 2.

The total amount of cash accounted for by these four SKUs is at most $29,575 if no store credit was used. Is it believable that roughly $83K, minus maybe $15K-ish of usual daily revenue, was spent on the unlimited SKUs? I leave that to you to decide.
« Last Edit: February 15, 2017, 04:59:28 AM by dsmart »
Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #5 on: February 23, 2017, 01:14:30 PM »
Night of Feb 21st, 2 backers gave CIG $30K by buying 2 completionist packs (don't contain all the ships btw).

I kid you not.

Let's wait and see if they later put in for refunds; as that may be a clear sign of money laundering.



And as someone pointed out, a 3rd of the income came from those two purchases



UPDATE: Apparently one of the Completionists is this dude
« Last Edit: February 23, 2017, 01:16:58 PM by dsmart »
Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline nightfire

  • Newbie
  • *
  • Posts: 34
Re: Star Citizen Analytics Project
« Reply #6 on: February 25, 2017, 01:35:18 PM »


Since the Hurricane went on sale, the funding ticker now clocks at over $144M.  :cripes:

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #7 on: February 26, 2017, 07:39:45 AM »
Latest Hurricane sales track compared to previous comparable JPEGS

Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline Darklegend1

  • Newbie
  • *
  • Posts: 38
Re: Star Citizen Analytics Project
« Reply #8 on: February 26, 2017, 10:54:08 AM »


Since the Hurricane went on sale, the funding ticker now clocks at over $144M.  :cripes:

This means either they are forging this data or or either the whales are even more blind than anticipated!! :laugh: :laugh:


Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #9 on: February 28, 2017, 12:52:37 PM »
Latest sales analytics





Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #10 on: February 28, 2017, 05:50:14 PM »
Absolutely no way in hell that 5 whales (or a combination of same) each dropped $15K on a Completionist package at this stage. They've got to be cooking the books. There's no other explanation. Either that, or someone is actively engaged in money laundering.

Quote
Five Completionist packs purchased in the past week, well done CIG! Game is good now, and there are 10 more Hartwell Music Sentinel 88G's in the 'Verse!

That's most likely more Completionist packs purchased in 7 days than were purchased in Feb-Dec 2016. (I don't have records for Jan 2016.)



« Last Edit: February 28, 2017, 05:52:48 PM by dsmart »
Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #11 on: March 10, 2017, 05:27:57 PM »
Our resident Goon data analyst, boviscopophobic, came up with some interesting (and startling) stats.

Quote
For comparison, I went back and looked at the stats for patch 2.6 as well. I am assuming patch 2.6 ran from 2016/12/23 23:04:58 through 2017/2/18 02:27:15, or about 1347 hours.

Star Marine:

122,711 users played a total of 115,781 hours across the two game types. However, there were likely many free fly players contributing low playtimes.
Half the total playtime was contributed by just 7178 players.
The top 170 players (0.14% of all SM players, or 0.01% of all citizens) contributed 10% of the total playtime.
The bottom half of the playerbase accounted for less than 10% of the total game time.
The average concurrent leaderboard-enabled player count was 86 over the course of the season.
The max time logged was 248.3 hours.
The playtime Gini coefficient was 0.685.

Arena Commander:

34,482 users played a total of 91,801 hours.
The mean playtime was 161 minutes while the third quartile was 146 minutes. I didn't bother to calculate any further but this also suggests a considerably top-heavy distribution.
The max time logged was 175.5 hours.

Murray Cup (racing):

10,089 users played a total of 3,285 hours, meaning that less than 2.5 people on average were racing at any given moment. One dedicated racer (pgabz?) spent 63.4 hours in this game mode.

Thus far since release, only 2039 people have played 2.6.1 Star Marine for more than 83 minutes, and together they contributed half the total playtime of SM



Nothing to worry about though, the other rumored 1+ million backers are just waiting for the game to finish.
« Last Edit: March 12, 2017, 06:30:24 AM by dsmart »
Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #12 on: March 28, 2017, 04:48:33 AM »




Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.

Offline Narrenbart

  • Jr. Member
  • **
  • Posts: 55
Re: Star Citizen Analytics Project
« Reply #13 on: March 28, 2017, 11:39:50 PM »
Today (29.Mar)  at 1am GMT another Completionist Pack has been sold :) (hourly tracker was over $16k)

Well I can understand this in kickstarter projects with nice perks but these are simple promises of ships from a company that keeps breaking their word.
The people who could afford them are basically the people who won't buy them - Wealthy people aren't wealthy because they spend thousands on dreams ...

My best guess is that a streamer or someone got it directly for free from CIG as a gratification - but this would lead to legal problems in many countries (decieving customers that there is cash flow)

Offline dsmart

  • Supreme Cmdr
  • Administrator
  • Hero Member
  • *****
  • Posts: 619
    • Smart Speak Blog
Re: Star Citizen Analytics Project
« Reply #14 on: March 29, 2017, 06:19:47 AM »
Today (29.Mar)  at 1am GMT another Completionist Pack has been sold :) (hourly tracker was over $16k)

Well I can understand this in kickstarter projects with nice perks but these are simple promises of ships from a company that keeps breaking their word.
The people who could afford them are basically the people who won't buy them - Wealthy people aren't wealthy because they spend thousands on dreams ...

My best guess is that a streamer or someone got it directly for free from CIG as a gratification - but this would lead to legal problems in many countries (decieving customers that there is cash flow)

That's the thing; we simply don't know. It makes absolutely no sense whatsoever that someone would - today - be buying $15K packs.
Game developers are just human beings who happen to make games for a living. If you want to hold us up to higher standards of conduct, then go ahead ...but don't be surprised if we don't uphold them.