Is number of Steam reviews a good measure for popularity?

Hi,

the question topic says it all. As Steam is very closed about game sales data, what is a good measure of how well a game is doing on Steam? It seems like the number of reviews (and the ratio of positive/negative reviews) is the only indicator of popularity available for us. Do you think it is a good one?

 

This may sound cynical, but I believe what the number of reviews shows you is the number of people who the game can evoke a moderately strong reaction from, for good or ill. People don't generally rate a game they don't care about. If you're, "Yeah, whatever," and keep playing or delete, you're probably not going to rate the game. If you feel strongly about whether it's good or bad, you're more likely to rate the game.

I figure it is the same psychology why forum users for various games and tools tend to either be very pro or very against. The "in the middle" types don't bother posting, except maybe for tech support.

There are exceptions, of course. Some people enjoy rating games. Some people feel like rating a game or product is providing a helpful service to others who may come after; kind of paying back for helpful reviews you've read. Some people like trash talking in ratings.

This doesn't mean I think total reviews is unrelated to total sales. The more eyeballs on the game, the more likely to evoke a strong enough reaction to illicit a review.

Just my two cents based on internet observation. I do not have a game on Steam, nor do I sell any products or services online.

 

I would say yes, its an indicator of how popular the game is, however, you need to look at how long the game has been on steam. It might have 100 reviews but if its in a course of 4-5 years. than its not that popular.

I have a game on steam that has a dozen reviews, and its been up since the end of January.

 

Number of reviews and amount of players playing (steamcharts.com)

 

I suppose it depends if they're good or bad reviews! Generally I would guess about 1 in 100 people add a review.

Talking of Steam. I hear on the grapevine that they will get rid of Greenlight and make it an open platform like Android or IOS. For example a game I put on there is apparently 75% to the top 100 even though it's only getting about 1 yes-vote a day (and 3 no votes). And in the last batch they greenlit 500 games. So it seems at the moment no matter how good, or bad your game is eventually everything will get greenlit. I guess they will make more money if from an open platform but might upset some Steam fan-boys who want Steam to be more exclusive.

I would like it to be an open platform then I could port all my mobile games to Steam. Even though this is exactly the kind of thing that some Steam fan-boys fear will happen. But I told them, "If you don't like the look of my games don't buy them! They're obviously not designed for you!" But it may mean the user-base of Steam becomes like more of the general public like people who play Candy Crush Saga. And Steam will no longer be an elite gaming platform.

On the down side it means that getting on Steam will no longer be a badge of quality and you will no longer be guaranteed untold riches.

But also I think it will encourage me to make more PC games because currently the only way to sell them is via your website and set up some kind of PayPal key system which is very cumbersome.

 

Better to be good or bad than have a non reaction.

 

Actually the current greenlight program is a good indicator if your product suits the market or not. When noone votes for your game the interest isn't really high enough. So building prototypes to show your idea is a good way to find out what the market currently wants/offers. Our studio actually drops project based on the greenlight feedback, there is no point in funding a game that the majority of the gamers don't have an interest in.

You can already see the negative impact of "greenlighting everything" in Steam's Early Access, most of the products there cannot last longer than a week in the Top 100 Charts and once out of the list sales do go downhill. This "spam" of crappy games also drives off gamers buying Early-Access games to a huge extend.

As for reviews it really doesn't matter if they are good or bad. You will find multiple games that sold more than 100.000 Units with only 50% positive reviews while you also find games with 95% positive reviews that sold less than 10.000 units.

 

As long as you can handle the heat when Jim Sterling roasts you for it

 

I disagree with this. If 10% of the population like your game that's a whole lot of people. It doesn't have to be the majority. I'm sure the majority of people hate Mr Little Pony, but as long as 10% love it it's still going to make a whole lotta money. Also niche games can be more successful.

 

Plus you only need to make like $101 to consider it a success, since that pays off the greenlight fee AND you made a dollar!

 

Steam turning into an open over-saturated market like android is horrible and I don't understand anyone that is happy about it.

 

Gaming has been a saturated market for a while. Steam greenlight high lights this more than anything, but now there's a place for the overwhelming supply of titles to appear. And if they suck and sell nothing, well, steam still makes $100 per POS just for being there

 

I doubt anyone here cares how much $ valve makes...

The $100 entry fee is a good point though. Should they fully open the market, this would be a good idea to keep the worst trash out at least.

 

Thought I would mention that the $100 submission fee is donated to charity according to Valve's website. Unfortunately it doesn't seem to filter out all of the subpar games that are showing up.

http://store.steampowered.com/app/219820/

 

Maybe people are just making crappy games for charity? haha

What wouldn't you do for lord GabeN?

 

You need to remember that people won't generally look at games that are outside their interests. I would not look at a Mr Little Pony game, so I would not have a chance to vote yes or no on such a game. If somebody votes yes or no to a game on Greenlight, it is because the game interested them enough for them to read about it and then react by voting. They thought the game was something they might be interested in. So if most people who took the time to vote on a game decided to vote no, then you have a problem. For example, if only 10% of the people who voted decided to vote yes, then that is not the same thing as 10% of the general population voting yes or even the same thing as 10% of all Steam gamers voting yes.

 

The point is that 10% of the population don't even vote. The people who vote on Greenlight are maybe just 1% of the Steam Community. By experience i can tell you that if your game doesn't jump into the Top50 within a week you can dump the project and start-over. Yeah, you might get greenlit anyway after a few months - so if it takes you 2 months to get 10.000 votes you can be sure that it will take you 2 years to get 10.000 sales - and only if you keep development up on this title.

If less than 10.000 people vote for your game in the first week of greenlight it will sell less than 1000 times in the first week of release. Take your greenlight votes, devide them by 10 and you know your sales when you put it up for sale.

 

$100 goes to charity. At least developers would feel good about it/

 

@elmar1028 that's kind of the conclusion I was leaning towards.

GabeN is a good fellow.

They have a better turn out than 'Murica, wooo!

Please don't ban me!

 

Interesting. Nice to have some numbers.

What about overall sales, say in the first year?

 

Can anyone back things up with numbers?

 

Honestly, I would prefer it if steam required $100 per game put up for Greenlight not for an account that's good for any number of submissions.

Chances are, if Steam had the bar set as high as it was 3 years ago - my game would never ever make it on Steam. But to be honest, I would rather have the bar much higher than it is now, even if that means my game wouldn't make it.

 

@frosted I agree with the $100 per title. It isn't a lot of money if you're doing well, and flooding the market with crap definitely means you won't do well, thus there will be less crap haha.

I was also under the impression that unity 5 being free meant more crappy games, but the misunderstanding came from plug-ins being needed for steam integration. Games can still be on the store without that, right? I had thought that there was previously a barrier to entry of $1600 (greenlight fee + unity pro).

Or is that necessary? Misunderstanding-ception!

 

General standard is between 0.2% and 2% will rate a game, with a reasonable expectation for a good game being 1 in 100. The 1% is a safe enough gauge that I multiply the number of reviews x100 to estimated how popular most titles are. Some of my products get more, some less.

PS - Minecraft on iOS has 349,667 reviews. Five nights at freddy's has 5,867. Silly Sausage in Meat Land has 280.
PSS - Across my 9 products, ~200,000 downloads and about 2,000 reviews.

Gigi

 

Is 25% likes good or bad then?

 

When examining Steam reviews i typically write off anything under 80% positive.

 

This means 75% of reviews didn't like the game. Not promising.

 

I'm assuming you're talking about a metacritic review or the actual game reviews. I don't think Greenlight numbers are ever made public.

When dealing with Greenlight, apparently to be at the top of the pile the approval ratings you're looking for hover between 30-40% according to most shots of the stats screen I've seen in the last year or so.

 

Doesn't anyone go for old fashioned LPs anymore to determine if a game is good or not?

 

OP did not elude to Greenlight. I'm referring to regular reviews.

 

But if 100% positive earned you a million ponds. 25% positive would earn you 250,000 pounds. Sounds promising to me! $_$

Greenlight is about if you would buy the game or not. I would not buy 99% of games out there doesn't mean they're not good. Just not to my taste. For example I would never buy a sports game.

On the other hand reviews are different since they are rating your actual game.

 

That's a nice way to look at it, except you did earn the million ponds since the review came from people who purchased and then disliked the product xD

 

yeah its probably a really loose way of figuring out how well a game did.

Binding of issac sold millions of units -30k reviews
http://store.steampowered.com/app/113200/?snr=1_7_15__13

So I guess Id say 10k+ stars did great, 1k to 9k stars good, 999 to 100 stars bad, < 100 stars crying home to moma.

 

900 stars is bad? 30k reviews, with millions of units, is in the 1% benchmark range. So, 900 ratings might mean 90,000 purchases. If the game is $10, that's $900,000 (minus 30%), which is more than enough to keep a small studio running. It's all relative.

Statistically, it's easier to get people to part with money than leave a review. /boggle

Gigi

 

Keep in mind it is unlikely somebody will vote either way on a game in a genre they are not interested in. If you don't like sports games, then you probably won't read through the Greenlight sports games and vote on them. I always assume somebody would need to be interested in a game before voting yes or no.

If 75% of people who were interested enough to vote on your game decide to vote no, then I think you have a problem. I am assuming that the 75% voting no are interested enough in the genre your game is in. If they had no interested in the genre, I doubt they would have voted at all on the game. That is how I look at those stats.

In your example of 100% positive leading to a million pounds of revenue, I would assume 25% positive to lead to about 100 pounds of revenue, not 250,000 pounds of revenue. Would you buy a game that only had 25% positive reviews? I would not. If the reviews were only 25% positive, that would likely kill sales.

 

Well I just threw out some numbers to give a general star rating, probably many of those sales come from steam sales (so they wont get the full price for it).

If you look at dustforce most of it was at the inital launch -- then the spike comes from the sale spike (more reviews but less money). Now probably alot of those sales come from the humble bundle (where they got the keys for next to nothing).



Okay lets get a bit more data. Hotline Miami sold 300k units now thats across other platforms probably not all on steam. It has 20k reviews.

http://store.steampowered.com/app/219150/?snr=1_7_7_151_150_1

 

Hmmm, I don't think that's quite right. Mostly you don't choose which games to vote for on Greenlight they just randomly come up and you choose "I would buy this game" or "I wouldn't buy this game". The games in the top 50 have average 30% yes votes. Also, nobody gets to see these votes except you.

I agree though 75% bad reviews would be bad but that's something different.