Detailing the expectations of Early Access reviews, the Rating of Completion/Competence system, examples of on-going and completed Early Access games, and future policy with Early Access products.
Early Access/Games in Development is not a practice I have always believed to be successful until I had witnessed how it can be a resourceful means to make games often too niche or too risky greater than by traditional development. Many bad examples and news stories have been made about these models such as Towns, Paranautical Activity and hundreds of other Survival games that have crashed-and-burn as a result of overestimating their ambition, their funding or their goals. However, there have also been outstanding stories such as Divinity: Original Sin 2, Grim Dawn and other modern classics. Whether we want to accept that this practice devalues games as completed works or that its acceptance entices publishers to release games incomplete with "Games as a Service" models, Early Access is here to stay as a modern solution to game development. Like any other tool, the core issue isn't what kind of tool we are using but how we use it responsibly.
As George Weidman (a.k.a. Super Bunnyhop) describes it in his Ethics of Reviewing Early Access, “I don’t feel [Early Access] is being talked about to actually solve the problem. The problem is how we criticize Early Access games, and I’m not excluding myself to just talking about the Steam Early Access program […] The very concept of Early Access isn’t causing problems; there are lots of games that have done it right […,] but there are a lot more games that use this system […] to turn it into a problem.” George would later go on to illustrate that these games need higher standards and push journalists to give these games quantifiable scores, yet he sticks to traditional means of scoring them. The truth of the matter is they need new scoring systems. This is the impetus that made me want to create an objective, yet respectful, system to illustrate for the common consumer the quality/completeness of a project to give them information to decide whether a game warrants its support.
Specifically, this system is one I have created to be as much as objective for the consumer as it is for the developers because it is one based around measuring the expectations of the developer. It not only measures basic expectations like content promised but also considers the optimization, the quality of the gameplay as well as the developer's productivity—with respect to the team’s size. The rating itself is not meant to be a one-time grade; it's more so one of many litmus tests for consumers to judge the future of a project. Over-time, these data points should show positive growth as well as indicate that the developers will complete their game, and that will result in more people feeling comfortable to support these projects.
While I am under no delusion that my system will become popular enough to achieve that result, this system is the foundation I would aspire other reviewers to consider if they wish to resolve the dilemma with reviewing incomplete games.
Link to Super Bunny Hop's video: https://www.youtube.com/watch?v=2TeH8fXBkcM
The Rating of Completion/Competence
Now to the actual meat behind this post: The Early Access reviews. My reviews are broken up into two components, the detailed summary/critique of a game and the Rating of Completion/Competence. The first part is similar to any other review, though it will not be as long as my others due to character limits. The latter section is my solution to establishing quantifiable standards for Early Access games that gives consumers a litmus test on its current state.
The categories behind these ratings are simple: Content (What is promised/planned?), Quality (How playable is it to being polished as a full-release?), Optimization (How does it perform?), and Productivity (How efficient are the developer’s achieving their goal-posts or timeline?). In each numerical rating, it takes into consideration these four components that impact a simplified score (1-5) and the more detailed scores across all four categories (1-20). It’s always illustrated that the score itself is not one about the rating of the game but of its currently reviewed state from completion, and it is subject to change the closer the project gets to full-release.
Here is a detailed list of my scale:
1: Games that are bug-filled messes, have little to no content, perform terribly that it would be hazardous to play right now, or suffer from a severe lack of development with no change in sight.
2: Games with the beginning of ideas or a thought-out concept, yet they are too early in development to invest your time. These games also suffer from a lack of content, especially promised features that have not been implemented such as promised multiplayer modes. Furthermore, there may be technical problems that may cripple the experience, unpolished but serviceable gameplay, and they may require waiting for updates to see if the game will improve. These updates may be too infrequent to be an incentive to return to the game consistently.
3: Games that are in a playable state while their technical issues are more like nuisances than crippling the experience. In addition, these are games with all promised features accounted for while they may not be finalized. Content may be lacking or just enough to get your money’s worth right now, so these games are cautious recommendations. You can wait and see to learn if things improve, or you can rest assure updates will be frequent enough to keep you coming back to play.
4: Games considered playable for a full-release, technically competent with relatively few issues, and acceptable for their value proposition with its current (or future) price. Features originally promised are all accounted, and the game may have expanded these ideas, polished the gameplay some more or added more features without setting back the game from its full release. Updates and progress are frequent enough to encourage cautious buyers to jump into the game with little to no complaints.
5: Games that are, for all intents and purposes, complete. These games are within their final stage of development focusing on balancing the game, ironing out its remaining technical problems or assets, or preparing the final update before its official launch.
After playing and evaluating a game, I then write up two ratings. Simple ratings are more of a personal evaluation for the reviewer to consider everything together whereas detailed ratings are to put the reviewer on the spot. In addition, there should also follow some detailed information or examples as to why the game earns a specific rating. The point of asking the reviewer to prove to others with a qualitative metric is to force the reviewer to confront any biases or any potential problems that would be ignored by the game's incompleteness. Consumers can take those evaluations, not as fact, but as demonstrations of what the standards for Early Access games should be over time.
In the past, I have tried to follow a few projects to their completion, but if there is going to be a surplus of these reviews, then I will plan to update these reviews triannually or quarterly. This will hopefully show productive growth while also giving me ample material to evaluate the changes made over time. If this system seems confusing to you, then I would suggest reading a few examples I have provided down below to see the system in action.
Examples of On-Going and Completed Early Access
Sword of the Stars the Pit: Infinity (On-Going, Reviewed May 20, 2019)
Simple Rating: 3/5
"Games that are in a playable state while their technical issues are more like nuisances than crippling the experience. In addition, these are games with all promised features accounted for while they may not be finalized. Content may be lacking or just enough to get your money’s worth right now, so these games are cautious recommendations. You can wait and see to learn if things improve, or you can rest assure updates will be frequent enough to keep you coming back to play. "
Complex Rating: 14/20 (70% Complete)
Content: 3/5
As far as delivering what Kerberos Productions promised, a cooperative first-person take on the original Pit, the devs have already delivered on that goal. In addition, the Crafting system has most of the recipes that I can recall from the original game. It’s more difficult to compare how much is available if you don’t separate the items from all the expansions. There still is the omission of the final floor, bio-mods and other former mechanics listed on the Features List, but nothing that you would expect to be canned in the future (with the exception of the Safe Room.) The only concern I have with the roadmap is that it doesn’t state what content may be included from the Osmium Edition. As a result, my decision is to give this section a lower rating than it probably deserves.
Quality: 4/5
In this category, Kerberos nailed not only its presentation but specifically its sound-design, which this latter category shouldn’t be surprising since they have reused their sound assets. (Not to imply that they were lacking to begin with.) There is far more care and attention to detail put into the recreation of every model, and the added sense of scale is appreciated. Also, the interactable animations, which you can turn-off just like in Far Cry 5, are another little touch that goes a long way. The only blunders that can take you out of the experience include procedural glitches with the exits such as a hole in the ceiling sending you (down) to the next floor as well as when the AI goes braindead or they get stuck on the walls. Also, as one more gameplay-related problem, grenades need to be better designated on the UI because you cannot tell one type from another, which is only a problem in this game because you cannot pause the game in the middle of combat. Finally, despite the art-style being more washed out compared to the colorful 2D sprites, the charm is still there to match the dark comedy of the original.
Optimization: 3/5
Now I alluded to this section way back at the start, and there are a couple of things to say. First, the default settings are helpful enough FOR most users who may struggle with the shadows and the heavy fog effects. If you are still suffering from performance issues, then you can also apply some general Unreal Engine 4 tweaks such as resolution scaling currently not configurable within the in-game options menu. (You can visit the forum link down below for more details.) The real issue is how much the game will overheat most hardware if you don’t tweak these settings, which can turn some people away, but that seems to be a common Unreal Engine 4 problem. Finally, as I was unable to test the multiplayer servers, I cannot say anything about its net code or any possible issues.
Productivity: 4/5
Similar to nailing its presentation, Kerberos has also been transparent and responsive towards player feedback as well as providing multiple updates each month. From its Early Access release on October 26, 2018, Kerberos has released two major content patches, one for the crafting system available since November and the multiplayer mode since March of 2019. Throughout these months, minor improvements and quality-of-life fixes have been made, which the last one was posted May 1, 2019. All of this information leads me to trust that Pit: Infinity will be released within the Q2 and Q4 timeframe of this year, which the devs have stated upfront the full-release date and state of its features are based on the level of support through Early Access.
With all this information out of the way, thank you for watching and if you have any constructive criticism, questions or suggestions on how to improve my Early Access rating system, please comment down below. May you enjoy the rest of your day whether in or out of the Pit.
Grim Dawn
(Completed, Date: Dec 3, 2015)
Simple Rating: 5/5, which coincides with the developer's statement that 99.9% of the game is complete.
I'm willing to give this game a full raiting because the developers have earned my admiration for why EA can be a great idea if done by the right people.
Detailed Rating: 19/20 or 95%
Content: 5/5
The only things left for the game are some polish that will make a full-release build within the first week of Feb.
Quality: 5/5
It's a finished product
Optimization: 5/5
Besides my PC being too weak to run at the max settings, I have had zero problems.
Productivity: 4/5
Despite the limited development team of ten and of over two years working to make Grim Dawn a great game, Crate Entertainment has been able to add meaningful content or add useful monthly updates to make the wait acceptable.
They are the best example of EA I have been a part of.
Distance
(Eventually got a full release in 2018, Original Date: Dec 3, 2015)
Simple Rating: 3/5
Detailed Rating: 11/20 (55%). A cautious recommendation only for the gameplay.
Content: 3/5
Multiplayer, all game modes, the second type of car (which gives a double-jump at the cost of no flight mode) and the Steam Workshop level editor are in the game. However, what remains besides the story mode is vague.
Quality: 4/5
The core gameplay is refined enough that I would consider it in a complete state. The only thing that may hamper it are the optimization problems.
Optimization: 2/5
If you wish to get the best mileage with Distance, then you don't want other programs to be in the background.
Productivity: 2/5
While I appreciate the community involvement from the developers making fan-made contests and showcasing their creations, I would appreciate more if they added their own content more frequently. Or simply finish the rest of the game to be in a playable state to get more interest.
The updates and bug fixes are frequent enough to let me know they are still working on the game. But I think so many people on the fence with Distance would buy this game if all the features were complete. We're nearing the first year mark of Distance--hopefully, as the devs state, we'll get the full product by 2016.
You can only get so far with selling me with the car before I start to question whether we'll ever reach the end.
Serious Sam Classic: Revolution
(On-Going, No Sign of Change, Original Date: Dec 3, 2015)
Simple/Personal Rating: 2/5. (More personal than objective.)
To CroTeam's credit, they do tell people not to buy into SS:R right now unless you want to test out their servers or get early into the game. But I don't understand why they put this through EA to begin with if they had nothing new besides the mutliplayer.
Detailed Rating: 11/20 or 55% (Buy the HD version of The Second Encounter, which comes with The First Episode levels, to play with friends.)
Content: 3(?)/5
This is an odd thing to state because the game does and doens't have all its content. In one sense, it does have enough to warrant a purchase if you don't own the first two encounters, all of the First and Second encounter are present. And it does have a revamped multiplayer to satiate old-school players.
The final expansion is still not going to be released anytime soon, which besides the updated engine and the 64 player cooperative mode, was the only thing I was interested in playing.
Quality: 3/5
This game plays as good as the classic/HD games.
I marked this down several points not because of the low textures from the classic games which were improved with both HD games. The reason is because they didn't improve the technical faults of the original games.
Most notably, the sound limitations of the original game. Enemies' sound effects do not tell you where their location is, the one major flaw of the classic games when it came to hordes of Kleers and the suicide-bomb runners.
Optimization: 4/5
For single-player and local multiplayer purposes, the game runs just fine. I've read on the forums that servers can be a pain to get working properly and have issues for some people.
Productivity: 1/5
This product won't be finished anytime soon. CroTeam made this into a fan-project, which I am happy in one sense for them to support a classic series, yet I fail to see any reason to buy this product anytime soon unless they add in the new campaign. (I'll look forward to Serious Sam 4.)
Ty the Tasmanian Tiger
(Complete, Date: March 25th, 2016)
Simple Rating: 4/5
Detailed Rating: 10/15 (66%)
Content: 5/5
TY is a finished game being put through performance testing. No additional content or gameplay changes are planned.
Quality: 3/5
On a Steam controller or keyboard, the game plays just as finely tuned as on consoles.
However, and what I do not recall, there are audio sync issues as well as audio balance issues when it comes to cinematics. For example, character voices will be drowned out by environmental noise (like a waterfall).
There is another issue that will grate on some people's nerves: The camera will latch on to the environment, so be sure to rotate the camera without getting it stuck on trees.
Optimization: 2/5
Game cannot run a smooth performance with anything above Low shadows, no reflections and low draw-distance.
Productivity: (Too soon to judge)