Mafia QC v3

Discussion in 'Review' started by Fuzzysqurl, Sep 30, 2014.

Thread Status:
Not open for further replies.
  1. Fuzzysqurl

    Fuzzysqurl baa baa mareep I do what I want Server Owner Developer I do what I want Server Owner Developer

    Joined:
    Sep 12, 2012
    Messages:
    2,096
    Likes Received:
    967
    Or somewhere close. Sorry for the massive delay, we were working as a team to try to get things ironed out. The previous reform was pretty much a solo work so there were many things that ended up not working. If you've been with involved with Mafia for the better part of this year, you might find certain sections that practically describe an exact scenario that happened. This is especially true in the adoption section.

    This thread will also be used to document any major changes that occur (because that worked so well for the last thread. Wait a second...).

    If you would like to discuss the QC system, please post in this thread

    §1 Submission
    §1.1- Pre-Approval
    A brief pre-approval process will be required for all themes submitted into review. Themes will be posted in the section titled "Proposals." Only you and the Lead QC will be able to see your thread.

    During the pre-approval process, your theme will be verified that it is error free, it fits the concept of Mafia, and the flavor is usable in Mafia. Themes will also be checked to see that they can accommodate a certain number of players (40) or, if its a non-peak theme that it doesn't support more than 12 players.

    §1.2- Submission
    The following submission guidelines will help streamline the process in order to review your theme quickly and efficiently. The guidelines are just that, guidelines. You don't have to follow them exactly, but you should include all relevant information with as little extra clutter/information as possible. You want the reviewers to focus on the main part of the theme, not everything else.

    Submissions should ideally include the following 6 things:
    1. Link to the coding- Any site with a "raw" capability is allowed. Popular sites used are Pastebin, Dropbox, or Github.
    2. Basic overview of the theme- Appeal to the review criteria for maximum effect. You should state what is important and unique about the theme and any creative twists on the theme
    3. Role Listing- Breakdown each role by what side they are on and tell how they interact with one another. Are there any "counter" roles or ideal/intended strategies between roles? How do the roles affect the flow of information or amount of deaths? (Should be handwritten, list the major points for each role). Please avoid adding images here. We can all use google to find pictures if we wanted and this is what "extra clutter" refers to.
    4. Spawn Listing- Are there any points in time you are concerned about balance that should be focused on? Try to break down certain points in the theme to make sure there is an acceptable distribution of roles for each team and that there won't be too many actions used each phase. (Can copy from RiceKirby's Theme Checker)
    5. Priority Listing- Just to verify things are in the right order (Can copy from RiceKirby's Theme Checker)
    6. Concerns or Ideas for expansion

    Use spoiler tags as needed, especially on longer lists. You don't need to use a spoiler tag for a short blurb of text and you especially don't need it for your theme link.

    Please note that constantly badgering the QC, or even other users, to look at or review your theme can lead to the premature closing of your thread. Reviewing is not a full time job and all Reviewers have other responsibilities and obligations. Bothering them only wastes their time and can even make them less likely to look at your thread!

    §2 Reviewing
    §2.1- Graded Criteria
    Themes will be reviewed according to various aspects, instead of a simple approve or decline system. A theme must achieve a certain number of points, out of a maximum of 100 (25 per reviewer) in order to be accepted, based off a number of criteria. The first 4 valid reviews in a thread will be considered in determining approval state.

    A reviewer may reward each of the following criterion a point value from 0 (worst) to 5 (best), for a maximum total of 25 points per review. A good review will post comments about the criteria and then assign a point value, rather than selecting a value then attempting to justify it.

    In order for your theme to be approved you must score at least 75 out of 100 points in the review process.
    **Please note, the final grading criteria may shift frequently. Each thread will be handled on a case by case basis by a Lead QC or sMA. Themes posted after a change will never be eligible for a previous grading scale.

    Creativity-
    How well the theme role matches its derived source (adapted themes) or creates a unique lore (original themes).
    How creative the game messages, roles and sides names and overall flavour is.

    Cohesion-
    Overall consistency of elements throughout the theme;
    Avoiding conflicting or contradicting statements (2 Roles saying they are the Strongest) or concepts (Robot in Middle Ages) in the theme;
    How well the elements in the theme mesh together to create a strong bond rather than scattered and random characters;

    Craftsmanship (Balance, Tactics)-
    How fair the game is for all sides that spawn in any game size.
    Which advantages each side has, and how other sides can fight them.
    How much the theme allows players to change the game using their own skills, tactics or mind games.

    Course (Learning Curve, Flow)-
    How easy is to learn the theme's roles, sides and rules specific to theme.
    Overall pace of the game (deaths, information, plot twists, etc)
    How easy is to follow what's happening during a game.

    Concept-
    Key roles and rules for the theme
    What makes the theme fun to play

    Guidelines for grading criteria:
    Code (text):
    1.  
    2. o: Theme goes against what's expected from this criterion.
    3. 1: Theme doesn't feature what's expected from this criterion.
    4. 2: Theme has some problems regarding this criterion.
    5. 3: Theme has no problems regarding this criterion, but it doesn't feature any good points either.
    6. 4: Theme does a good job on this criterion, but a few improvements may still be necessary.
    7. 5: Theme does an excellent job on this criterion, with no issues at all.
    8.  
    §2.3- Excluded Criteria
    A review should not base anything off of the complexity of the code, fixing issues, or the author's history. Suggestions on easier or cleaner ways to code things are encouraged. Fixing errors/issues and ensuring accurate information delivery are obligations of owning a theme.

    §2.4- Technical Approval
    In order to be officially approved or tested on the server, all themes must pass a "Technical Review." During the review process, certain issues should be looked upon in order to create a better end result. Grammar, spelling, informative messages, coding matches what is described, and "clean" /roles, /sides, /priority (essentially, remove duplicates/useless information) are the key aspects of this Technical Review. This "Approval" can be posted at any time by any member of the QC staff. There will be a checklist of things to verify. If the theme meets all the criteria, it can be awarded a "Technical Approval." If a theme fails to meet the criteria, the author just has to fix everything if they want their theme to be officially approved.

    §2.5- Tests
    If a theme fails to achieve the required number of points for approval but earns at least 60 points, the theme will be granted a testing period of 2 weeks. After the testing period is over, new reviews will be made for the theme. New reviews should be done by the QC who originally reviewed the theme. In the case that one of those QC becomes inactive or is fired during the testing period, another QC can review the theme in their place. If a theme still does not achieve the necessary points for approval, the theme will be considered declined.

    §2.6- Events
    If at any point an official event is hosted, a theme cannot automatically pass review if it is declared the winner. All participating themes, including the winner, can receive up to a maximum of 15 points towards approval at the discretion of the event host(s).

    §2.7- Assignments
    Lead QC can restrict QC members to review themes based on assignment if they feel it is necessary. This is to prevent any QC member from posting reviews in many themes and not letting other QC have a chance to make official reviews. If you do not have any assignments or all of your assigned themes' reviews are finished, you can review whichever themes you like.

    §3 Removal
    §3.1- Activity
    At the end of each month, play data will be reviewed. If a theme fails to achieve the lesser of 0.2% of the total games played or 10 games, the theme is removed. A maximum of 3 themes may be removed per month for lack of activity. If a non-peak theme fails to account for at least 5% of the total non-peak games played for 2 consecutive months, the theme is removed. Only 1 non-peak theme may be removed in a 2 month period.

    The threshold for activity removal can be changed at any time but requires notification of such changes. The change will take effect the following period, unless the next period starts in less than 1 week time, at which point it will take effect the period after next.

    §3.2- Quality
    At any point in time, a QC member can petition a re-review on any active theme. At this point, a post will be made and the petitioning member will state their case by re-reviewing the theme. If at any time, more than 3 QC assent to this notion (by also re-reviewing the theme), the theme will be removed. At least 1 of 3 assenting QC must be a Lead QC. All removed themes can be resubmitted at a later date for review provided issues brought up during the removal process are addressed in some way. If no other QC assent to the notion within 14 days, the case is dismissed and a new petition cannot be started by the same reviewer for 2 months.

    §3.3- Quality Assurance
    Every month a small group of themes will undergo a quality assurance process. During this process, Reviewers and the current Theme Owner(s) will discuss various aspects of the theme to assure the theme is up to par with current standards and give feedback on ways to improve the theme. While in discussion, the theme will stay enabled on the server to help provide live feedback to various changes. The theme will also be immune from removal for Activity or Quality while undergoing this process. There is no designated selection process or limit but generally the same theme will not undergo this process more than twice a year.

    §3.4- Post-removal
    If a theme was removed due to inactivity, it can be immediately resubmitted for review without any changes required. If a theme was removed due to quality, the theme must address a majority of the issues brought up before being resubmitted for review.

    §3.5- Limitations
    No theme can be removed or petitioned within the 2 months of it passing review, unless the author changed something that raises a concern. A reasonable argument must be provided to even be considered and would require Lead QC to be involved. Super Mafia Admins reserve the right to remove themes at any point that partially fit any above criteria or are negatively impacting the channel.
    If at any point the total amount of themes would drop below 30, themes set to be removed for inactivity are bypassed. Quality issues may remove up to an additional 5 themes past this limit. At no point will there ever be less than 25 themes on the server.

    §4 Adoption
    §4.1- Adoption Process
    Theme adoptions will be performed exclusively, and publicly, on the forums. If at any point you are told you can adopt a theme through any other medium, the previous statement will supersede the decision and halting the adoption process unless the appropriate channels are utilized. This will prevent any questions or issues on who owns a theme as it will be publicly visible for all to view.

    §4.2- Transferring Ownership (Closed Adoption)
    At any point, a theme owner can place their theme up for a closed adoption. In this process, a announcement will be made that the theme is currently for adoption. Interested parties will contact the current owner. The current theme owner will then choose who they want as the new owner. At any point following the announcement, the theme owner can select any interested party to assume control over the theme. There is no requirement for the theme owner to divulge reasons why that person is selected.

    §4.3- Retirement
    If a theme author would no longer like their theme on the server, the theme can be retired and barred from future adoption. Retirement must be made in good faith for it to be honored. Any retirements based on malicious reasons will not be honored and the theme will be relinquished to the QC as a whole to deal with accordingly.

    §4.4- Standard Adoption (Open Adoption)
    A theme owner may also choose to place their theme up for an open adoption. In this process, a announcement will be made that the theme is currently for adoption. Interested parties will have to present a roadmap of planned changes and come up with a justification on why they deserve to be the new owner. After an designated, but reasonable time, all interested parties will be judged by the Reviewers. An official announcement with the new owner of the theme and reasoning why they were selected will be posted. In the event only one person is interested in adopting the theme, they will still need to be approved before becoming the new owner. In an open adoption, the original theme author will have no say in who assumes control.

    If a theme is removed from the server and the author is not active, the theme will be eligible for standard adoption as well.

    §4.5- Assumed Control Adoption
    The QC staff as a whole can assume control over any theme currently up for open adoption or any theme being retired for malicious intents. This can be done to preserve the integrity of staple themes, or prevent drama over a new owner.

    §4.6- Adoption Trial Period
    During the first 3 months after adoption, the theme will be monitored by QC to ensure the author is handling the theme properly. Ownership can be revoked at any time during this period if the theme's quality, direction, or handling does not meet a certain standard. The revocation of ownership will be accompanied by an official reasoning stating the case. If ownership is transferred in a closed adoption, the original theme owner is the one that revokes ownership. In an open adoption, the QC staff will be the ones that revoke ownership.

    After 3 months, the theme is considered fully adopted and will be treated like the theme was initially made by the adopter.

    §4.7- Limitations
    If a theme undergoes a large amount of changes or the structure of the theme is changed, the theme will have to be reviewed again before it is considered approved again.
    No person may own more than 2 themes in an adoption trial period.

    §5 Staff
    §5.1- Hiring
    A user is eligible to become a Reviewer if they are in good standing within the community and are capable of reviewing themes accurately and to a desired standard. There will be no Trainee or Trial roles. If you are interested in becoming QC, don't ask for it. Instead, start "unofficially" reviewing themes to prove you are capable of doing the work. You will need more than one review to show you are consistent in your review quality, but there is no official number. When you've proven yourself, you will be contacted.

    §5.2- Firing
    At any point in a time, a Reviewer can be fired for reasons including, but not limited to: inactivity, abuse of position, poor behavior with other members, continual poor performance, or extended restrictions in the channel. If at any point a Reviewer is fired, it will be considered a permanent decision barring special circumstances recognized by the QC Leaders.

    §5.3- Turnover and Life of Review
    A Reviewer's tenure ending or beginning in the middle of the review process is not an uncommon phenomenon. If a Reviewer is hired, they reserve the right to make their "unofficial" review official at any point until the theme has finished the review process. The newly appointed QC can also elect to create a new review to count instead. If a QC is fired, all reviews for themes currently in review are immediately voided and will not count for or against a theme. If a QC leaves on their own accord, their reviews will remain valid but they reserve the right to void any review of theirs they wish within 24 hours of departure. If a QC leaves but later is determined they were on the track to be fired, the validity of the reviews is up to the Super Mafia Admins and can be voided on a case by case basis.

    §5.4- Staff Duties
    • Mafia QC will always be led by the current Super Mafia Admins. At any point in time, they can add to, remove, or modify any part of the review process with or without notification. They can also perform any role designated in the process and are the only people that can decide if a member should be hired or fired.
    • Lead QC are the next level of staff. These members are able to move themes from Proposal to Review and from Review to Active themes. They are responsible for keeping order during discussions and make sure Reviewers are properly performing their duties. Lead QC can also void reviews if valid reasoning exists.
    • Reviewers are expected to maintain a minimum level of activity in order to allow the process to flow properly. Reviews are expected to maintain a certain level of content and posts are expected to maintain a professional tone.
    §5.5- Current Staff
    Super Mafia Admins
    • TraceofLife
    • Whereabouts Unknown
    Lead QC (Max of 3)
    • EljayQuiz
    • Shazan
    • Whereabouts Unknown
    Reviewer (No Max)
    • Fiery Espeon
    • Joeypals
    • Windblown
    • Perspective
     
    Last edited by a moderator: Jun 22, 2015
  2. Whereabouts Unknown

    Whereabouts Unknown Absolutely Stunning Forum Moderator Channel Leader Forum Moderator Channel Leader

    Joined:
    Aug 17, 2010
    Messages:
    1,008
    Likes Received:
    538
    PO Trainer Name:
    Miki Sayaka
    Due to low activity among the current QC and a stack of themes in Review that have not been cycled through yet, we're going to put Review on rush mode.

    This means:
    • The number of Reviewers needed per theme currently in Review has been reduced from 4 to 3.
    • The minimum points required to pass Review is now 57.
    • The minimum number of points required to gain a Test is now 45.
    Until otherwise stated, this will not be effective for any new themes submitted to Review. They will require 4 Reviews and follow the same rules outlined in the Opening Post.
     
    Pangaea likes this.
Thread Status:
Not open for further replies.