Too much themes not getting reviewed enough by QC members.

Discussion in 'Mafia' started by Shazan, Sep 24, 2013.

Moderators: Pamitha, Water
  1. Shazan

    Shazan Obey to Kyubey

    Joined:
    Jan 21, 2013
    Messages:
    591
    Likes Received:
    14
    PO Trainer Name:
    Obey to Kyubey
    Ok, i think that we all know that from this summer that a great amount of QC members gived their dimissions the new themes reviews is getting really slow, with themes in test for more months and even some themes waiting for a test vote from a long time.
    One of the worst thing that is happening is that mafia players are distrusting us, meanwhile saying that the only themes getting reviwed in time are that owned by MAs and QCs members selves when it isn't even true, you can simply look at DBZ and Lord of The Rings themes to understand that they too suffer from this situation.
    The point is that maybe we need only some organization, like maybe a post where QC members claim what themes they will give priority this week and where Theme coders can request a review for their themes because they are waiting it for a long time.
    Obviously it wouldn't be obligatory that QC members post their intentions every week because they too have obligations in real world so maybe some week they have to give their time to those activities in real world. but at least this way if a QC member post that they will take a commitment for starting, testing and reviewing a particulare theme: another Quality Controller could decide to give priority to that theme too so that at least such theme review situation will be solved.
    On this purpose here my intentions for this week.

    Theme to review, test and approve/reject: Creatures by Karp Karpity ( It waited too long):
    Theme to decide to give a test or not: Lord of the Rings by Windblown and JhenMohran.

    Let me know if you think if this is a good idea or not.
     
    Last edited: Sep 24, 2013
  2. The Hades

    The Hades My Wall

    Joined:
    Mar 26, 2013
    Messages:
    142
    Likes Received:
    5
    PO Trainer Name:
    zimmy
    This sounds like a great idea, I would love to see more themes get loved
     
  3. Angel Flonne

    Angel Flonne macaron

    Joined:
    Jan 24, 2013
    Messages:
    509
    Likes Received:
    3
    PO Trainer Name:
    Angel Flonne
    This may make me sound like I'm really desperate, but whatever
    I've PM'd Roild several times on the server to do something about Toys but to no avail. I think it's one of the themes that's been in Review for a bit too long. I'll quote my post from there
    I agree with you, many of the themes [see review page 3 onwards] are left hanging and as a result, locked. There really isn't a sense of closure at all if it is just not going to be completed. Seeing as there is a limited number of QC, it is definitely is difficult to review all the stuffs, so good work anyway! :3
     
  4. Epikhairz

    Epikhairz Delta Stream

    Joined:
    Mar 15, 2013
    Messages:
    929
    Likes Received:
    102
    PO Trainer Name:
    Epikhairz
    yeah something should be done to speed up the process.

    juggernaut was hung in review for about 4 months (since April iirc) with 2 total approves before it was eventually rejected a week ago or so
     
  5. IceKirby

    IceKirby A.K.A. RiceKirby

    Joined:
    Apr 26, 2011
    Messages:
    2,176
    Likes Received:
    635
    PO Trainer Name:
    RiceKirby
    I feel that the Testing process is slightly hurting the review process. We often see a theme being put to test and then getting forgotten for a long while, or even QC members giving a Test[1] because it's easier than actually reviewing the theme. IMO, Tests should be reserved for themes that really need it due to some different game mechanic, not as a way to gauge balance for absolutely all themes.
     
  6. Hey Day

    Hey Day Argumentum ad ignorantium

    Joined:
    Jul 13, 2012
    Messages:
    138
    Likes Received:
    0
    I've always been in favor of a "QC X claims Theme Y" system where you need a flat 3 Approves rather than a majority and 3 QC review a particular theme sequentially. It would be very organized, increase speed, and allow maybe even for 1 on 1 help sessions between QC and theme author as necessary. However, before a theme reaches this stage, it would have to pass an initial check (i.e. JSON compatibility & theme checker).

    Majority leads to issues like something being too easy to approve, something being too hard to approve, themes being approved when QC resign because the threshold lowered, things like that. The sort of helter-skelter system we have now also makes it fairly easy to be lazy (I confess to being one of those lazy QC) or neglect a theme altogether.

    I'd also like a (possibly separate) grammar review where all the grammar in the theme is fixed (no, I don't think themes should be approved/rejected based on how good the grammar is).

    Of course, these are just my musings, and I'm not sure everyone would agree that these are good ideas.
     
  7. JhenMohran

    JhenMohran Random Art Guy

    Joined:
    Jun 2, 2013
    Messages:
    181
    Likes Received:
    0
    PO Trainer Name:
    JhenMohran
    My musings with HD:

    (20:37:52) JhenMohran: so each QC member has a max of 3 themes to review at once
    (20:37:52) JhenMohran: these are randomised to them
    (20:37:52) JhenMohran: so Stocke could end up with reviewing Lotr, Mafia Players and Creatures until
    (20:37:52) JhenMohran: a [accepted] or [decline] is reached
    (20:37:52) JhenMohran: But then Jhen could be looking at Toys, DBZ and FMA until likewise
    (20:37:52) JhenMohran: Once a [acc] or [dec] is reached then that QC member has a spot "open" to QC other themes
    (20:37:52) JhenMohran: if there are themes waiting to have QC people attached then those with spare spots are randomly allocated
    (20:37:52) JhenMohran: A theme cannot enter Mafia Review unless there are suffieciently enough spots of QC to look at it
    (20:37:53) JhenMohran: GG
    (20:37:54) JhenMohran: solved
    (20:37:58) Human Destroyer: mute jhen for spam tho
    (20:38:04) Human Destroyer: nah I'm reading it
    (20:38:23) Human Destroyer: I kind of like that idea except I think it should be more voluntary
    (20:38:35) JhenMohran: yeah, you choose what spots you have
    (20:38:41) JhenMohran: so you could only have 1 spot for instance
    (20:38:46) JhenMohran: if v busy
    (20:38:50) Human Destroyer: (15:37:55) JhenMohran: these are randomised to them
    (20:38:55) Human Destroyer: =/= voluntary
    (20:39:02) JhenMohran: no
    (20:39:08) JhenMohran: this promotes less bias
    (20:39:09) JhenMohran: IE
    (20:39:18) Human Destroyer: not really
    (20:39:22) Human Destroyer: there's always bias
    (20:39:32) JhenMohran: HD: "I want to review Lotr because Jhen made it and he is so cool" "I dont want to review theme x cause looks bad"
    (20:39:49) Human Destroyer: >implying jhen is cool
    (20:39:50) JhenMohran: means that QC member has to look at that theme
    (20:39:55) JhenMohran: >implying otherwise
    (20:40:01) Human Destroyer: But what if I don't /want/ to look at that theme?
    (20:40:08) Human Destroyer: (as a theoretical QC member)
    (20:40:09) JhenMohran: means that "looking bad" themes never make it through QC
     
  8. Roild

    Roild Member

    Joined:
    Jul 21, 2012
    Messages:
    861
    Likes Received:
    11
    PO Trainer Name:
    Roild
    Not all the QC know how to read code and require testing to see if it functions well. Also, an indication of player reception and generic balance (for an understanding of how a bunch of theoretical roles would work together) helps. For some things, I can tell. I'm more interested in player reception to the theme. Some themes are enjoyed a lot, others are just 'ok'. It also kind of sucks when a theme is JUST 'ok' and functional, but nothing really special about it. :x
     
  9. IceKirby

    IceKirby A.K.A. RiceKirby

    Joined:
    Apr 26, 2011
    Messages:
    2,176
    Likes Received:
    635
    PO Trainer Name:
    RiceKirby
    This is why the theme author is asked to explain the roles when making the theme thread.

    I'm not saying tests serve no purpose, but that QC have been relying way too much on them, which has lowered the quality of the feedback the theme makers can get directly from the reviewers. What I mean here is that we often see reviews that are basically saying "the logs explain everything", so the theme maker gets barely any suggestion or explanation about stuff that is not working. Ideally, a Test [1] shouldn't be the first/only thing a QC member says when reviewing a theme, but it should come when there are points that really need attention instead of "Looks interesting, let's test just to make sure".

    Also, I like HD's suggestion of having a fixed number of Approvals required. A rule explaining what happens when the number of approves required changes was added recently, but that's mostly likely going to cause some confusion. I also like the "sorting themes for specific QC members to review" idea.
     
    Last edited: Sep 25, 2013
  10. Beast

    Beast I do what I want

    Joined:
    Sep 7, 2012
    Messages:
    925
    Likes Received:
    31
    At one point I suggested posting the number of approves needed for each theme when the theme was made and that was the number that theme needed to be approved. It was deemed unfair thought because people would just submit themes now and the later themes when we have more QC would be harder to get through.
     
  11. Scatterbrain

    Scatterbrain You only live once*

    Joined:
    Jul 5, 2012
    Messages:
    865
    Likes Received:
    4
    i don't see why we can't just get more QCs, I mean the idea in itself sounds basic but it seems like the only other approach to it as opposed to making lists that are going to have multiple variants and may not always be up to date. I think QCs should be qualified to understand JSON (the language themes are made in), too, because code isn't always on par with what the roles on the thread say (and vice versa). This helps because seeing as how QCs can give criticism like all other people, they can help further their arguments and make their decisions a lot faster than it would be with people who don't know how to read code.
    If we're dealing with themes from 2012 when there were like 12 or 13 QCs, then you can subtract and invalidate all the retired QC's opinions. One thing I've rarely ever seen is a QC changing their Approvals or Declines, or removing them at all, outside of Fuzzy. Opinions change over time, and I believe that getting Tests is the only way to see if the QCs (and of course, these beloved Mafia Players) are going to like it.
     
  12. Shazan

    Shazan Obey to Kyubey

    Joined:
    Jan 21, 2013
    Messages:
    591
    Likes Received:
    14
    PO Trainer Name:
    Obey to Kyubey
    I don't think that the number of Qc members is the main problem @Matt Ayala after all the number of approve needed rise when there are more of them.
    And personally i don't think that we should have a system where a QC member has full liberty of what themes they will review , a system like this promote themes with an appealing flavour like a famous series or anime and damage other themes with a less appealing flavour but with an interesting gameplay, personally this is how i felt when my theme trust was immobile for an entire month in need for the final test vote and then approved pretty fast for that period when it has been well received by players.
    Anyway looking at your comments i have elaborated some new thoughts.
    1) the number of approves needed fixed on a precise number or at least it should be equal to half of QC members rounded for defect instead then half of them +1 this way we can have a better distribution of Qc members between themes reviewed.

    2)A forum post (maybe for only QC members and former) where to or discuss the themes that needs priority to ne reviewed or where randomly decide 2 themes that needs review and then half Qc will chose to review half of it and the other half will have the other theme.

    3) Test shouldn't be obligatory but only pursued in case of particular gameplay and complicate coding like it could be themes ad Oracle and chrono trigger, or when a qc member is not sure if the theme could be enough appealing for server. In any case if there is a test who proposed it should post a precise motive for testing and when it s tested they should post at least a log and at the end of the test says if the problem that they pointed has been solved or not.
    Personally one of the motive why the tests get so prolonged i think is because there isn't always a precise motive for it, so if who propose it give a precise problem to Search the test will be faster.
    P.s. Maybe the test should endure 1 week instead of 3 days but without extra time like lately happened.

    4) Maybe we could have a post where theme coders can request to be reviewed so to have a better idea of what themes chiose to review.

    I don't know maybe my points are too harsh so if you guys comes up with something better i will appreciate it , surely things can't stay as they are now
     
  13. Scatterbrain

    Scatterbrain You only live once*

    Joined:
    Jul 5, 2012
    Messages:
    865
    Likes Received:
    4
    Why not just have a forum for QCs and MAs? (could be hidden from regular users) Would solve some of the problems you guys have.
     
  14. Angel Flonne

    Angel Flonne macaron

    Joined:
    Jan 24, 2013
    Messages:
    509
    Likes Received:
    3
    PO Trainer Name:
    Angel Flonne
    Matt, there's the Victory Road forum, and MAs and other channel mods have special access to that place. I don't think QCs do, though.
     
  15. Scatterbrain

    Scatterbrain You only live once*

    Joined:
    Jul 5, 2012
    Messages:
    865
    Likes Received:
    4
    Still a QC forum would be really nice especially in terms of trying to organize everything
     
Moderators: Pamitha, Water