Suggestions Forum

Discussion in 'Suggestions' started by IceFrog, May 28, 2009.

Thread Status:
Not open for further replies.
  1. Mazil

    Mazil Well-Known Member

    Hey,

    I never read the whole 37 pages (please, that's a lot just to make a 100% informed reply). But I do have a few questions.

    Was there ever an agreed format/solution to the suggestions forum? Will this ever be introduced?

    I'm actually quite interested in seeing something pull through out of all this discussions. I agree completely with the original post of this discussion (by icefrog). There are too many ideas, and we need to move to a forum discussion of only the best quality, and most usability. I've seen many suggested frameworks and I have imagined a few that could possibly work.

    I'd love to contribute here and give my 2c, but I just wanted to know if these discussions are still open. If I got and post my wall of text, will people be around to read and discuss it?

    Just curious :)
     
  2. EacieX

    EacieX New Member

    I have a suggestion about buying items in any shops..

    Many players also wrongly buy items... Like buying an item that is not intentionally bought.. I would like to suggest, "When buying an wrongly item.. It can be sold again for same price, but it has a time like "10-15sec",
     
  3. Foede

    Foede Forum Manager

    @Mazil: There has been a lot of discussion on this thread, mostly around why the old DA system is not good, and some ideas tossed here and there about how a possible new system should work.

    There has been no definite answer by the admins, so there's nothing left for us but to keep brainstorming and creating new ideas for Ice :p

    @EacieX: This thread is ONLY for discussions about the "Suggestion Forum" as a whole, and any of its sections. If you want to suggest something, create a new thread in the corresponding section (use the search button before you do, in case your idea was already posted).
     
  4. darxide

    darxide Well-Known Member

    On that note, would the forum staff or even IceFrog be willing to make the suggestion forums have an interface?

    You would select the type of new suggestion you want and have a uniform format.
    ex:
    - you click "New Item Suggestion"
    - it asks for the name of the item
    - it asks for the URL of the image/images you want to use
    - it asks for the effects, cost, and recipe of the item (recipe pictures would automatically be included)
    - it asks for flavor text
    - it asks for additional notes
    -----------------------------

    Heroes and other ideas can have a uniform format as well. The page that asks for these things would look like a "create new account" type of thing so you can browse for links and not having to switch windows. (when you do the list tag on posts, a popup asks you for descriptions/links and you cant really browse for them unless you previously already copied the link)
     
  5. Foede

    Foede Forum Manager

    Although that could work nicely for certain suggs like Heroes, Items and skills where they are kinda limited in regards of *what* to suggest (the kind of suggs that benefit the most of templates), I'm not sure it would work with Remakes, Visuals and Others.
     
  6. EacieX

    EacieX New Member

    Ok, Sorry two mistakes
     
  7. JJE92

    JJE92 Moderator

    Well, at least for remakes and visuals:
    If I were IceFrog and wanted to remake something or change a visual, I'd use the search function and look through some threads. They are not that many of each hero/skill, it's in most cases only 3 or 4 pages and half of that is filled with threads in which the hero was just named in one post, but not touched.
    I did this quite a few times if I wanted to look for something and it's pretty effective, would be more effective if the search function wouldn't be as retarded as it is, but w.e.
    And else, I'd just ask some people to look through the most popular threads or also do it myself if there's enough time.

    For hero, ability and other suggestion, it's more difficult, however and well, I guess these are the main problems to fix, because Imo generally Remake and Visual forums are fine as they are now. The only thing I'd really like to see changed there is to remove balance suggestions out of remakes. They are mostly not well thought out and for this we have Balance Debates already.


    I personally would appreciate to have some place where I can find the most unique hero ideas, too, but that's hard to do. Sometimes, I'd just like to search for a specific hero model, or hero concept (e.g. a hero that damages by running around as much as possible, or w.e. else) and finding such things in the mass of threads is nearly impossible.
    The problem is organisation, because we can only search for mostly useless things. Searching for a specific skill type (blink, teleport, displacement, damage, ...) in ability sub-forum is impossible.

    I doubt that there will be an all-in-one solution, where the best threads are pointed out and we can easily find what we look for.
    The problem with the first one is already a basic one: what are the best hero suggestions. Hero suggestions that contain one unique skill? Why not, they can already be of so much use. Hero suggestions with a unique concept, but improvable skills? Why not, they also can be of so much use.
    Most hero suggestions only have a skill or an idea implemented anyway.
    The problem with the second one is that it's very hard to do I guess.
     
  8. Mazil

    Mazil Well-Known Member

    I know that something like my framework has already been posted. I thought it could work, and work well. I'll just jump right in there and explain my thoughts and we can just discuss it from there.
    [nqb]
    The Problem:


    Good skill, item and hero ideas are often lost in the current forums. Finding something good is often far too much effort for Icefrog and his team.[/nqb]
    [nqb]
    The Solution:


    There needs to be filtering, both automatic and manual. Ideas that are good need to be recognized. Potential in these ideas need to be brought out, and investigated. More than one audience needs to see the ideas and contribute to them.[/nqb]

    My proposal:


    Keep the current forums, they work to an extent. Most suggestions that have proper formatting will get a little bit of community feedback. Keep this as a base level, a "sandbox" for new and developing ideas.

    People need a place to put ideas that they think are good. Yes, the suggester them self. Not a moderator, or anything like that. Individuals need the chance to place suggestions they think are good, into a higher level of appraisal. In my opinion, we should let people have this opportunity.

    Step 1: Make a new subforum called "Published Suggestions":

    This will either be a large one containing: Heroes, Items, Abilities (or whatever) or make the "Published" subforum stemming from the original forum that it pertains to. I.E: inside heroes, you will fine the Published heroes. Inside abilities, Published abilities. Etc.

    Conditions of posting here:

    • 1 thread, per user, per 1-2 months
    • Created thread must already exist in the sandbox for 1 month
    • No further modification of suggestion once posted here
    • Public is not permitted to discuss ideas here
    • Staff(explained later) will review suggestions here
    • Threads are closed 2 weeks after they are posted

    This, in my opinion is good filtering. Users cannot spam this forum with any old idea they think has a chance. They must pick their best ideas. Furthermore, there needs to be an attempt at previous discussion. By staying in the sandbox (the forum that we have now) for at least one month, there is atleast some potential for the idea to be reviewed and improved upon. Users must become self critical. "If I post this here, will it be liked". "Have I wasted my suggestion for the month (or 2)?"

    If our users become their own filters, this alone creates better ideas, which are more relevant. Users who submit ideas that are quickly turned down in this forum will learn quickly what is expected of them (at least I hope. Is that assumption too optimistic?). Regardless, the Published forum will guarantee that we can easily find suggestions that have EFFORT behind them. At least, I hope.

    Step 2: Hire "Reviewers" or "Voters" and "Review Moderators", above them

    This is the bit that becomes harder. Face it, any amount of automatic filtering is fine, but it's not going to work after a while. Real people need to view the suggestions and think about what's being written. It's just a given. At some point or other, we need people with the capability to do something about a suggestion. We need people who can say "I like this idea, and now I'm going to make sure you're recognised".

    We need to hire staff for this to work. I look at the staff lists and there are plenty of people assigned to the moderation of suggestions. You CAN trust people to be reliable, and post frequently to a particular standard. It's possible.

    For my idea, we need to hire 1-2 teams per section. (1-2 teams for heroes, 1-2 teams for items. Etc). Each team will have 1 moderator, who has increased forum capabilities (like the staff we have now. The selected people could even be existing staff. Rather than hiring new ones). The teams will also have 5 or so delegates. These delegates are just regular people. They will be known as "voters" or "reviewers". They have no extra forum capabilities or tags. All they have, is a team leader, who they can speak with about the ideas they find.

    When a suggestion is published, the voters of the section the suggestion was posted in, will post their thoughts. Should a voter see potential, or something he likes. He will add to his post, "I have forwarded this to my superior" or something like that.

    The intention would be, that only 25% of published threads get a mention to a superior.

    The moderators will get these messages from their delegates. They will be directed to threads that have been seen to have potential. The Moderator then examines the thread using a strict marking scheme.


    Step 3: Develop an All Encompassing Marking Scheme:


    Hiring staff can be easy. Just go and look at users who make consistent, good quality posts, and offer to hire them. It can be done and the proof is found in some of our current staff today. If we are talking a maximum of 6 moderators across 3 forums (heroes, items, abilities) with 5 other forum users assigned to them... then that is 36 people we need to hire. Considerable, but not impossible to find enough people.

    When a moderator is given a thread to look at, they should be using a Marking Scheme, to grade the suggestion. Hiring staff to find potentially workable ideas is easy. But creating a scoring mechanism will be so much harder. But it would be worth it.


    Benefits of a well created marking scheme:

    • Will make a moderators job easier. They do not need to think of things to say. All parts of a suggestion that need covering will be laid out for them
    • Will quantitatively show what parts of a suggestion are good, and what needs improvement.
    • Will allow users to see exactly what IceFrog looks for in a suggestion
    • Should IceFrog need a different style of idea, the Scheme could easily be changed to suit.


    Downfalls of a poorly created scheme:

    • Could make a moderators job harder to get a point across
    • It could oversee parts of a suggestion that actually have great potential but need work
    • It may cloud what is actually expected from a suggestion
    • It could cause more confusion than clarity

    Appropriate marking schemes would be difficult to develop. However, I maintain the belief that a well made marking scheme will help the suggestions forum immensely.

    A moderator that is send a link to look at a published thread will create their report card for a suggestion and post it in the thread, after all the voters have posted. The report card will examine whether the suggestion will move on, or has failed. I expect 5% of all published suggestions to get this far.


    Step 4: Move the suggestion to private moderator discussions.


    From here, the other moderators of the suggestion forums will be instructed to judge the suggestion. Should there be a unanimously positive acclaim for the idea (expected 1% of published threads). Icefrog will get a final say. He could either reject, accept or deffer the suggestion.

    Rejected suggestions become just that. The marking scheme is designed to allow the desired suggestions to trickle through. However sometimes, moderators may view it as a good suggestion and Icefrog will not. The suggestion will be unsuccessful like ever other.

    Accepted suggestions and Deffered suggestions will have obvious potential and will either be implemented without futher discussion or moved to the "Open suggestions" Forum.


    Step 5: Open suggestions and the wider forum's views:


    I return to the original post of this thread. The aim is to have a low number, of good quality ideas, that can be developed through meaningful discussion and brainstorming.

    Go look at the requests forum. Icefrog posts there projects that he wants the community to be involved in, for further inspiration and work. With a little for structure to the conversation, if should be a fantastic format for the published suggestions that become highly acclaimed. If Icefrog can get workable ideas from these discussions, that's good and the purpose of this filtering system has been achieved.

    Conclusions:

    In my opinion, this requires very little continuous work from staff. A group of 30 or so people just give a few quick reviews to a few threads a couple of time a week. Moderators are notified of a handful of threads that have top notch ideas, and some marching scheme will be applied. If the idea holds up against the marking scheme, Icefrog and his team can take a look and either accepct, reject or pass the idea to further "open" discussion.

    It fulfills its purpose. We filter from the astronomical amount of suggestions down to only a handfull. These ideas can be futher discussed. This fulfills the intention of the forum redesign. A couple of top quality ideas are discussed with meaningful intention. People have motivation here, if the discussions lead to good results, the suggestion ultimately gets implemented. This will please both users who helped to make it, and Icefrog, because the idea will no doubt suit him and his needs as game developer.

    My only personal concern is creating a marking scheme that can reward concepts that are not developed enough yet, but have potential. However, there may be no need for that if my clause of "ideas must be in the sandbox for 1+ months before being published" yields any fruitful discussions.

    Anyway, I hope this post can fuel some more discussion.
     
    Last edited: Mar 2, 2011
  9. NoThlnG

    NoThlnG Well-Known Member

    @Mazil : What you said is almost same with the old DA potential system and it is proven that it is not effective because ppl just need to make suggestions THAT can pass the scoring instead of being implemented.

    IMO, we need to filter "bad" suggestion instead of "good" one. In this way, at least suggestion that is unworthy being looked will be removed.
     
  10. Mazil

    Mazil Well-Known Member

    Fair enough. I never got the chance to look in depth at that system. I wasn't so interested in this section of the forums back at DA. Obviously, now that it's closed, I can't go investigate the system and it's flaws anymore. (not so much having people tell me how it worked. I'd much rather go and see some thread examples myself where the system is clearly failing)
     
  11. Mazil

    Mazil Well-Known Member

    How does a "Point spread" evaluation system sound? Just an idea I had. I have seen this work in surveys and "personality" tests.

    Their use in this forum would mostly be for heroes (mostly skillsets), however they could be applied to items and single skills.

    Their drawback is that they are relative. They will not say "skill 1 is good" rather they will indicate "skill one is better than the others" or "all skills are equally good (or bad)". That is their only problem, they can tell you what a user likes more than something else, but they cannot say if it was any good to start with. That distinction is something I am trying to come up with.

    As an example: Using PotM
    [NQB]

    Skill Originailty:
    Distribute exactly 20 points among the skills, based on their relative originality​

    [TABLE=" "]Starfall|Elune's Arrow|Leap|Moonlight Shadow
    3​
    |
    7​
    |
    5​
    |
    5​
    [/TABLE]
    [/nqb]
    [NQB]

    Skill Balance:
    Distribute exactly 20 points among the skills, based on their relative balance​

    [TABLE=" "]Starfall|Elune's Arrow|Leap|Moonlight Shadow
    5​
    |
    4​
    |
    6​
    |
    5​
    [/TABLE]
    [/nqb]
    [NQB]

    Skill Enjoyment:
    Distribute exactly 20 points among the skills, based on their relative enjoyment​

    [TABLE=" "]Starfall|Elune's Arrow|Leap|Moonlight Shadow
    4​
    |
    7​
    |
    6​
    |
    4​
    [/TABLE]
    [/nqb]

    To save time I'll just give some discussion about the originality scorecard only.

    So in my opinion, I find arrow the most original skill of potms set. You can easily see that there, I gave it a 7, the highest. However, you can't gather whether I like PotM at all. The 7 could mean "this is the best of a bad bunch" or "this 7 is the icing on the cake". E: And in this case, by no means, would I call starfall an unoriginal skill. Its just when I'm comparing it to Arrow, I need to consider that Starfall is just an aoe nuke with a little extra single target damage. Whereas Arrow is IMO a really awesome and original ability.

    E:But you see, if there is a way I can say, "The entire set of skills is very good" then the reader would go "oh, he likes it all, but arrow is just his favourite". If I can find a way to incorporate another question like that, you can actually gather a lot of meaning from a scorecard like that.

    However, I've been thinking about this and think that it's workable.

    I was unsure if a votes system was still something being considered. Or if it had been completely rejected for the future suggestions framework. Just asking/interested :)
     
    Last edited: Mar 4, 2011
  12. kitchen

    kitchen Well-Known Member

    I don't think anything with numbers is going to help, and that includes votes and rubrics and scores and anything like that. Discussing ideas and bringing about good ones is what we're after, and we just need a format that is conducive to that.
     
  13. Mazil

    Mazil Well-Known Member

    But you do need some sort of criteria. If there is no way to say "This idea is better" or "The Concept of this skill is workable", then how do you know what a good idea is? Furthermore, how do try to sort it?

    I'm not trying to say a vote is the answer, hell no. I wasn't trying to say that a Points-Spread Vote would be the only thing you would use to judge something. However, if there is no quantified value assigned to a suggestion, you can't ever really sort it.

    While initially that doesn't sound like a big deal, even, a good thing. We'd just discuss ideas, no quantifying of the concepts involved. It will run into a problem. We will have no means of separating ideas into ones that are worth improving and those that should have their discussion discontinued.

    If, as IceFrog has requested, we need to move to discussing low amounts of quality ideas, there has to be some way to sort a suggestion by its quality. I won't reject the possibility of determining a general level of quality through pure discussion, that's easy. But, if it comes down to, "You must choose one of these two", or, "You must choose one of these 5", what are you going to do?

    Ofcourse, the 5 ideas are all fantastic, everyone knows that, they've been discussing it for weeks. But nobody can put their finger on exactly which one is better. "Oh but this one is more creative", "Yeah but this one is more balanced", "Pfft, but this one is far more fun". See what I mean? Once you start comparing decent suggestions, determining quality and value to the game, through pure discussion becomes difficult.

    I don't want to say you need numbers in the system, but you will need some sort of criteria. Eventually.

    Even art and design work at a college or university. Everyone who enrolls in an art or design degree would be fantastic. They would know that, because that's why they enrolled; they were really good. Music is another example too. How do you think you grade a performance, or composition, or design, or an artwork? There sometimes are numbers in the marking schemes. but most importantly, there is a marking scheme. How else do you determine the students grade? You can't just say "yes, thats an A+ or no, that's a failing grade". Not without a way to measure its worth.
     
    Last edited: Mar 4, 2011
  14. kitchen

    kitchen Well-Known Member

    There's an old axiom in hero suggestion theory that states: good ideas will get attention and views and replies.

    The problem is that this is only half true. Lots of good suggestions go ignored, and lots of meh suggestions suddenly become the phenom of the forum.

    That said, I still don't think criteria are the way to go. Coming up with a decent way to qualify hero ideas, for example, is a pretty tough deal, especially when you take into account the potential that some suggestions contain that further discussion can bring to fruition.

    One option I'd like to discuss is anonymity. Should the posts and topics here really show who wrote them? Perhaps if they didn't, we'd have an environment more fertile for concept development rather than possible ego-stroking or attempts at e-fame.
     
  15. Mazil

    Mazil Well-Known Member

    It's true, there are suggestions out there with great potential that get pushed down into oblivion. Nobody notices them. Often its because a bunch of stale and known-to-be-useless suggestions come back to the top of the pile. As you said, (and lots of others too) all too often, good suggestions take time to develop and need discussion to let their true potential shine through. The only thing I can think of that would ever solve this is a guaranteed read through + review from a staff member. That's just insane and unfeasible. But I just wanted to mention that because I can't think of a proper way to adequately get all suggestions noticed and discussed.

    On initial consideration, anonymous replies would actually do their job well. IceFrog could reply, for all we care (not that I think he would). But in all seriousness, anonymous replies (well, not anonymous for staff) would remove quite a lot of the psychology of reviews. "I won't listen to your opinions cus your a 0 post nub". "I'll use any idea you give me because your the most respected person around"... would all turn into "I'm going to actually think about this text unbiased, because there is no name attached to it". I can respect that idea, because it would help in achieving this forums goal.
     
  16. Axidos

    Axidos Well-Known Member

    Keep in mind the basic pitfall of metrics: people give you exactly what you ask for. Metrics can and will be gamed.


    Charles Dickens was paid according to how many words he wrote. So what did he give his publisher? Long books full of lots and lots of words.
    He could have said "It was the best and worst of times" but he was paid almost twice as much for writing "It was the best of times, it was the worst of times" instead. His books could have been shorter and yet just as good (maybe even better), but his publisher was going for words and that's exactly what they got.


    If you start asking for original, balanced and enjoyable spells, you get original, balanced and enjoyable spells. Are any of them actually worth implementing? Who knows, you didn't ask for that.
     
  17. FleshTRNDstone

    FleshTRNDstone Well-Known Member

    What might make sifting through all the masses of suggestions a bit easier perhaps, say, under Suggestions, there are oragnized pages for each group of ideas.
    Might look something like:
    Suggestions>Hero>Remake>e.gTraxex OR
    Suggestions>NewHeroes>UnusedModels>e.gDragonHawkRider

    this way, whenever the Dev Team wants to, say, do a remake of say hero, or want to implement say model, its way easier to go through all the suggestions, scanning for possible ideas, concepts etc.
    Because its one thing if your idea gets disapproved of, but way worse if an actually awesome idea, no matter whose it is, goes unnoticed.

    Ranging from very old to very new, all suggestions concerning a specific hero/model would be under one list.

    Just my thought, not an entirely new suggestions forum, but might help the current one till then.
     
  18. Mazil

    Mazil Well-Known Member

    Sorry, this was a bit of a rushed reply. I hadno time to expand my ideas.

    I was just thinking that if you had the criteria, but it wasn't shared publicly. People might think they have a great idea that they think fits the criteria. Only to have it rejected. It could be used by judges as an aid. But since it is not avaliable to the public they cant 'cheat the system'

    Sorry, I was short on time and can't elaborate more.
     
  19. Fire525

    Fire525 Well-Known Member

    I think that anonymous post idea would actually be quite a good one to be honest. It would remove any kind of elitist attitude, and might also help to prevent people making suggestions for the sake of making suggestions. It could also help to better distribute traffic for suggestions, as currently, if someone (Such as Yagachi) posts 15 bad suggestions, he gets more traffic in a day than many great suggestions get in a month. Likewise, a well known person will get more traffic than an arguably better suggester who is just starting out.

    The sorting system has been suggested more than once, and to be honest I see no reason not to implement. I think the current interface should be kept, but for the remake section, an option should be given to see suggestions sorted by hero/item (More or less how the guide section works).

    My two cents.
     
  20. kitchen

    kitchen Well-Known Member

    Anonymous posts might be a good idea but it'd be important for mods to know who's who so they can discipline and whatnot.
     
Thread Status:
Not open for further replies.