Level Judge Application

Topics about events/announcements that are no longer relevant.
Zenithbreak
Purple Yoshi Egg
Purple Yoshi Egg
Posts: 133
Joined: Thu Apr 23, 2015 10:52 pm

Re: Level Judge Application

Postby Zenithbreak » Wed Aug 17, 2016 9:34 pm

Spinda wrote:But I mean with the removal of rating subforums is there really a need for numerical scores?
Even with the subforums the ratings weren't entirely necessary imo.
Spoiler: show
Image

User avatar
Zeldamaster12
Cid
Cid
Posts: 4100
Joined: Mon Feb 24, 2014 7:43 pm
Flair: stop looking at my flair dangit
Chat Username: Zeldamaster12

Re: Level Judge Application

Postby Zeldamaster12 » Wed Aug 17, 2016 9:35 pm

I don't understand why a review absolutely has to have a score with it. It's nice to have I guess, but if you read the review, you'll still know what the reviewer thinks of the level.
Image

Enjl
Cute Yoshi Egg
Cute Yoshi Egg
Posts: 8562
Joined: Mon Jan 20, 2014 12:58 pm
Flair: Orphion Egamalenitar Osmos IV, Esq.

Re: Level Judge Application

Postby Enjl » Wed Aug 17, 2016 9:36 pm

I don't think scores are necessary for determining the quality of a level. Most scores I saw from official judges seemed like they were pulled right out of nowhere, with a lot of cover-up through aforementioned fillertext and arbitrary weighting of categories. Simply saying how much a judge enjoyed a level based on the flaws and strenghts of a level mentioned previously is as legitimate and causes less confusion, in my opinion. Especially with the removal of the subcategories I hardly see any reason to use scores.

E:
Re: Aeromatter:
First one seems legit, second one hardly gives any suggestions on how to improve which IMO should go into a review.

E:E: Literally everyone ninja'd me to the point with the scores thing now GG.
Last edited by Enjl on Wed Aug 17, 2016 9:36 pm, edited 1 time in total.
The ABC of Creation
________________________________________________________
Image

PixelPest
Link
Link
Posts: 7116
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Current Project: Boom Boom Evolution
Chat Username: PixelPest
Contact:

Re: Level Judge Application

Postby PixelPest » Wed Aug 17, 2016 9:36 pm

AeroMatter wrote:http://www.supermariobrosx.org/forums/v ... 54#p226232 In this review I gave specific examples that the level creator could change. My shortest reviews had screenshots of where the designer messed up and how to fix them, like right here http://www.supermariobrosx.org/forums/v ... 85#p221168 and another snow level that I can't seem to find and I think they would have been just as good to include too but I already sent three reviews instead of 2.

EDIT: Since I didn't put up a random number and go on and on about nothing in my reviews, they're trash. PP for LJ.
There really isn't enough content still and although you provide some specific examples, it isn't nearly enough. The numbers in my reviews also are definitely not random. I have a certain methodical approach that I use in most cases. Although you don't go on and on, you don't even go on to be honest

Zenithbreak
Purple Yoshi Egg
Purple Yoshi Egg
Posts: 133
Joined: Thu Apr 23, 2015 10:52 pm

Re: Level Judge Application

Postby Zenithbreak » Wed Aug 17, 2016 9:40 pm

What's the point of review scores anyway? What does a meaningless number add to the review?
Spoiler: show
Image

User avatar
Aero
Palom
Palom
Posts: 4622
Joined: Fri Mar 28, 2014 2:51 pm
Flair: Pirate economy
Chat Username: aero

Re: Level Judge Application

Postby Aero » Wed Aug 17, 2016 9:40 pm

I don't care about your methods? I keep my reviews concise so people actually read them instead of looking at a block of text with numbers.

Enjl
Cute Yoshi Egg
Cute Yoshi Egg
Posts: 8562
Joined: Mon Jan 20, 2014 12:58 pm
Flair: Orphion Egamalenitar Osmos IV, Esq.

Re: Level Judge Application

Postby Enjl » Wed Aug 17, 2016 9:43 pm

Let's get rid of filler in one of your reviews then and see how it compares, PixelPest!
I didn't find this level to be nearly as bad as people made it out to be. It's a relatively solid level and a good length for what it has to offer. Although it is relatively simple, it mostly exhibits positive design elements throughout; good spacing between BGOs to avoid clutter, good placement of NPCs including power-ups for its difficulty (however not great for coins which I'll discuss after this), and sufficient variation.This level is good for the most part. The leap of faith at the end is a bit much, not only because there isn't anything else in the level like that, but it also just isn't really good to throw that at the player when the rest of the level is really easy. You shouldn't have leaps of faith in your levels though.Some areas needed some more coins, but for the number there were they were arranged in a pretty good manner, however it would have been a little better if some of the designs were angled, for example in an "m" pattern.The two Bullet Bill launchers also really weren't needed since they weren't present anywhere else in the level. Why? Graphics-wise this level was fine. Everything seemed to fit nicely, including the Bob-ombs. You could look into using a custom graphic for an SMB1 Roulette Exit in the future to help improve the SMB1-ness of your levels. The level looked good.I thought that the music was a fantastic choice and really helped contribute to the dreamy atmosphere of the night.
This review offers less feedback that's actually helpful than Aeromatter's Birdo level review. It ends on a 6/10 which I don't know where it's coming from. I'm seriously doubting your position.
The ABC of Creation
________________________________________________________
Image

User avatar
WindyDelcarlo
Swooper
Swooper
Posts: 100
Joined: Sat Apr 12, 2014 7:06 pm
Current Project: Who knows? Back after what feels like absolutely forever!

Re: Level Judge Application

Postby WindyDelcarlo » Wed Aug 17, 2016 9:50 pm

A lot of that space is the explanations. Which, I personally don't think is right to cut out. The red text that you have removed includes upsides of the level: good spacing and placement of certain objects, and downsides. I think you need to look at it with an unbiased eye, as you seem to be making a mistake about commenting on having too much text as a result of (I can assume based on the rest of the conversation) previous encounters with either PP or Aero, when, frankly, a lot of it does serve a purpose.
Electrokinesis is fun.
Struggling to remember what and why I did things when I was here the last time.

Enjl
Cute Yoshi Egg
Cute Yoshi Egg
Posts: 8562
Joined: Mon Jan 20, 2014 12:58 pm
Flair: Orphion Egamalenitar Osmos IV, Esq.

Re: Level Judge Application

Postby Enjl » Wed Aug 17, 2016 9:56 pm

I agree I went a bit harsh on the roulette exit thing and the placement sentence, though a lot of the stuff I commented out was really arbitrary, personalised and random, or could've been summarised as I did in a short sentence. If the review merely reflects on the level and declares what's good and what's bad without elaborating on why it's good or bad, it might aswell not mention it specifically at all.
For instance, I would love to know why m-shaped coin patterns would be such a good idea, or why having a only a few bill blasters in a level is a bad idea. There is no obvious reason, the designer is just lead to believe that "this is just how it is", which, in my opinion, is a really bad move. In the examples I gave it boils down to a particular judge's personal preference in level structure which doesn't universally apply as something good or bad.
The ABC of Creation
________________________________________________________
Image

User avatar
WindyDelcarlo
Swooper
Swooper
Posts: 100
Joined: Sat Apr 12, 2014 7:06 pm
Current Project: Who knows? Back after what feels like absolutely forever!

Re: Level Judge Application

Postby WindyDelcarlo » Wed Aug 17, 2016 9:59 pm

Yes, it's not perfect. If I stick around in the community longer than a few weeks, I'll probably try to do reviews myself, but I think it's a reasonable level to be at.
Electrokinesis is fun.
Struggling to remember what and why I did things when I was here the last time.

PixelPest
Link
Link
Posts: 7116
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Current Project: Boom Boom Evolution
Chat Username: PixelPest
Contact:

Re: Level Judge Application

Postby PixelPest » Wed Aug 17, 2016 10:00 pm

I think the ratings are important because users can use them to see improvement. If they're accurate and the user is improving, they should be able to see their scores start to work their way upwards. This is important imo and I'd definitely want to see it when my levels are reviewed.

In response to Enjl's post: a lot of that isn't filler. I commend the user for specific things they've done right. If I just said "good for the most part", they wouldn't know what is good about it. The way I mentioned the leap of faith part not only refers to them being bad, but it also eludes to the fact that it causes difficulty spike into a relatively easy level. I thought that the arrangement of coins was a little boring since if I remember correctly, they were just placed in horizontal lines, so I gave a suggestion on another way to place them to provide more variety to their positioning in that statement. Although I probably should have made it clearer, I mentioned the Bullet Bills from a variety perspective--they didn't really need to make two appearances in the level; if you're going to use an NPC in your level, you should really actual use it. Instead of saying "it looks good", I told the user what fit and gave them a suggestion as to something that could be changed to give the level a more SMB1 feel. The way I do my scores in most situations is starting at a 5 and then adding or removing points based on good and bad things I find (usually different amounts of points anywhere from 0.1 to 2.5 based on how frequent things are good or bad in a level). I didn't pull that score out of thin air by any means

User avatar
Aero
Palom
Palom
Posts: 4622
Joined: Fri Mar 28, 2014 2:51 pm
Flair: Pirate economy
Chat Username: aero

Re: Level Judge Application

Postby Aero » Wed Aug 17, 2016 10:15 pm

PixelPest wrote:I think the ratings are important because users can use them to see improvement. If they're accurate and the user is improving, they should be able to see their scores start to work their way upwards. This is important imo and I'd definitely want to see it when my levels are reviewed.
Everyone has different metrics for their scores, so levels don't just gradually get better by a certain standard. What you get is a circle jerk in what was The Best forum with only select design styles were ranked higher by LJs with a selective view on how good a level is.
PixelPest wrote:In response to Enjl's post: a lot of that isn't filler. I commend the user for specific things they've done right. If I just said "good for the most part", they wouldn't know what is good about it. The way I mentioned the leap of faith part not only refers to them being bad, but it also eludes to the fact that it causes difficulty spike into a relatively easy level. I thought that the arrangement of coins was a little boring since if I remember correctly, they were just placed in horizontal lines, so I gave a suggestion on another way to place them to provide more variety to their positioning in that statement. Although I probably should have made it clearer, I mentioned the Bullet Bills from a variety perspective--they didn't really need to make two appearances in the level; if you're going to use an NPC in your level, you should really actual use it. Instead of saying "it looks good", I told the user what fit and gave them a suggestion as to something that could be changed to give the level a more SMB1 feel. The way I do my scores in most situations is starting at a 5 and then adding or removing points based on good and bad things I find (usually different amounts of points anywhere from 0.1 to 2.5 based on how frequent things are good or bad in a level). I didn't pull that score out of thin air by any means
A lot of that you could have explained in your review instead of here defending your bad reviews.

PixelPest
Link
Link
Posts: 7116
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Current Project: Boom Boom Evolution
Chat Username: PixelPest
Contact:

Re: Level Judge Application

Postby PixelPest » Wed Aug 17, 2016 10:23 pm

There's still a general correlation. And btw, most of what I said except for the clarification part regarding the Bullet Bills was in my review

Zenithbreak
Purple Yoshi Egg
Purple Yoshi Egg
Posts: 133
Joined: Thu Apr 23, 2015 10:52 pm

Re: Level Judge Application

Postby Zenithbreak » Wed Aug 17, 2016 10:32 pm

The issue with review scores is that every reviewer has a different method of determining the score, rendering them pretty much worthless to the level creator.
Spoiler: show
Image

PixelPest
Link
Link
Posts: 7116
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Current Project: Boom Boom Evolution
Chat Username: PixelPest
Contact:

Re: Level Judge Application

Postby PixelPest » Wed Aug 17, 2016 10:38 pm

Zenithbreak wrote:The issue with review scores is that every reviewer has a different method of determining the score, rendering them pretty much worthless to the level creator.
But they'll still be in the same range

Enjl
Cute Yoshi Egg
Cute Yoshi Egg
Posts: 8562
Joined: Mon Jan 20, 2014 12:58 pm
Flair: Orphion Egamalenitar Osmos IV, Esq.

Re: Level Judge Application

Postby Enjl » Wed Aug 17, 2016 10:56 pm

Except they won't? Have you seen the reviews for ANY contest?
The ABC of Creation
________________________________________________________
Image

Zenithbreak
Purple Yoshi Egg
Purple Yoshi Egg
Posts: 133
Joined: Thu Apr 23, 2015 10:52 pm

Re: Level Judge Application

Postby Zenithbreak » Wed Aug 17, 2016 11:00 pm

PixelPest wrote:
Zenithbreak wrote:The issue with review scores is that every reviewer has a different method of determining the score, rendering them pretty much worthless to the level creator.
But they'll still be in the same range
How would a range be helpful when every reviewer will have different opinions on every level?
Spoiler: show
Image

PixelPest
Link
Link
Posts: 7116
Joined: Sun Jul 12, 2015 5:38 pm
Flair: Tamer of Boom Booms
Current Project: Boom Boom Evolution
Chat Username: PixelPest
Contact:

Re: Level Judge Application

Postby PixelPest » Wed Aug 17, 2016 11:01 pm

Enjl wrote:Except they won't? Have you seen the reviews for ANY contest?
And if Wraith Adamknight isn't a judge they're often within 1-2 points

User avatar
WindyDelcarlo
Swooper
Swooper
Posts: 100
Joined: Sat Apr 12, 2014 7:06 pm
Current Project: Who knows? Back after what feels like absolutely forever!

Re: Level Judge Application

Postby WindyDelcarlo » Wed Aug 17, 2016 11:02 pm

The thing is, ratings by different judges create an average that gives you a scale of where you're at. Systems where judges give a rating out of 10 are always going to be different for each judge unless it's absolutely flawless, but that doesn't mean the system doesn't work.
Electrokinesis is fun.
Struggling to remember what and why I did things when I was here the last time.

Zenithbreak
Purple Yoshi Egg
Purple Yoshi Egg
Posts: 133
Joined: Thu Apr 23, 2015 10:52 pm

Re: Level Judge Application

Postby Zenithbreak » Wed Aug 17, 2016 11:13 pm

Or people could just read the review which will give them a much more accurate idea of the strengths and flaws in their levels, but then they'd have to read, and reading is hard!
Spoiler: show
Image


Return to “Events and Announcements”

Who is online

Users browsing this forum: No registered users and 1 guest