Story navigation: 1 | 2 | 3 | 4
On November 28, 2018, level submissions for Make a Good Mega Man Level Contest 3 (MaGMML3) came to a close. After a generous seven-day grace period for late entries, we had a total of 186 levels. That's almost twice the number of levels in MaGMML1 and 2 combined.
Our new contest motto was, "The judges are gonna die."
The devteam loaded the entry levels into a standalone executable "game" for the judges to play. This consisted of a small hub with a teleporter used to access the levels, a costume selector in case we didn't want to play as Mega Man (Flash Man was still in the works, so I picked Roll), NPCs of the judges for comic relief, and infinite E-/W-/M-Tank refills. To maintain blind judging, the levels were randomly numbered instead of identified by creator name, and any in-level material that obviously gave away the creator's identity (eg, self-insert NPCs) was temporarily redacted.
On November 28, 2018, level submissions for Make a Good Mega Man Level Contest 3 (MaGMML3) came to a close. After a generous seven-day grace period for late entries, we had a total of 186 levels. That's almost twice the number of levels in MaGMML1 and 2 combined.
Our new contest motto was, "The judges are gonna die."
The devteam loaded the entry levels into a standalone executable "game" for the judges to play. This consisted of a small hub with a teleporter used to access the levels, a costume selector in case we didn't want to play as Mega Man (Flash Man was still in the works, so I picked Roll), NPCs of the judges for comic relief, and infinite E-/W-/M-Tank refills. To maintain blind judging, the levels were randomly numbered instead of identified by creator name, and any in-level material that obviously gave away the creator's identity (eg, self-insert NPCs) was temporarily redacted.
The devteam needed several weeks to load all the levels and test them for basic functionality. Two levels were cut before they even reached the judges, one for being irreparably broken in its construction and one for containing a note threatening the judges to give the level a good score. In an effort to get us judges playing and discussing as quickly as possible, the remaining levels were added to the judge EXE in batches of 30 or so, with each judge receiving a randomized playing order for each batch. Although I didn't argue it strongly, I was a proponent of waiting for all the levels to be loaded; I felt the benefits of playing in batches didn't outweigh the potential for bias inherent in all of us playing the levels in more or less the same order.
Still, I had a system in place to minimize bias on my end as much as possible. After playing all the levels in batches and seeing the best and worst the contest had to offer, I would go back and replay everything in a newly randomized order (ignoring batch divisions), scoring and reviewing as I went. Afterward, I'd compare scores to ensure everything felt correct and consistent, replaying individual levels again if necessary.
In terms of specific process, I knew I wanted to:
Playing as often as my schedule (and the batch release schedule) allowed, and taking breaks when I started feeling tired or cranky, it took me 50 hours and 48 minutes spread across 2 months to clear the first round of playthroughs. During that time, I continually fine-tuned my process.
In the beginning, I was recording multiple levels in a single video, being a little lax about revisiting levels right away to search for missed secrets, and playing mainly for my own enjoyment. By the end, I was recording each level separately (which made it MUCH easier to track down footage later), diligently verifying 100% completion before moving on, and playing with more critical intent—in addition to playing for fun, I was checking screen transitions and testing the feasibility of a buster-only run, saving myself the trouble of doing so on a replay.
Aiding my process were a couple quality-of-life improvements along the way. First, the implementation of a map feature in one of the later batches. Second, the realization that the cheat menu had an option to show invisible collision objects, saving me the trouble of leaping into every spike pit to determine whether the creator had pulled a Magnum Man and placed spike tiles but not spike objects. Third, the revelation that noclip mode had been accessible via a hotkey this whole time, which facilitated more efficient playtesting and relieved the burden of redoing sections I'd already played into the ground.
Still, I had a system in place to minimize bias on my end as much as possible. After playing all the levels in batches and seeing the best and worst the contest had to offer, I would go back and replay everything in a newly randomized order (ignoring batch divisions), scoring and reviewing as I went. Afterward, I'd compare scores to ensure everything felt correct and consistent, replaying individual levels again if necessary.
In terms of specific process, I knew I wanted to:
- Record my blind playthroughs for future reference and eventual upload to YouTube
- Approach my blind playthroughs as a regular player and any subsequent playthroughs as a playtester
- Jot down notes as I went, to supplement the observations captured in my recordings
- Go for 100% completion without asking for hints, unless I was hopelessly stumped
Playing as often as my schedule (and the batch release schedule) allowed, and taking breaks when I started feeling tired or cranky, it took me 50 hours and 48 minutes spread across 2 months to clear the first round of playthroughs. During that time, I continually fine-tuned my process.
In the beginning, I was recording multiple levels in a single video, being a little lax about revisiting levels right away to search for missed secrets, and playing mainly for my own enjoyment. By the end, I was recording each level separately (which made it MUCH easier to track down footage later), diligently verifying 100% completion before moving on, and playing with more critical intent—in addition to playing for fun, I was checking screen transitions and testing the feasibility of a buster-only run, saving myself the trouble of doing so on a replay.
Aiding my process were a couple quality-of-life improvements along the way. First, the implementation of a map feature in one of the later batches. Second, the realization that the cheat menu had an option to show invisible collision objects, saving me the trouble of leaping into every spike pit to determine whether the creator had pulled a Magnum Man and placed spike tiles but not spike objects. Third, the revelation that noclip mode had been accessible via a hotkey this whole time, which facilitated more efficient playtesting and relieved the burden of redoing sections I'd already played into the ground.
Despite my livestreaming, recording, and playtesting experience, going through these levels was unlike anything I'd ever done. It's one thing to play levels of varying quality and character in the context of a finalized MaGMML game—the level scores, tier placement, creator names, and nearby NPCs give you a sense of what you're in for, and it's fine to let your gut reactions define your opinions. It's another thing entirely to experience the levels completely blind, in a random order, relying solely on what the levels themselves tell you about their length, difficulty, stability, and so forth, all the while maintaining a consistent process to ensure fair and impartial judging.
I encountered plenty of difficulties. On more than one occasion, I set out to play "one more level" before taking a food or sleep break, only to discover 20 minutes later that the level was not going to end anytime soon. I'm not particularly photosensitive or subject to motion sickness, but a few levels made me queasy with their gimmicks and visual effects—not so much that I couldn't finish, but enough to make judging uncomfortable. Due to programming oversights in some of the contestants' custom objects, I triggered a handful of hard crashes and had to make up lost progress (usually before learning about noclip mode).
I had to switch my player avatar when I discovered at a most inopportune time that Roll didn't have any custom sprites created for Rush Bike, so she simply turned invisible. With the devteam making updates to the engine between batches, it wasn't always obvious which glitches were newly introduced and which should've been caught by the level creators, so I started a running list of issues to raise with the devteam so I could judge Functionality fairly. The FIRE button on my controller got progressively stickier as time went on, eventually spreading that stickiness to the JUMP button, effectively rendering the controller unusable.
Oh, and I was sick with an upper respiratory infection for almost the entire duration. I had already decided not to perform for an audience while recording my blind playthroughs—after all, these recordings were personal reference material first and public entertainment second—but I would've preferred not to sound like a plague beast when I talked. This didn't affect my playing ability, but there were a few mishaps early in the recording process where I muted my microphone in order to privately cough up a lung...and then forgot to unmute. I captured all the most important observations in my written notes, but I regret not having those initial audio reactions preserved for the level creators to hear.
Of course, my initial round of playthroughs wasn't all hardships. I actually had fun. Lots of fun. There's a reason why I'm so invested in this contest series, and a reason why I livestream Mega Man fangames: I am fascinated by the things people come up with when they set out to make a good (or deliberately mediocre, terrible, or ridiculous) Mega Man level.
I encountered plenty of difficulties. On more than one occasion, I set out to play "one more level" before taking a food or sleep break, only to discover 20 minutes later that the level was not going to end anytime soon. I'm not particularly photosensitive or subject to motion sickness, but a few levels made me queasy with their gimmicks and visual effects—not so much that I couldn't finish, but enough to make judging uncomfortable. Due to programming oversights in some of the contestants' custom objects, I triggered a handful of hard crashes and had to make up lost progress (usually before learning about noclip mode).
I had to switch my player avatar when I discovered at a most inopportune time that Roll didn't have any custom sprites created for Rush Bike, so she simply turned invisible. With the devteam making updates to the engine between batches, it wasn't always obvious which glitches were newly introduced and which should've been caught by the level creators, so I started a running list of issues to raise with the devteam so I could judge Functionality fairly. The FIRE button on my controller got progressively stickier as time went on, eventually spreading that stickiness to the JUMP button, effectively rendering the controller unusable.
Oh, and I was sick with an upper respiratory infection for almost the entire duration. I had already decided not to perform for an audience while recording my blind playthroughs—after all, these recordings were personal reference material first and public entertainment second—but I would've preferred not to sound like a plague beast when I talked. This didn't affect my playing ability, but there were a few mishaps early in the recording process where I muted my microphone in order to privately cough up a lung...and then forgot to unmute. I captured all the most important observations in my written notes, but I regret not having those initial audio reactions preserved for the level creators to hear.
Of course, my initial round of playthroughs wasn't all hardships. I actually had fun. Lots of fun. There's a reason why I'm so invested in this contest series, and a reason why I livestream Mega Man fangames: I am fascinated by the things people come up with when they set out to make a good (or deliberately mediocre, terrible, or ridiculous) Mega Man level.
I laughed at the sight gags and comedic dialogue. I marveled at the diversity of level concepts and the sheer amount of cleverness and creativity on display. I relished the sections that meaningfully put my platforming and problem-solving skills to the test. I lost myself in the immersive atmosphere created by the music and graphics. I was bemused by the zany, inexplicable design decisions and mind-bogglingly obvious playtesting oversights. I enjoyed trying to break people's levels—and sometimes, I enjoyed being denied that pleasure because the creators actually anticipated my nonsense.
Compared with any previous MaGMML, the highs were higher and the lows were much, much lower. I was proud of the levels that took to heart all the advice and feedback coming out of these contests, and I writhed in pain at the levels that clearly did not. Through it all, even when a play session wore me out, I remained excited to see what was next.
Of course, I knew what was next, in the "big picture" sense. Next was replaying, scoring, and reviewing these levels. This had been merely a warmup; the real judging was about to begin.
Compared with any previous MaGMML, the highs were higher and the lows were much, much lower. I was proud of the levels that took to heart all the advice and feedback coming out of these contests, and I writhed in pain at the levels that clearly did not. Through it all, even when a play session wore me out, I remained excited to see what was next.
Of course, I knew what was next, in the "big picture" sense. Next was replaying, scoring, and reviewing these levels. This had been merely a warmup; the real judging was about to begin.
That's it for now! Once the contest results are announced, I'll upload my blind playthrough videos to YouTube and update this post with a link.
RSS Feed
