SkillCraft.ca: A SC2 Expertise Study

SkillCraft.ca: A SC2 Expertise Study

We are researchers at the Cognitive Science Lab (Simon Fraser University in Vancouver, Canada) and are conducting a large-scale study on the development of expertise in StarCraft 2. We believe that the analysis of StarCraft 2 performance has the potential to be very valuable to researchers studying attention, learning, and cognition. Our initial project involves using replay files and survey questions to characterize differences across all skill levels, from bronze to professional. With the community’s support, it could be the largest expertise study ever conducted and may eventually generate positive press coverage for e-Sports and the SC2 community.

Our biggest challenge at this point will be to get a wide variety of players to participate, and to get a lot of them. The size of the study will be one of the big selling points with the academic community so more is definitely better. There are 2 simple ways you can help.

1. Complete the survey, and upload a replay file yourself.
2. Encourage players you know to participate, especially newer players who might not be active in the online community.

If we can get 20,000 players across the full range of skill levels to participate, it will be, by several orders of magnitude, the largest expertise study ever conducted in any domain; bigger than any study on traditional games such as chess, or any sport expertise studies.

We hope that you find our project as exciting and interesting as we do. Your willingness to participate and promote our project is much appreciated.

Participate at: http://skillcraft.ca/

EDIT: PLEASE DON'T FORGET TO UPLOAD YOUR REPLAY! We are noticing that a lot of people do the survey but don't upload the replay. We are studying screen movements, hotkey use, APM and other factors that we can only get from the replay data, so we really do need the replay. We have about 1000 replays so far. Keep them coming and spread the word. Thank you for your help!

EDIT2: Reddit AMA (Ask Me Anything) link is here

EDIT3: I did a one-hour interview with InfestedMrT from rCraftGaming. It's on their twitch.tv page here: http://www.twitch.tv/rcraftgaming/b/293105199

Latest Data:
[image loading]

Last edit: 2011-09-06 17:19:49

SkillCraft Status: Reviews were good, making some revisions

ribboo Sweden. August 12 2011 22:46. Posts 1840

Profile #

Done, done. Wasn't very many questions. Took 5 minutes.

Last edit: 2011-08-12 22:46:54

CrushDog5 Canada. August 12 2011 22:55. Posts 180

Profile #

On August 12 2011 22:46 ribboo wrote:
Done, done. Wasn't very many questions. Took 5 minutes.

We wanted to make it pretty painless. Thanks for the bump.

I'll keep an eye on the thread; if people have questions I can answer them.

SkillCraft Status: Reviews were good, making some revisions

BushidoSnipr United States. August 12 2011 23:01. Posts 910

Profile #

awesome-just took the survey and looking fwd to results

Krallman Sweden. August 12 2011 23:08. Posts 653

Profile #

done

Im better than Stefan

Anachromy United States. August 12 2011 23:10. Posts 168

Profile Blog #

Just submitted. good luck with your study!

Last edit: 2011-08-12 23:11:16

Production Director for the NESC2/Semi-Pro Gaming League

enecateReAP United Kingdom. August 12 2011 23:18. Posts 377

Profile #

Done, good luck with your research, I'll be interested to see your findings.

"Stargate units imba" - oGsMC

Kerm France. August 12 2011 23:21. Posts 370

Profile #

Looks like an interesting project. Will be answering when I get home.

BTW, be sure to bounce back the study's results over here

Last edit: 2011-08-12 23:22:42

What i know is that I know nothing - [http://twitter.com/UncleKerm]

sebsejr August 12 2011 23:21. Posts 209

Profile Blog #

Done ! GL !

galtdunn United States. August 12 2011 23:27. Posts 799

Profile #

Did it

Last edit: 2011-08-13 13:36:03

Currently editing items in the DotA 2 wiki. PM for questions/suggestions.

Pharaun Germany. August 12 2011 23:28. Posts 131

Profile #

done...dunno what u have planed for the replays as i just upload a random 1v1 replay.

I would like if u can tell us where /when to see any results of this.

It has to start somewhere. - It has to start sometime. - What better place than here. - What better time than now?!!!! -All hell can't stop us now

Equity213 Canada. August 12 2011 23:31. Posts 867

Profile #

Hey this is really interesting, I will do it asap. Cant wait to see the findings when its done.

p0lyph0ny United States. August 12 2011 23:34. Posts 216

Profile #

seems legit

Rodiel3 France. August 12 2011 23:36. Posts 1061

Profile #

Gl hf with ur project

http://www.youtube.com/user/rodiel3 SCBW FPVOD

Beyonder

Netherlands. August 12 2011 23:40. Posts 13523

Profile Blog #

Ohh cool, good luck! Oh man, you are stealing my ideas T_T need to get a phd faster!

Micket United Kingdom. August 12 2011 23:48. Posts 2163

Profile #

If I upload a replay of me 1-1-1 ing Protoss and winning easily, how will it be used to determine things?

CrushDog5 Canada. August 12 2011 23:51. Posts 180

Profile #

[B]On August 12 2011 23:28 Pharaun wrote:[/B
I would like if u can tell us where /when to see any results of this.

We will post results as we get them. Our initial plan is to collect data for about 3 weeks (though it will depend on the response), and the data analysis will take some time. Getting demographic data processed will be easy, though, so we can post up those results not too long after we close the survey.

We also have lots of interesting follow up studies planned. We mainly study attentional learning (how people learn what to pay attention to and when), so doing some eye-tracking studies (we have four trackers) is part of the plan. Ultimately I'd like to go to MLG with some eye-trackers. We are also planning on collecting replays from individuals over a long period of time, say three years, to see what the transition from bronze to diamond looks actually looks like. There are also interesting questions around joint attention and communication in team games.

If our initial attempt can show some interesting results, it will open the doors some research grants and to many interesting followups.

SkillCraft Status: Reviews were good, making some revisions

WoundedKite United States. August 12 2011 23:51. Posts 17

Profile #

Done, painless

glgl!

LunaSaint United Kingdom. August 12 2011 23:53. Posts 601

Profile Blog #

Was half way through this when I realised I hadn't laddered in like a month.

I really need to get back in the habit D: