Comments on: If I’d only known…

By: David Brake

David Brake — Sat, 09 Jun 2007 12:26:46 +0000

OK this is really obvious but when doing interviews make sure to do pilots early – preferably before you start formulating research questions. Even if you think you know your interviewees’ subject well don’t assume you will start by asking the right questions.

By: eszter

eszter — Fri, 08 Jun 2007 13:02:47 +0000

Thanks, good points.

And yes, when I pretest students’ surveys and tools (or my own), I try to do unexpected things precisely to see how unanticipated actions are handled.

By: Andrea

Andrea — Fri, 08 Jun 2007 12:50:18 +0000

When doing an online survey, don’t just pilot the instrument. Pretend you’re in a QA lab for a software company, and try to break it. This would have saved me from using the wrong question type on a survey.

Also, a general recommendation from dealing with hand-mined social network data for my thesis is to consider your intended analysis (skills, techniques, methods, etc) when determining how data is recorded. I saved myself a lot of misery by collecting data in the rawest possible flat-file format, because I already knew that I would end up using Perl scripts on it to turn it into a network input file (or two) but that I would also analyze the raw data directly.

I was also very lucky to have a mentor who was willing to invest the time to get me going with a number of the tools I needed to do the research–not just Perl scripting, but also LaTeX, R, GUESS, GraphViz, and quite a bit more. The learning curve just to get yourself started on data collection and analysis can be significant, but the experience has really improved my computing skills much more than I could have foreseen, and I wasn’t any slacker to begin with!

By: eszter

eszter — Fri, 08 Jun 2007 01:41:35 +0000

Very good comments, Noor, thanks! You make several helpful points. Much of it relates to the issue of _everything_ always taking longer than you anticipate. But here we also have issues of depending on others (whether IRB staff or Web sites being accessible). I’ll definitely incorporate these pointers, thanks!

By: Noor

Noor — Fri, 08 Jun 2007 00:40:49 +0000

I’m not sure if this is related to your chapter but when I was working on my thesis (which involved sampling bloggers), dealing with the IRB consumed A LOT of my time (which we didn’t anticipate). It doesn’t seem like IRBs understand the nature of public/private information on the Internet or Internet research. My small scale harmless study of public data turned into an inquisition (and even required full board review).

The network analysis part of my thesis required a lot of work to be done with free/shareware software, most of which wasn’t very user-friendly or well-documented. I also ended up writing a lot of ASP code – for data mining (from Technorati) and cleaning up the mined data. And things didn’t always go smoothly – the scripts often took hours to execute, failed, couldn’t connect to Technorati, databases got too big and slow, etc.

Had I known the amount hacking around I’d have to do to get it all done, I probably would have picked a different project/research questions (I tend to be more qualitative than quantitative). In general, there is a certain amount of tacit knowledge that isn’t very well documented with some of these methods (necessary steps are assumed or simply omitted). I was lucky that I was being mentored by someone who was an expert in the field and could steer me in the right direction when I got stuck.

Hope this helps — otherwise thanks for reading my venting!