Should we be harsh about each innovative research paper (in order to be in CHI) must have a strong (and safe) usability study?
Wait a second, if the answer is yes – why? why doing usability study in the first place?
Imagine parallel universe is real and in the other world, the CHI community even have no idea what ‘usability study’ is. What differences would there be? Is that much a big difference? (here starts the endless argument…)
Let alone ‘usability study’ and back up to the design itself, in my shallow experience in HCI, I humbly think it is very important to generate new ideas. And what is really difficult after that is being able to say, this is actually a good idea. How can one prove and validate his/her ideas? This is much more interesting and worth more effort into it, compared to some of the usability study that just tries to ‘fit’ in the innovate stuff.
Therefore I believe usability study is just one of the many ways toward questioning, proving and improving the innovative design. That means, sometimes we might not need it at all, simply because it is not applicable or there are better ways to do it. However, what is really important, is not the how, but the what – what is the problem – and the why – the motivation and direction of such attempts to ‘gauge’ it.