The very goal of COST294-MAUSE is to bring usability evaluation methods (UEMs) to maturity.
How we define "maturity" is yet an open issue. It seems we know what it means. However, when we come to assess how mature a specific UEM is, there is no explicit set of criteria to do so. To extend the question beyond the boundary of UEMs, how mature usability and user experience as a conceptual as well as practical notion is remains unclear, because we do not yet have an unanimous yardstick to assess its maturity.
Indeed, the integration efforts in the remaining lifetime of MAUSE is to consolidate our findings and views on assessing how far UEMs and usability has reached maturity and what we as well as the HCI community at large to bring them closer to maturity.
First and foremost, we need to have a common understanding about maturity.
Let's share our understanding of it!
Cheers,
Effie
skip to main |
skip to sidebar
Posts
3 comments:
The Oxford Dictionary defines the word "maturity" as "deliberateness of action, mature consideration, due deliberation, fullness or perfection of natural development, ripeness, due promptness, the state of being complete, perfect, ready," and so on.
Usability maturity is a measurement of the level of assurance that an organization can develop an artifact with effectiveness, efficiency and satisfaction (i.e., providing usability from the process). That is, Usability Maturity is an expression of an organisation’s ability to deliver expected results with usability.
An intersting challenge, to discuss a common yardstick for the maturity of UEMs (usability evaluation methods). I agree with fmontero that organizational uptake is relevant, however other important aspects of UEM maturity have also been discussed in the MAUSE project. Personally, I believe that the following aspects are critical:
(1) Validity and thoroughness (the challenge being to decide on a common criterion)
(2) Downstream utility
(3) Uptake in industry (the degree to which a given UEM actually is implemented in industrial practice)
(4) Common practices and sources of documentation (the existence of commonly agreed on procedures/handbooks for a given UEM).
User testing seem to be a highly "mature" UEM in the sense that it is commonly implemented in industry practices and thoroughly described in handbooks. User testing also is often used as a criterion against which to evaluate the validity and thoroughness of other methods. Is it reasonable to say that user testing is the most mature UEM we have?
Process maturity in an organizational context is well defined by the maturity stages in the Capability Maturity Model (CMM-I):
0 Initial - everybody has this
1 Managed - systematic planning, repeatability
2 Defined - standardized and continuous improvement
3 Quantitatively Managed - statistical process control
4 Optimizing - improvement based on quantitative control
Well, how does this translate to usability evaluation? I think, we have to distinguish first, what
can be done because enough experience and know-how is there and what is really common practice
A few examples:
Systematic planning is possible due to numerous practitioner handbooks on UT etc. Most professionals probably do it in a way.
Repeatability - there are some concerns based on the results of CUE-4. At least between organizations there is large variability in practice and results
Quantitative control - There are some models for statistical control but these are too simple to reflect what's truly happening in the wild and thus are unreliable.
Post a Comment