The #1 Problem in OSS Usability and What I’m Going to Do About It
I was doing some research on a paper I am writing about open source usability practices when it hit me:
Lack of user research is the #1 problem in open source usability
Not that this wasn’t an obvious problem before. Many of us in the open source usability community knew that there is a lack of user research, and that many — if not most — projects do not have it documented, and that no user research makes designing very difficult. There are certainly a number of other problems which contribute to poor usability, but it didn’t dawn on me that this is the #1 problem until I began to compile experiences of other usability engineers from projects outside of KDE. Other projects also cite issues such as lack of management, authority, or designers, but user research was nominated as a major problem by every usability initiative I researched.
Most open source projects lacks three things related to user research:
- A defined vision
- A defined set of users
- A defined functionality/tasks
We don’t know who we are designing for, what they are doing, or why they are doing it. Projects have poorly defined visions which make development a day-to-day job rather than long-term investment. This information doesn’t exist in people’s heads, let alone documented somewhere it can be shared. I really find this astonishing. User research isn’t necessarily hard to do, it just requires you ask the right questions and to think a bit.
User research is the number one requirement in any user-centered design or usability project. Even if a client has not requested user research to be done, I will strongly suggest at least some primary research activities be done. If they can’t afford it or don’t want to pay for it, it is not unusual for me to do it at-risk anyway. I cannot design a product when I do not know who it is for or what they will do. Projects with minimal documented user research are painful and never end well. A small investment of time and brainpower up front will make the entire development process run so much smoother.
If we want a sustainable usability project in KDE we need documented user research
Up until now, most design activities have been small, focused efforts with personal one-on-one attention with a project. This method has been successful in targeting key applications and solving many design problems. However, this method has also failed in creating a sustainable usability effort. If a designer disappears for some reason, most of the usability efforts in the projects they were involved with slow or cease. The focused method of working together produces a lot for a single project, but does not do much to improve KDE as a whole. Segmented efforts do not create a sense of community which does not help with recruiting more designers to the project.
A few weeks ago I asked about the existance of a KDE vision document. I was quickly disappointed. KDE4 does not have a defined vision which is easily accessible by developers. There are a few documents where individuals have tried to define a vision or direction for KDE, but they are not well known, nor “blessed” by the community as being official. There is no answer to the question “Who is KDE for?”. Your knee-jerk reaction will probably be “KDE is for everybody”, but is this really the correct answer? (By the way, it is not).
Help me help you be more usable
A few weeks ago I tried to kickstart user research in the community by talking about user research for everyone. I outlined a few high level topics that every project should think about before they write a line of code. These are things such as a vision statement, list of users, use scenarios, etc. Really basic stuff. Unfortunately, it went pretty much unanswered. I guess that you either do not understand what I was talking about or you are lazy. I can attempt to explain it better if you didn’t understand, but I can’t do anything about you being lazy.
What I can do is try and make the research and documentation process a little easier. I have created a Project User Research Template on techbase.
The purpose of the user research profile template is twofold:
- To document valuable user profile data about KDE projects
- To get developers to think about users
This template is intended to help projects get started on documenting their user profile data. It is not the end-all, be-all solution to your usability problems, but a beginning step towards a more user-centric development approach. However, I’m not going to do this for you. This is for you (the developer) to include and maintain alongside your other project documentation.
I wouldn’t want to rob you or your project the valuable learning experience of thinking about the purpose of your project and how you will serve your users. Documenting user profile data isn’t hard, but it does take some time and sometimes requires thinking hard about seemingly simple questions. This does not mean I will not help you get started, answer questions, or give you resources that will help you. The template also includes a few links to useful resources on creating user profiles/personas.
Getting started (with a kick in the ass)
Next week I will begin calling out projects. If I call your project, I plan on bugging you until I see you have started working on your template. I will continue to bug you until you show me a completed project profile on techbase or your project website. I will get developers who are missing out on one-on-one time with me to bug you because I have to spend my time tracking down delinquents. Some of you might not even like me anymore by the end of this. I am prepared to dish out some tough love for the good of the project.
Please look over the Project User Research Template. Think about some of the questions it asks. Copy it and begin filling in some of the content. Once you begin working on it you will probably notice how hard it is to complete some of the sections and understand why this is important.
I suggest you do your homework. It could be you I call on next week.
seele :: Mar.18.2008 :: General, KDE/Kubuntu, Usability :: 37 Comments »
Let me know if you need help creating an “Official” document.
Let me know if you want/need some help with creating an “Official” document.
You should speak to the people at http://www.openusability.org/ They do much for OSS usability.
Carlos: Celeste is already part of the OpenUsability project. :)
Celeste: I think, more than the lack of user research, another problem that seems to be common in FOSS development is the lack of interest on this kind of “high level” activity. They’re probably more interested in the “development” part of work, coding, fixing, patching, packaging, etc., and less on the “theoretical” parts. And it’s quite easy to get lost in the thick of things. So while it is true that developers are more aware now of usability issues, integrating that into mindsets and, consequently, into the development process is probably still not there yet (KDE 4 HIG?).
I’d hate to use Kubuntu as an example, but a few months back, a series of events and comments led me and others to ask for a similar “vision/mission” document for Kubuntu. Nothing came out of it. A few weeks later, openSUSE released something of that kind. And now Xubuntu seems to be undergoing some identity crisis and needs to formulate their own. I hope the day doesn’t come that Kubuntu will fall under similar circumstances just so that it will start making its own definition.
I wish you the best this project and hope you find lots of people to help you with it :)
(and sorry for the half-rant comment)
http://brainstorm.ubuntu.com/idea/3001/
I agree !!!!
I can help. I have supported users for near on 6 years now !
I agree with Jucato, the main is the lack of interest in that kind of activity. Most projects do not have a single skilled “UI designer”. I think a talented person can achieve great results without any (expensive) user research. When you have a great UI you can try and make it better with user research.
Whoa, great idea and initiative. There’s nothing like a gentle but determined push in the direction of responsible maintainers. (Confirmed by my own painful personal experience.)
Three cheers for Celeste!
The high level thinking is a problem, and indeed is a reason why there is no documented user research. No one is taking a step back and asking these questions. Why? I don’t know. I guess they either they don’t know that they should ask these questions, don’t know which questions to ask, know how hard it will be to answer these questions, or know they won’t like the answers. One of the reasons why KDE4 doesn’t have a vision statement is because the community would never agree on one. This has been a big problem for projects such as Plasma which are tightly integrated in to the environment and effect almost every KDE user.
The problem is, you *can’t* get a good UI without the user research first. A developer has to be pretty out of touch with reality if they can’t provide a 25 word description of at least one major user group. But questions such as “Would the user you just told me about really need this function?” or “How does user x use this differently from user y” are the hard ones, and the important ones. You can get by without user research for very low-level things such as widget placement or what color something should be, because we have experience and styleguides as a crutch. It is the high level (conceptual) thinking, the stuff that really supports a certain level of usability and user experience, that can’t get done without that information.
I’m not a developer, I’m a user and a writer. Mostly I use a KDE desktop, but inevitably some non-KDE apps too. Gnome apps in particular, such as Gimp and GnuCash, don’t fit in well. Chief problem is that things that are normally single-click on KDE are double-click in GNU land, so clicking through sub-directories on Files > Save for example isn’t intuitive. Any chance that:
1. KDE and Gnome make friends and have a common system?
2. KDE develop Kimp, KnuCash etc so they work like other KDE apps?
3. KDE develop a ‘wrapper’ so that Gnu (and other) apps are somehow automagically transformed to work with KDE standards?
@above: Erm, aren’t you asking for a bit too much? I, a gnome user, do use several kde apps myself, and adjusted myself to the single-click thing just fine.
Generally though, this isn’t a surprise. Nor a surprise to the devs. Many just don’t bother about it - because for them it’s the “scratch your own itch” thing, everybody else is “either give me a patch or gtfo”. Biggest project with that philosophy is Pidgin.
I find your proposal amazingly helpful and necessary. This is the way to go. Think from the user’s point of view, think about what the everyday workflow looks like.
I believe that the plasma project and in particular Aaron’s visions with regard to redefining the appearance and the use of
the desktop and thereby our general workflow will actually be a significant improvement in the way you are hinting at. (Read e.g. his ideas about containments, see http://aseigo.blogspot.com/2007/07/desktop-zooming.html)
Once the basic ideas are implemented, application developers will become aware of the features and integrate some of the new possibilities into their apps. A good example for such features is the extender concept: http://plasma.kde.org/cms/1069
The main problem with plasma is that the ideas are to a large extent not yet properly understood or not communicated sufficiently. The plasma guys are, however, finally trying to pull together a vision statement on techbase now, afaik.
I don’t think that the lack of high level thinking is a problem with big GUI projects like KDE and Gnome.
On the contrary, the problem often is that some developers think about usability too much, even though they usually are not experts in the field (but really think they are).
These developers don’t even want any kind of studies. The only thing they want is to get their *own* “great” vision into the project that they (usually quite rightfully) consider as his or her *own* project.
This is the price we sometimes have to pay when our developers are “running free in the OSS world”. Things are quite different in the world of proprietary software where the developers are under the control of their “masters” (i.e. the company they work for and its paying customers).
So IMHO, it generally goes something like this:
- closed source devs *have to* create software for *their users*
- open source devs *want to* create software for *themselfs*
Nevertheless, I’m a big fan of OSS :)
Kick ‘em Celeste! Get them to do… whatever it is you’re trying to do. Make stuff more usable. (I generally am smart enough to figure out user interfaces without any trouble, but it would be nice if we can make them even more consistant.)
For instance, someone once mentioned that Konqueror, Ocular, and Amarok each have their own style of sidebar. I thought about it and agreed, why? They ought to combine the best features of each to make a standard “super sidebar” widget.
Celeste,
Thank you for taking the initiative on this. I’m in strong agreement on all the points that you’ve expressed, having worked in the proprietary and FOSS development arena and seeing what a solid set of usability research can do for a product. (Yes, for those that wonder, I view KDE as a PRODUCT, not a PROJECT. Mainly due to the fact that software viewed as only a project won’t go anywhere, however products have defined lifetimes, etc.) I look forward to seeing the fruits (and trials) of the research and momentum on good end-user usability in KDE. :)
Usability, like security, is more about architecture than anything else. Sure, you can try to force everyone to audit their user base, come up with a formal vision, and then implement it. But the root of the problem is that it is too easy to make software with poor usability. (just like it is too easy to write insecure software with C/C++) Are Mac developers just magically better UI designers? No! And yet almost all Mac software has far superior usability/polish than KDE/GNOME/Win apps. The reason is that Apple designed their APIs and development frameworks to encourage good design.. in fact, to make it hard to create really terrible designs. Granted, with a closed platform, it is a lot easier to achieve standardization too. (by closed I mean the native OSX libraries like Quartz, Cocoa, etc.)
The old unix philosophy is oft repeated but seldom followed. To apply it to modern software: “small components that are easily combined to do anything we need.” That’s not what we have anymore. We have huge monolithic apps and GUI frameworks that try to do everything and fail miserably at all the little details (like usability) while making it hard to fix one thing without breaking everything else. (tight coupling of apps to Qt versions for instance)
A huge paradigm shift is in order.
Usability should not be as much in the hands of application developers as the “Platform GUI Framework” developers. Just to use a commonly understood example, no app devs should be designing their own file open dialogs today. Well, you can apply that concept pretty far upward in design-land. Why should a file open dialog even be hardcoded in the app to begin with? All the app really needs is a data stream with a format it knows how to interpret. Who cares how or where it comes from. That is a platform design decision. The same can be said for a “menu” system. Maybe one platform uses ordinary drop down menubars, another uses ribbons, another uses some sort of zoomed multi-touch hierarchy display, another uses a voice command interface. All of these are valid ways to send commands to an application. They should not be hard-coded. Voila!! There goes your need to do usability studies to figure out how to organize your menus and make them most intuitive to various groups of users. All the app designer needs to do is extensively annotate the various commands. (name, isActive, multi-lingual description, parent, documentation link, user experience level, icon, vocal pronunciation, etc.) Then, let the platform framework decide how to present the final interface to the user in an aesthetic, friendly, personally customized manner. Now, take this exact same principle and creatively apply it to every other type of application widget imaginable.
Eventually, take it further. A really well designed platform will self-adapt to the user in ways individual apps could never dream. It can remember frequent commands, typical order of operations, interaction between various apps and components, etc. Before long, you no longer feel like you are using individual applications. You are manipulating a seamless information workspace. Data and functionality that you need is “just there” when and where you need it. Yes, this part is futuristic, but we’ll never get there if we refuse to start fundamentally rethinking how we design user interfaces.
You might argue, “That’s great.. but there are practical needs today and we are nowhere near ready to do what you just described.” Frankly, look at reality: Linux is never going to become a mainstream desktop OS before other platforms have already moved on to genuinely innovative ideas such as just described. If the goal of usability research is to expand to new user bases, the only way to do this is stop playing with software designed by and for geeks and power users. Leave KDE and GNOME as they are. Their target user bases are already satisfied. Trying to have a measurable effect here is like nailing jelly to a tree, as we’ve already seen. Move on to bigger and better things. If OSS has a future, I guarantee KDE and GNOME in their present form are not part of it. People like us who “get it” need to band together and blaze a new path. The rest will follow suit once it becomes clear there is a better way.
You are hot.
@OgPlease sounds like Qt to me. Guess we are in the future already.
I usually read your blogs through the planet. I find your point of view on usability very interesting, and worth the effort.
I am currently a developer for KMyMoney. Even if it is based on KDE it does not have any formal links to the KDE project. So, I would like to ask for your permission to use your research template on our project. I think it would be very useful to us since we have a complex userbase.
@OgPlease: A framework might help control low-level issues such as consistency and guidelines compliance, but it cannot address high level conceptual thinking or put users in context of tasks on its own.
@asoliverez: Absolutely, the template lives on techbase.kde.org but can be used by anyone.
Hello,
sounds pretty good. I think KDE should do for desktop usability what the iPhone did for cell phone usability.
I personally think Kickoff is going in the right direction. But it needs more consistency. It should complety work by clicking only and not hovering AND clicking. And the arrows should move with the menu items etc.
I think movement and animation can really help usabilty and it can give the GUI a very futuristic look. Just have animations everywhere and make them slow for newbs and fast for power users for example.
But what do i know .. ;) .. i see the potential in KDE though… i dont see it anywhere else.
@seele: I’m proposing a framework that goes beyond “consistency and compliance” to very complete abstraction of all UI elements and workflows.* Qt definitely doesn’t do what I’m talking about. Imagine if, say, the Kontact interface could render to multiple platforms. On each platform it might even look like a completely different application, but it would have the total look/feel/usability *of the platform* and would provide the same data access and “business logic” functionality on each platform, where feasible. (sure, it might have to scale some things back for a mythical wrist-watch computer!) Today, we hard code applications to a particular GUI framework like Qt or Gtk+. I’m talking about eliminating that coupling. I would even consider that the UI design could be done entirely with high-level markup and annotations. Then it gets rendered out to KDE or GNOME or Win32/WPF or even XHTML. Or it might render out to the iPhone SDK or Google Android or Nokia tablet or Eee PC or special audio/tactile interface for the blind or whatever custom device people dream up next. (3D Cave environment? hehe) Meanwhile, the application developer doesn’t have to do a thing. They don’t have to port or tweak for each and every device. They don’t have to try to make the app usable at small resolutions. They don’t have to worry about accessibility. That’s all up to the platform to handle.
So, to reiterate: Usability should be a platform concern, not an application concern. This is the paradigm shift. The rest is details. Yes, there is a huge amount of research needed to make this possible. Yes, I think it’s well worth the time this would take.
* It might be useful during the early stages to provide optional “render hints” that help the platform framework make the right choices with specific layout and workflow issues.
@OgPlease I think you underestimate what Qt does for each platform and comes much closer to what you purpose than anything else I have seen at this time.
I am not sure what you mean by “hardcode” for a toolkit. A toolkit such as Qt is already very abstract, true the code is not portable straight across to lets say GTK but it does have the ability to integrate Qt’s event loop. As far as “harcoding” I would say thats just plain coding, how many languages can you name that will compile against another language, or more often then not even for the same language on a different platform? ANSI C was already a step towards abstract standardization. What you are proposing imho fails for two major reasons.
1. Getting vendors to agree with an API that will be able to interpret this abstract language across all these devices is a task so large I do not personally think it will ever happen. The closest thing we will ever see is toolkits like Qt that provide an abstracted interface and hide all the platform dependencies from the programmer.
2.As with Java,Perl,Qt, etc the more abstract you get the more you are going to lose the finer controls and end up with compromises for corner cases and paradigm shifts from platform to platform.
What you purpose sound good in theory and would provide a very nice way for a programmer to just write once and it shifts as needed to a platform and framework the user is used to. But I think it has no real world value where we have for profit vendors, open source initiatives, and others who just want to do it there way. Your proposals are so far abstracted there would never be any innovation other then from the platform evolution itself, and if that is the case we would probably never see any. Much like a free market software design feeds on itself and twists and turns as it evolves often going in directions no one could predict.
I am not sure if you program or not but I think you underestimate what Qt provides and is a very large step in the direction of code once (And if done right) and UI shifts to more closely resemble the platform it is being run on.
Some other questions are what if this abstract language does not have an equal function on every platform? What if it was a mission critical functionality just not supported on the platform you are using. You would think the application is broken, junk, not as good as X app etc. What if I am trying to convey a really important schema and the platform you are on just does not lay it out (associate key points) correctly? Or it does something totally off the wall in an attempt to make the app run rather then not crash due to it’s confusion on the issue. You purpose “render hints” well why stop there we are going to need a boat load of other “hint” types and the more of these we put in the more the abstraction diminishes and we run into platform problems. This level of abstraction would lead to many errors and insane kludges in an attempt to retain the “spirit” of the app in every environment (this is a big problem even now).
I believe you give to much faith to a system to understand the intent of the programmer across many platforms and be able to offer the same quality across them all. It is hard enough as a UI designer to get all these cats herded on a single platform.
As far as removing the coupling? Exactly how do you plan for this abstraction to be compiled, run, etc?
@Seele still think you are hot.
> @Seele still think you are hot.
and I think you are socially challenged ;)
Hello Seele
I agree with you everything that you mention.
It’s very hard work to encourage all developers to KDE
Create Documentation Project.
You should know that in the engineering software is one of the hardest things, and usually left to the last by developers is the documentation.
Why?
Simple … Often seen as boring, tedious, and many do not have sufficient knowledge.
Your initiative to create a template to help developers I feel fantastic, I hope you have success.
I think the best thing is to facilitate as much this task to make it more attractive and less difficult for developers.
I think it would be a good idea to build more than a template.
A web application that through text boxes, combo box, radio button, and a little logic below help create an outline of documentation and an overall structure that guides developers.
Something like what we accomplished UML.
If you like the idea, I can help you assess, design, think, create (text, coding, making shiny graphics) and test.
PS: I am a computer technician, usability studies, software development process and an active developer.
Sorry my English. I do my best.
@muk socially challenged? Sure why not.
[…] true to my promise last week, here is this week’s project pick for beginning their user research […]
[…] true to my promise last week, here is this week’s project pick for beginning their user research […]
@future
> I think you underestimate what Qt does for each platform
Qt abstracts some lower-API details of its platforms. It does not abstract much of anything in terms of actual UI design. It is still “hardcoded” to the WIMP paradigm. (windows, icons, menus, pointers)
Also, Qt4 still does not have a strictly resolution-independent graphics engine. That in itself disqualifies it in my book. Yes, it is placating to the lowest common denominator, but that’s why we need to abstract further.
> As far as “harcoding” I would say thats just plain coding, how many languages can you name that will compile against another language
Markup languages. Too much today is still done in traditional languages which are hard to decompose/recompose. In terms of abstraction, Object Oriented design has taken us about as far as it can. We need looser coupling than it can provide. This is why XML has been such a boon. But we haven’t seen anything yet because the tools are still evolving.. Prediction: Within 10-15 years programmers will average 60% markup/annotation/wiring/rules-writing using GUI tools, 35% high-level scripting, 5% hard coding. Look at the Web. Things are only going to get more universal and more abstract. Traditional desktop software is a dying breed.
> 1. Getting vendors to agree with an API that will be able to interpret this abstract language across all these devices is a task so large I do not personally think it will ever happen.
Who said anything about an API? That’s exactly the point I’m trying to make in all this - we need to get away from coding strictly to API’s. Indeed, creating the “mother of all API’s” is not the answer. (ie. the approach taken by Sun with Java and MS with .NET.) If vendors have to agree on something large and complex, it has already failed. It’s a lot easier to agree on small, simple, elegant things that can be easily swapped out if they outlive their usefulness.
Yes, I realize it is necessary to have concrete implementations at some point, but they need to be way further down the stack than what we have today — to the point where very few programmers actually have to touch them.
> 2.As with Java,Perl,Qt, etc the more abstract you get the more you are going to lose the finer controls and end up with compromises for corner cases and paradigm shifts from platform to platform.
Abstraction is about handing off finer controls to someone else who can “zoom in” on them better than you can. As for compromises in corner cases, this is only a symptom of abstraction gone wrong and in need of refactoring. I see no inherent problems here. The solution is to do abstraction at natural interface points. Bad abstraction cuts in right in the middle and requires leakiness across the interface to make things still work.
Yes, there should be paradigm shifts between platforms. That’s exactly the point! Enough of the WIMP hegemony! There are dozens of new HCI methods that need explored! Today’s desktop apps are a huge impediment to serious progress.
> Some other questions are what if this abstract language does not have an equal function on every platform?
It’s not a language. It’s a meta-language. (think XML) It doesn’t have to be everything to every platform (and vice-versa). Yes, there is a boatload of research needed in making highly-modular software degrade gracefully. It’s not intractable however. (simple example: lynx vs. dillo vs. mozilla) I agree the “render hints” idea would be bad.. leaky abstraction is treacherous. I only suggested it as a temporary measure that would be phased out ASAP.
> Your proposals are so far abstracted there would never be any innovation other then from the platform evolution itself, and if that is the case we would probably never see any. Much like a free market software design feeds on itself and twists and turns as it evolves often going in directions no one could predict.
I disagree. The exact opposite would occur. Think outside the box of today’s “stand-alone applications” that are procured and installed as big packages. Instead, innovation would generally occur on a smaller scale — component level. But it would be purer innovation, not 95% reinventing a competitor’s product, 5% new features to attempt to set your product apart. (ex. how many email and IM clients do we need?!?) If the “platform” was universal and software components were interchangeable, you would have an ideal free market and thus a huge incentive for innovation. The barriers to entry in such a market would be incredibly low. People wouldn’t have to design huge “kitchen-sink” applications from the ground up, just a useful widget that lots of people want. Yes, this is futuristic, but we’re talking long range to begin with, given statements like “someday” vs. “it’ll never happen.”
120% right
I’m mostly a user but I’m currently trying to get into open source development.
about High Level Thinking: yes, that’s actually a very big problem. I attended a speach in FrOSCON (a german OSS Con) last year where KDE developers discussed about the issue of open source marketing. It was very obvious that many developers are more interessted in coding than spreading their work. Altough this is already a higher level than usability it shows how this is a problem: developers are developers (coders). they don’t want to make marketing, usability research and some even don’t like documenting their work (=not fun).
But for the open source movement in general developers have to face these issues if they want their work to become popular.
Hi, I’m not sure if is “usability testing” what you are talking about, but I get surprised when I saw the data in the http://www.betterdesktop.org/ project.
@Carlos: No, this is not usability testing. Usability testing happens later in the development process. User research happens in the requirements stage. It is a pre-requirement for any design and development which leads to usability testing.
@OgPlease I like you rebuttals. Rather then just dragging back and fourth though, I will just say Qt4.4 does move towards resolution independence.
We are starting to see some paradigm shifts, just how much and how long I guess we will see in time.
This is a huge step in the direction of what I’m talking about in terms of different architecture as the key to better usability:
http://code.google.com/android/devel/building-blocks.html
QUOTED: “You can think of an Android application as a collection of components, of various kinds. These components are for the most part quite loosely coupled, to the degree where you can accurately describe them as a federation of components rather than a single cohesive application…”
My idea would go a little further by adding additional abstraction to the components they call “Views” and “Notifications” such that they not necessarily be GUI elements but rather generic user input methods that get translated into GUI elements at the platform level if this is appropriate.
[…] I have also been browsing through the history of posts on Planet KDE and I found a post about lack of user research by Celeste Paul. She also put together a user research template which I will try to fill out as […]
[…] Celeste started the target user research project only a couple of weeks ago, the first results were presented to […]
[…] there: The #1 Problem in OSS Usability and What I’m Going to Do About It, which Antti Kaijanmäki, another Kesäkoder, told me about some weeks […]