Figure 12.1. 

Mara Hancock serves as Associate Director for Educational Technology Services at UC Berkeley, and oversees the Learning Systems Group (LSG). She manages an extremely talented team of educational technologists, software programmers and architects, User Experience Designers, and training and support folks. We work with UC Berkeley faculty, students, and staff, as well as other educational technology professionals around the world to develop, adopt, and support collaboration and learning systems to enhance the teaching and learning experience.

Wednesday, July 11th, 2007 by Mara Hancock

Open source software has moved up the technology stack. We are now seeing consumer software such as Content Management, Learning Management, Portals, and other Web 2.0 tools all emerging directly out of open and community source efforts which provide unique opportunities for higher education to address the unique needs of their academic constituencies. But do we have what it takes to do this successfully? Do we have the right skills in our development shops? Can we bridge the divide created by distributed development teams to make for meaningful and seamless applications that will meet the work flows of all our users? How does the fact that we are in a teaching and learning environment impact that work and the methods we apply? This blog entry may be destined to ask more questions than I can answer! I hope that at the very least it might help to instigate a healthy dialog, illicit some emerging best practices from open and community source communities and from our — often less visible — local environments.

Let me enter into this conversation by way of a brief introduction to the work that has influenced my thinking in this area.

UC Berkeley has been actively working on Sakai since early 2005, when it was solely a grant and University funded project. We continue to be actively involved as it transitions to a full-fledged open source foundation model. I have been on the Sakai board of directors through this time. We are also a core member on the Fluid Project, recently funded by the Mellon Foundation, along with the University of Toronto (PI), Cambridge University, York University, UBC, and experts in usability, accessibility, and UI design across the globe. This community source project was created to focus on addressing the precarious value of UI and accessibility design in community and open source development work. In addition, UC Berkeley will be a core partner on the upcoming Kuali Student Project, which, as a project, has boldly declared a determination to be user-centered from day one. As you can see, as an organization UC Berkeley is deeply embedded in – and increasingly reliant on – open source applications, and in particular the community source projects, to deliver critical and integral functionality to our student and instructors every day. If these users – the heart and soul of our university’s endeavors – cannot use these tools to successfully fulfill their goals we are not doing our jobs.

Most of the findings in this entry are from personal reflection from my experience with the above community source projects, talking with colleagues involved in a variety of open source projects, and blogs and writing from across the web.

When I first heard my fellow Sakai board member, CIO Brad Wheeler from Indiana University, refer to “user delight” as a strategic goal, I was slightly uncomfortable. The term “usability” is so much more utilitarian and sets a nice solid, non-evocative baseline. Don’t get me wrong, I want the BEST user experience possible, but “delight?” So I ask you, why not “user delight?” In fact, shouldn’t usable software simply be the bottom line? If we are going to be in the software development business, shouldn’t we be aiming to, at the very least, satisfy, and even better, create an experience that is welcomed — even sought after! Wouldn’t that be success?! In fact, when I step beyond my prudishness and my fear of failure, I do agree with Brad. Community and open source communities, where higher ed IT shops are striving to create superior software “by academia and for academia” are ideally positioned (at least theoretically) to achieve user delight. However, in order to do this we need to carefully examine the skills and resources and sometimes-unusual alliances that may be required to be successful in achieving this goal.

To begin, let’s be clear, poor usability in software applications is not relegated solely to the domain of open source. Many a commercial product has been slotted for demise – often prior to launch — because of poor usability. Indeed, as evidenced in many a UI listserv, UI design faces challenges in communicating its value across the spectrum of workplaces (spend a day or so on the IXDA list to observe this). Clearly usability problems are not the sole reason for what is reportedly an over 70% failure rate of software projects. But I would hazard to guess that if you are willing to broadly define usability as “a useful and satisfied user experience (UX),” and not just solely issues related to interface design, that a large portion of these failures are likely to indeed be tracked back to usability. While many of the symptoms experienced by commercial and open source development teams are similar, I expect that the solutions applied will often, and necessarily, be different in order to accommodate the cultural and organizational differences between the environments, as reflected in Eric S. Raymond’s “The Cathedral and the Bazaar.”

I have attempted to outline some of the challenges to the development of a delightful user experience in OSS and Community Source products from the perspective of those projects coming out of higher education for higher education. Many of these issues are interlacing and multi-layered and I don’t expect to create an all encompassing list, but to at least capture a general survey of some of the salient points.

One of the huge benefits of developing open source products is that development can happen anywhere — and hopefully it does! In order to enable these distributed development teams to deliver in a timely manner, it is often necessary to create frameworks that allow the creation an implementation of loosely coupled tools. From many perspectives, this is a good thing to do: organizationally it allows open source teams to work efficiently (eliminate the coordination costs), and architecturally it provides much greater flexibility.

However, distributed teams introduce several UX challenges. Requirements developed in the silo of a remote team tend to focus on the requirements and business rules as expressed in that environment. For example, UC Berkeley might tend toward defining the business rules for the Gradebook based upon our campus policies rather than doing the extra work required to generalize across a wide range of institutions and global cultures. This behavior makes good local sense since as institutions we are driven by enlightened self-interest and need to ensure that we meet the needs of our local users with our local resources. However, producing a tool that only creates interactions based on the primacy of UC Berkeley’s business rules often effectively lowers the ability for other schools to leverage the tools and increases the total cost of adoption.

Another UX challenge is that working in tool silos makes it difficult to create a coherent, “holistic” environment for the end-user. Many user goals are based on work flows that cut across tool sets. This has been an oft-cited usability problem within Sakai. Users don’t think within the same categories and silos as the development teams work.

Open source software has historically been developed for and by developers. It is a meritocracy where individuals gain respect through their direct contributions to the end product. This creates an intrinsic reward system for the developers whereby respect and privileges are accorded to those who do things like “play well with others,” provide good feedback and assistance, but most importantly contribute good, solid, workable code.

UI Designers generally don’t produce code. UI Designer Rashmi Sinha talks about this issue in her blog,

While quite successful for projects such as Linux and Apache, this is problematic for end-user applications that are used by the faculty and students in higher ed to support their daily scholarly, teaching, and learning activities. Developers can no longer design for themselves; they have to design for users whose goals are nothing like their own (a good read on this is Alan Cooper’s book, “The Inmates are running the Asylum“). Developers need UI Designers and Instructional designers to help them translate instructional, scholarly goals into specifications and prototypes. However, in an environment where code is king, what rewards are available for individuals with these other critical skills to participate? Do we even have the right ecosystem in which for them to engage them in the first place?

As IT managers, we are probably the first to advocate for the right tool for the right job. However, we continually seem to hire a relative monoculture of IT professionals, thinking that if we just add another programmer all our problems will be solved! After talking with many IT managers across higher ed, it appears that UI design (whether it be User Research, Interaction Design, Visual Design, or Information Architecture) is rarely a formal part of their cycle or designers a regular part of the team. If UI Designers are part of the team, they are often so sparse a resource as to absolutely ensure that they won’t have enough time to get engaged early enough or long enough. This means that the few teams that are able to contribute UI designers to an open source effort, have a hard time being impactful. This is made worse by the fact that designers are often embedded in distributed teams and not looking across the product, inhibiting a holistic user-centered approach.

This inevitably creates a gap between expectations and deliverables and creates a tension that is exacerbated by the lack of recognition for UI deliverables that arrive unaccompanied by code.

Another challenge in creating applications for academia is that many of the user goals are embedded in pedagogical methods that may be discipline specific or not expressed in a generalizable way. Instructional designers and faculty are rarely part of a development team. In the higher education community source environment we have an opportunity to remedy this. It may require reaching across local organizational divides to ensure that the user and instructional goals are adequately being met: Often, instructors don’t speak the language of technology, so the instructional designer can help translate, generalize, and communicate their needs. In turn, the instructional designer often doesn’t speak the language of the application programmer, and the UI designer can help translate and represent their needs within the design and work flow of the application for the developers. This diagram attempts to express the relationship between these different areas of expertise.

Figure 12.2. 

The transparency of open source projects in higher education helps development and instructional support teams engage faculty and students in the process of creating the online environment that they need. We are uniquely situated smack dab in the middle of our own usability lab. There are few commercial or open source environments that can count themselves as this lucky. One of the biggest barriers to implementing a user centered design process that I have heard from UI Designers working in the private sector is their inability to gain consistent access to their users. Let’s make the most of our opportunity!

One of the largest benefits of open source software can also be a sizable UX challenge. The ability to easily localize and change the code means that often development teams and users don’t have a common or consistent experience and it is difficult to conduct user testing. On the one hand, proprietary products with closed code won’t let us make the experience more meaningful to our users, but on the other, with open source we have the unique opportunity to make a mess of it! An instance of uPortal at UBC may be completely different from uPortal at Yale. So how do we conduct usability tests? This issue is something that the Fluid Project is exploring now as it prepares for its first round of “user experience walk-throughs” on Sakai, Moodle, and uPortal. They have designed a set of protocols that are already being utilized by other community source projects.

Users of Open Source software are not only the end-users, they are also the designers, administrators, and implementation teams (hence documentation is also a huge barrier in OSS). When designing open source applications or platforms, making the software usable for these users is also important. In the case if UI designers, choosing presentation layer frameworks which are compatible with standard mark-up languages is important. The Fluid Project is working across projects, attempting to identify user interactions that cross academic software and develop accessible open source UI components that will be mapped to design patterns (http://developer.yahoo.com/ypatterns/, http://designinginterfaces.com). For site administrators and integration teams, documentation and set-up wizards will be key. Design patterns for these activities should be able to also be mapped to reusable components and documentation templates.

While these issues only represent a subset of the details surrounding the challenges to creating delightful academic software, I think they highlight some of the opportunity as well. I am optimistic that through the technical, UI, and advocacy work of the Fluid Project and participating community and open source projects, we will be able to impact change both institutionally and within the open source organizations. I expect and hope that through forums such as Terra Incognita there will be more occasions for those of us in IT management, those engaged in supporting the teaching and learning endeavor, UI design, and programming across our campuses to find ways to bridge the divides both organizationally and culturally, and to collaborate in creating user-delightful open source software. You can find links to a number of related articles on UI design and open source usability on my del.icio.us site, tagged as “OSUI” (Open Source UI). Happy reading!