Chronicles – An AAC Honours Project

Let me tell you about my Honours Project, I’ve been working on it for six months now and have just had a proposal for a paper I wrote about it accepted for the ISAAC 2012 Conference in Pittsburgh so it seems silly that I haven’t written a blog post about it yet.

In my final year of University, I find myself working on an honours project that is both rewarding academically but also, through the enormous sense of satisfaction of helping people with Severe Speech and Physical Impairment tell stories where doing so has been frustratingly difficult in the past.

My project is an AAC project.  That stands for Augmentative and Alternative Communication.  AAC devices help those with Severe Speech and Physical Impairment (SSPI) communicate.  People with SSPI vary greatly, while one user may have permanently or temporarily non-functional speech only, others may find themselves with much more physical impairments.  They could be confined to a wheel chair much of their lives, or simply have difficulties with motor control, meaning they find it difficult to operate a mouse or keyboard efficiently.

British Singer Adele temporarily found herself using an AAC app on her phone to speak after surgery.

A recent example of someone who found themselves without functional speech recently would be the British singer Adele who after surgery, was ordered by doctors to rest her voice, she downloaded an app on her phone to speak for her.  However, most regular AAC users are more permanently affected by non functional speech and often posses severe disabilities that hinder their use of a computer.

Adele apparently attempted to download several text to speech apps for her phone and finally settled on one particular app that would allow her to swear.  This is actually something that we share a belief in at Dundee University.  AAC devices are a user’s voice, under no circumstances should we censor them, even if they are designed for children.  You wouldn’t teach a child not to swear by cutting into their brain and removing the ability to do so.  Perhaps in some kind of cross between Orwell and Asimov, this might make an interesting story but unfortunately, many AAC devices do indeed censor their users.

Stephen Hawking is probably the most famous AAC user.

Probably the most famous user of AAC equipment is Professor Stephen Hawking.  Hawking is a perfect example of someone with Severe Speech and Physical Impairment.  With very little motor function, Hawking uses a computer attached to his wheelchair which he operates using a single switch that he presses using his cheek.

As so many AAC users do fall into this category of Severe Speech and Physical Impairment (SSPI), interface design becomes a huge factor as users with SSPI often require interfaces designed specifically for how they use a computer.  For instance, users who, like Hawking, use a switch to select items on a screen will usually require what we call a scanning interface.  This is where the system highlights a selectable element of a screen individually for a moment before then moving on to highlight the next element, after waiting for the desired item to become highlighted on their screen, the user can then hit their switch to select it.  As you can imagine, this can be slow and tedious however, it is of course more accessible than not being able to use the interface at all.  As such, systems need to be designed that respect how the user will be using their device.  Other users may not need a switch interface and while they can not necessarily operate several small buttons, can in fact use a touch screen with large, easily understood buttons to choose from.

Another factor that needs to be remembered is that disabilities are in most cases combined with other disabilities.  For instance, someone with SSPI is very likely to also have learning difficulties such as dyslexia.  One issue that arises regularly in the AAC user community is illiteracy.  As such, basic text to speech is often not the best method for an AAC device as if the user can not read or write, they can not type text to be spoken.

Chronicles – My Honours Project

Chronicles - My Honours Project

For my honours project, I’ve joined the AAC Research Team at Dundee University where I’ve joined in the development of a new project called Chronicles, a narrative telling system for adults with SSPI.

While there has been lots of research and development (R&D) in AAC systems, most of these have been in aiding transactional communication.  When we say transactional, we are referring to communication aimed at expressing needs, this could be as simple as “I want coffee” or “I need the toilet.”  However, in terms of narrative systems, there are less systems.  Narratives is an AAC term that really just means stories.  While this may seem like a simple matter, story telling is actually a major part of communication and you may not realise just how many stories you tell every day.

I've joined the AAC Research Team at Dundee University

When you speak to your friends, you may tell them about what you did last night.  You may tell your mother about a hard day at work or you may tell people about a vacation you went on.  With close friends and family, people with SSPI are usually more comfortable telling stories, possibly because those who are close to them have less difficulties understanding gestures or the broken speech that some with SSPI are capable of.  However, when meeting a new person, these stories are more difficult for users to tell and so they come to rely on AAC devices much more.

However, the other main part of communication is conversation.  When you tell a story, you do not just tell the whole story all at once and take questions at the end of it, you say a sentence and you stop as the person you are telling the story to asks a question.

I had a nasty experience at the shops the other day.

Oh yes? What happened?

There was a robbery at the store!

Really?! Tell me more!

When we tell stories, interruptions and questions are a major part of the narrative and so a story telling system has to allow for the story to be told in individual utterances, allowing questions to be asked.  This also allows the user to skip parts or tell the story in a different order to how it was written, allowing the story to be told in a different way each time to different people.

With most AAC systems, the user can enter a narrative in one long block of text that can’t be interrupted or have the sequence changed to allow the user to respond to questions.  As such, users with SSPI often don’t tend to tell stories with AAC devices and would choose to have a communication partner tell the story for them instead.  This may be a close friend or family member or a support worker.  However, when faced with a situation where they wish to tell a story, if the user’s communication partner is not present or indeed, if the communication partner with them does not know the story, many choose simply not to tell the story at all.

Chronicles extends the work of "How Was School Today...?" to adults with SSPI.

Chronicles aims to change this by allowing the user to generate their stories and store them in a system that allows them to be told as part of a conversation naturally and be kept with them on their own AAC device wherever they go.  This extends the work already done here at Dundee University’s by AAC Research Team members Rolf Black and Annalu Waller on a system called “How Was School Today…?” which generated stories for children based on what they had been doing in school, allowing them to tell stories to their parents and friends.

We hope to bring similar functionality found in “How Was School Today…?” to Chronicles, such as Natural Language Generation implementing data to text and text to speech. However, we plan to extend these proven applications of narrative telling to a device that can document all of a users life.  And thus we arrive at the part I’m working on, how do you design an interface for those with Severe Speech and Physical Impairment that allows the user to easily find one story, from what could be hundreds!

Retrieving One Narrative From Hundreds

The work that I’m doing on Chronicles is to investigate how best the narratives we’re collecting/generating are to be stored in a database on the user’s system and most importantly, how the user interface should be designed to allow a person with severe physical impairment, learning difficulties, and possibly illiteracy, to easily retrieve a single narrative from a feasibly very large number.

This has been my challenge for the last six months and through evaluation from my initial prototype designs, it is the challenge that will continue for the final few months of my time at university.

One of the highest held principles at Dundee’s School of Computing is User Centred Design.  For many, this simply means designing a system with the user in mind, for us, it means that the user group you are designing for is involved in the actual design process.  For the AAC Research Group at Dundee University, that means real AAC users with SSPI coming into the lab on a regular basis to give insight and thought into systems that we are designing.  This may seem entirely obvious to you as a reader however the sad truth (tragic even) is that almost all AAC development in the world is done by researchers who have never spoken to a real AAC user in their lives!

At the start of this project, myself and my supervisor had several assumptions as to what kind of system Chronicles would be and more to the point, how its interface would appear and be used, however through speaking to real AAC users, we could see just how much interaction with our user group is invaluable.  It is such an important factor that my supervisor, Dr. Suzanne Prior, wrote her PhD on the subject of user involvement in the development process of AAC systems.

A paper by myself has been accepted for the ISAAC 2012 conference in Pittsburgh.

We believed that a category system would be much more useful for finding stories than a timeline interface that would show stories in a line.  We had in fact resigned ourselves to our belief so much that we were quite sure that such a system would not be developed at all.  However, it turned out that when discussing their progress, our user group is often asked to to plot events in their lives on a time line drawn as a long winding road.  This visual aid was actually much easier for them to picture events that have happened in their past and so, Chronicle’s Timemap was born.

Notice that I haven’t just come up with an interface based on what we believed would be a good idea, if I had, the system would look extremely different to what it does and after receiving poor results from evaluation, a prototype would likely have needed scrapped and a new one made up, by involving users in the requirements gathering process right from the start, time is not wasted building systems that just aren’t suitable.

Evaluation of the initial prototypes, as designed based on the expert knowledge of real AAC users, has shown that the system is being received well.  There are elements that through watching members of our own user group interact with, I now know need changes.  Elements that I would not have thought of if I hadn’t involved them in the testing process.

I’m now on to making adjustments based on really useful feedback and in a few weeks time, I will be giving each member of the AAC User Group that I’ve been working with a copy of this software to take home and test in a longitudinal study and with the current search engine that I’m implementing in the system, I’m looking forward to hearing how my efforts have been received.  Even negative feedback is positive to the AAC Research Group as it teaches us more and more on how to design systems for users with Severe Speech and Physical Disability.

Dundee Radio Android Apps

The Discover Radio App is available in the Android Market Place

One thing that I find irritating about internet radio, is the fact that it is well designed to be listened to on your computer, but not on your phone.  Smartphones actually play internet radio very well, the issue is that for most, it is a little tricky to first of all, find a media player that can play streaming audio but then, it’s even more tricky to find out the internet radio address for the station you need.  These addresses are rarely publicised visually and require digging through website or playlist code to track down.

The station that I was having issues with was Dundee University’s own Discover Radio.  After tracking down the address for the radio stream, I found it tricky to find an easy to use player on Android that I could just feed this address.  This seemed odd to me as the Android MediaPlayer class can play these streams natively.

So I decided to make an Android App to do this for me.  The Discover Radio App is now available to download from the Android Market Place.

A few tweaks of the code and the Strathclyde Fusion app was made available too.

With a friend of mine being a DJ at Strathclyde University’s Strathclyde Fusion, I’d already become irritated by the lack of web player on their website, the link provided to listen to their station only produced a windows media player playlist which, when I’m working on my Mac, is not natively supported.  So with a few tweaks to the code, out came the Strathclyde Fusion app too.

Now Discover Radio, being a student radio station, has a habit of going off air, mainly at night or on Sundays when there aren’t enough students on campus to keep it running.  With this in mind, I wanted to listen to my other local radio stations, TayFM, Wave102 and TayAM.

The Dundee Radio App lets you listen to all four of Dundee's radio stations.

Once again, no apps existed for these stations and so I compiled all of my efforts into one single app, Dundee Radio for Android will allow you to listen to all four radio stations native to the fair city of Dundee.

Damn you Valentines Day

In recent years, I have developed quite the gift for destroying a relationship shortly before Valentines Day, as such, you can imagine my feelings when I woke up this morning. In the past, I can happily claim the joy I’ve taken in indulging in this, the most romantic …revolting of holidays.

I’ve always taken the time to prepare carefully for Valentines Day, custom hand picked bouquets of flowers, gifts, beds covered in rose petals, romantic meals, perhaps made by myself, perhaps at a restaurant. Well, with this being another year where I appear single on this day I had resigned myself with the statement, “Fuck girls! Fuck them all!” (Ironically, this also remains one of my life goals).

As you can imagine, I prepared myself for the day this morning in depression and preemptive disgust at the romantic events I would no doubt witness throughout the day. I showered and I dressed and put loud music in my ear and opened my door, my path was blocked!

I observed in front of me something I instantly recognised, a box. No not just any box but a delivery box for flowers! Sitting just outside my door was a large box of flowers! Dismissing the oddity of someone giving a man some flowers, all my feelings of disgust and depression melted away, some secret admirer had bought ME flowers for valentines day.

Or were they a secret admirer? Perhaps not, I leaned over to investigate the label, perhaps a return address, that’s when I saw it, there, written under the field of Recipient… my flatmate’s name.  The delivery guy had put the box outside the wrong door.

Google Image Search Concept

Being a bit of a fanatic for HCI (Human Computer Interaction) I’m often spotting little ways that an interface can be improved.  Today it was Google’s Image Search.

When we search for an image on Google, after entering the keyword for what we’re searching for, our eyes instantly go to the pictures offered to us.  Often though, a more refined search is required and while Google does offer an selection of related searches sometimes, I feel that the image search should be a more visual experience, so why not offer these suggestions in a visual way too?

The video above shows my idea for how this could be done, it would be nice to see Google doing something like this in the future.

P.S.  Google, I am looking for a job and I do like warm weather -wink wink-

Dundee University Gives Every Email Address to Hackers!

Spot that little book button near the top? Wonder what's in there.

While most of us are enjoying the new email system at Dundee Uni hosted by the nice folk at Microsoft’s Office 365, many of us are noticing something else new… the spam!

I may have a theory as to why this spam has suddenly started pushing its way through to us in loads like we have never had before.  Many of us have chosen to syncronise our online Outlook account with Outlook on our PCs, it’s simple to do and it means our emails are available on our computers offline, but it’s not just syncing your emails.

The complete address book of Dundee University.

If you are using Outlook on your PC right now, click the Address Book button on your toolbar, what do you see?

That’s right, the entire address book including emails of students and phone numbers of staff is synced directly onto your computer, not such a big deal right?  Think again.

There are litterally thousands of people who are potentially using Microsoft Outlook to read their Dundee University emails.  What happens when just one of them gets a virus?

One of the major characteristics of a virus is that it spreads itself.  The most common method of doing this is through spam and if you have a virus on your computer, where’s the first place it’s going to look for people to send emails to?  In Outlook’s Address book.

It just takes one person using Outlook to get a virus and thus give away every single email address of Dundee University Students and Staff.  I’m not just saying it’s going to happen, I’m saying it almost certainly has happened and it’s a fair bet that somewhere out there, there’s a complete list of our email address on someone’s server ready to use for whatever they want to send us.