Web Mail Blog

Thanks to everyone who participated in our recent survey.

I received thousands of responses, and statistically the sample size indicates a 99% confidence level that the quantitative responses (multiple choice) fall within +/- 2.5% of the mean for the entire Web Mail user population. That means the answers should fairly represent most users. There was also great participation in the open-ended questions, where you could describe what you need or would like to see in Web Mail. I added all those answers up too (thousands of open-ended answers and I read every single one) and there were a lot of repeated themes that I'll summarize and rank.

As promised, I'm publishing the full results here. We're still digesting all the information internally and formulating plans to try and better meet the top needs of our users. But you can count on a lot of this information leading to specific solutions.

The survey gathered 3 sets of useful data:

• Web Mail usage by tenure, frequency, connection type, browser, and OS.

• Web Mail satisfaction for reliability, speed, features, storage, and ease-of-use.

• Unmet needs and desired improvements.

We asked 8 required multiple-choice questions for quantitative and ranking data, and 3 optional open-ended questions for additional data on user needs and problems. Here's the results, and some comments summarizing what we learned:

First, the basic questions about Web Mail users.

If we count all the respondents who say they have used Web Mail for over 3 months, AND use Web Mail at least 3 times per week, AND use Web Mail at least as regularly as POP (Outlook or similar software), we get 77% of all the respondents. That's a high percentage of regular and heavy users providing the data. This "self-selection bias" of respondents means we are hearing from engaged users who have an opinion.

28% say they "usually" or "regularly" use Web Mail over dialup. This is a very important group that has specific needs, particularly speed optimization (more on that later).

• 15% use Safari, which is a unique browser shipped on Macs that sometimes has different behavior than IE and Firefox.
• 19% use Mac OSX.

• 9% are using a Windows version older than XP or Vista (at least sometimes, users could choose more than one answer).

These are significant population segments that must be supported.

Next we asked users about general satisfaction.

I've graphed the responses. Here's the results for all respondents combined.

Here's the results for only those users who said they usually use Web Mail over a high-speed connection in question #4.

And for the dialup users:

Both high-speed and dialup users rated "speed" and "features" the lowest overall score of the categories listed. The best overall scores were for "reliability" and "ease-of-use".

High-speed users rated features lowest.

Dialup users rated speed lowest.

High-speed users are generally a little happier about everything, with top 2 boxes averaging 79% vs. 72% for dialup.

Next we asked users to rank the importance of several enhancements we are already considering, in order to help us prioritize them.

The top ranked responses, in order, are:

1. "Auto-save while writing messages so the text isn't lost if I lose my connection."

2. "Never flag a message as spam if the sender is in my Address Book ."

3. "Save my default font settings for writing messages."

4. "Put common tasks in a toolbar across the top of the page."

Some others that were highly ranked by users are:

• "Insert pictures inline in the body of a message."

• "Allow more than 3 file attachments on outgoing messages."

• "Allow easy switching between mailboxes while logged in."

• "Improve searching and sorting in the Address Book."

We then asked three open-ended questions:

74% of respondents answered one or more of these, and 52% answered two or more. Here's the top 20 responses for each one. Some of the same general responses show up in each of the questions (we'll combine them in a minute).

9. Without limiting your answer to the list above, what one new Web Mail feature would be most useful to you?
Rank	Generalized response
1	More personal control of spam (higher block list limit, allow blocking top-level domains, allow custom filters)
2	More storage space
3	Auto-save while writing messages so the text isn't lost if I lose my connection
4	The feature or preference already exists for what the user wants but they don't know it
5	Better rich-text compose (save font settings as default, should work on Safari, make it easier to use)
6	Improve the layout (make it easier, toolbar at the top, signout link at top)
7	Improve the speed, particularly log-in time
8	Allow easy switching or combined viewing of multiple mailboxes, bypassing login screen
9	Improve address book sorting, searching, and syncing
10	Be able to insert pictures in the body of new messages
11	Eliminate the 3-attachment limit in Web Mail
12	Remove advertisements or move them out of the way
13	Improve the search feature, including message body text
14	Add a positive whitelisting feature for addresses to bypass all spam filtering
15	Improve reliability
16	Allow compose and/or read message in separate popup window(s) so I can do more than 1 thing
17	Improve the spell-check ( better dictionary, inline checking, allow add-to-dictionary)
18	Make forwarding easier and more accurate, particularly with inline images and certain message types
19	Support mobile devices
20	(3-way tie) Login without typing domain; Larger attachment limit; Don't require login every time from same computer

10. If we could fix one thing in Web Mail that doesn't work for you today, what would it be?
Rank	Generalized response
1	Speed, particularly log-in time
2	Advertising complaints or trouble caused by ads
3	Better control of spam (higher block limit, block TLDs)
4	Better rich-text compose (save font settings, support Safari)
5	The feature or preference already exists for what the user wants but they don't know it
6	Reliability
7	Desired messages are being caught as spam
8	Storage space is insufficient
9	Autosave to prevent composed messages being lost; save text on lost connection
10	Forwarding trouble, particularly with inline images and certain message types
11	3-attachment limit imposed by Web Mail interface
12	User is confused about POP vs. Web Mail, can't find messages, not in sync
13	Can't insert pictures inline in compose
14	Address book feature/usability improvements
15	Allow saving password or keep session open even after closing browser
16	Allow compose and/or read message in separate popup windows so I can do more than one thing
17	Allow easy switching between accounts, bypassing login
18	Need a preference to return to Inbox on delete, rather than display next message automatically
19	Mobile device support
20	Better searching, including body text

11. Please briefly describe any other improvements you would like to see in Web Mail.
Rank	Generalized response
1	Remove advertisements or move them out of the way; no animated ads
2	More storage space at no charge
3	The feature or preference already exists for what the user wants but they don't know it
4	More personal control of spam (higher block list limit, allow blocking TLDs, allow custom filters)
5	Improve the speed, particularly log-in time
6	Improve the layout (make it easier, toolbar at the top, signout link at top)
7	Better rich-text compose with more fonts and features; save font settings as default
8	Improve reliability
9	Support mobile devices
10	Better customer support via phone and chat
11	Add a positive whitelisting feature for addresses to bypass all spam filtering
12	Improve address book sorting, searching, and syncing
13	Auto-save while writing messages so the text isn't lost if I lose my connection
14	Improve the spell-check ( better dictionary, inline checking, allow add-to-dictionary)
15	Allow larger file attachments
16	Better searching, including body text
17	Allow saving password or keep session open even after closing browser (so I don't have to log in every time)
18	Allow compose and/or read message in separate popup window(s) so I can view more than 1 thing
19	Easier forwarding
20	Provide IMAP access

Now let's put it all together, combining the feature questions with the open-ended responses to come up with a top 10 list of what most users want.

Before I get to the top 10, here are a couple things worth noting. In the open-ended responses, a number of users said don’t change anything, they like it a lot just like it is.

A significant number were concerned that "changes" or "improvements" might mean making things fancier at the expense of speed and simplicity, like they’ve seen some other major email providers do. Most users want Web Mail to be fast and simple, rather than trying to imitate a desktop email program inside a web browser.

The most common comparison made to a "good" example, was to Gmail, which is full-featured but still clean and uncluttered. Only a very few users would like Web Mail to be more "like Outlook" or more "client-like" and attempt to behave like desktop software (which tends to hurt performance a lot in a browser-based application). I agree, and we aren't planning to go down that path.

Also note that in all three open-ended questions, a number of users requested a feature or setting that already exists. We released an update in April (right before the survey ran) that has a completely reorganized Preferences section, making it much easier to find the setting you are looking for. If you haven't looked at it lately, click Preferences and see what is available there.

Here's the overall top 10 user needs or concerns:

1. Advertising - in the way; slows me down; animation interferes; I object to it in general.

2. Speed - particularly login time; top issue for dialup users; ads perceived as a contributor too.

3. More personal control of spam - higher block list limit; allow blocking top-level domains; allow custom filters.

4. Storage space is insufficient.

5. Improve the rich text (HTML) compose - save compose font settings between sessions; don't require a toggle; don't lose text when toggling; make it work on Safari; allow inserting pictures inline.

6. Improve the layout - including a toolbar at the top with major tasks.

7. Auto-save text during compose and allow inline re-login if session is lost so no composed text is lost.

8. Improve overall reliability.

9. Allow whitelisting - so that email from my contacts never goes to Known Spam.

10. Improve the Address Book - including searching, sorting, ease-of-use, and syncing.

From the list above I can tell you that we are working on doing something about all of them, and I can make a few specific comments about some upcoming changes.

• There are several changes coming soon that will improve speed. One involves a quicker login step due to a change in how our servers are set up, and that will happen in the next few weeks. We're also doing some things to make sure advertisements don't slow you down. Ads already load last on the page (except login page) for users of any up-to-date browser, so that the pages are ready to use before ads begin loading and you should be able to click anything you want without waiting. A few older browsers don't obey this, so update your browser if you can. We're also working hard to enforce file sizes on banners, and using a stricter standard for dialup users, so that they load quickly. And, we have a new layout coming later this year that should display most pages quicker.

• Advertising - other than the things mentioned above, we're also about to change the main banner placement to one many users have suggested. Up until about a year ago the banner was at the top of the page, and we moved it to the right side. That is being moved back in early June. Not only is the top placement seen as a lesser evil by many users (not as in the way), it is also smaller and will load faster. And finally, the Ad-Free Web Mail option I've mentioned before is almost ready, and it will be available to subscribers in June. More details will be provided soon.

• Items 3,5,6,7 and 10 are on our roadmap for this year and should show up in updates over the coming months. Some other requested items that didn't make the top 10 are also going to be included. We're still working on #4 (storage) but I don't have any update for now. We're working on #8 (reliability) all the time. And I'm trying to get #9 (whitelisting) implemented and should have more news on that soon.

Thanks again for your participation in the survey. You're welcome to comment here on the results or the preliminary plans I've mentioned.

Permalink   |   Comments (22)