Frequently Asked Questions
General OpenHallway Questions
Q: What is hallway usability testing?
A: Joel Spolsky explained hallway testing best in one of his blog posts, "A hallway usability test is where you grab the next person that passes by in the hallway and force them to try to use the code you just wrote. If you do this to five people, you will learn 95% of what there is to learn about usability problems in your code." At OpenHallway, we believe in enabling people to do Hallway testing remotely (we are a team of four people, so there aren't many people in our hallway). OpenHallway allows you to create a testing scenario and email the test to friends, family, hired testers, or colleagues.
Q: Is there any software to install?
A: No! OpenHallway is completely web-based. You can create your tests in any browser (Mac or PC) and your testers can take the test in any browser (Mac or PC). We enable true remote usability testing. The only requirement for the user is to have Java 1.5 or later installed. Don't worry, we do a check on your system and help you get Java installed if you need it. If you want to get the latest version of Java on your computer, go to: http://www.java.com/getjava and follow the instructions.
Q: How do I get testers?
A: OpenHallway doesn't provide tools to help you get testers. We like to focus completely on the testing part. We recommend you read Steve Krug's book, Don't Make Me Think, for more ideas on getting testers (specifically chapter 9). The basic concept of Hallway Usability Testing is that you perform tests with people who can provide valuable feedback. This can often be a co-worker, a friend, a family member, or better yet, real users of the system. There are many ways to get testers including advertising on sites like Craigslist and offering compensation for testing. Many stay-at-home moms and dads are looking for extra cash and may be your target audience. The simplest approach is to find people who work for you and as Steve Krug says, "Take anyone you can get (within limits) and grade on a curve. In other words, try to find users who reflect your audience, but don't get hung up about it. Instead, try to make allowances for the differences between the people you test and your audience." The important thing is to lower your barriers so that you test often.
Q: Does OpenHallway record just the browser activity?
A: No, once the tester clicks "record," the entire screen and voice are recorded. This allows you to do a lot more with your usability testing. If you want to test software that has been installed on the user's machine, you could create a test, leave the URL field blank, and give instructions once the tester starts the test to launch an installed application. Then the tester will need to come back to the open browser window to stop the test.
Q: Where can I learn more about identifying 95% of usability issues with just 5 usability tests?
A: We recommend reading Jakob Nielsen's research on "Why you only need to test with 5 users." We really like this philosophy and find that it works. Many times, 2 or 3 data points from a quick usability test can shine the light on glaring usability issues.
Q: Does OpenHallway do live usability testing sessions with screen sharing?
A: Nope, OpenHallway is not a live screen sharing tool, it's a screen recording and voice capture tool (we record the tester's screen and voice and then upload the video for you to watch after the recording is completed). OpenHallway allows you to create a user scenario and then send a recording link to someone. You can use OpenHallway both in a moderated or an un-moderated approach – locally or remotely. If you use openhallway in a moderated approach with the participant in the same room, simply copy the recording link into a new browser tab or window to begin the testing session. To test remotely, copy the recording link into an email, IM client, or put on a website or social network. The test participant can then click on the link to see your instructions and begin the recording session.
Q: How do I cancel my service?
A: Visit our contact page for our info and send us an email or call us. When leaving a voice mail or email, please make sure to include the first/last name on the account and the email address associated with the account. If you cancel your account, we delete all your information, and we're not able to recover it.
Q: How will charges from OpenHallway appear on my credit card statement?
A: Charges on your credit card statement will appear as "OPENHALLWAY LLC"
Q: What happens if I upgrade or downgrade my plan during a billing cycle?
A: If you upgrade or downgrade your plan, you will not be charged the new plan fee until your next bill date.
App Specific Questions
Q: How many people can I have test? How long can I test each user?
A: You can have as many testers as you want record as many times as you want within your recording storage limit. Individual recording sessions on the free plan have a recording limit of 10 minutes per session. The time limit for a recording session on paid accounts is 20 minutes. Participants will need to provide their feedback within the recording session time limit. If the tester goes longer than allowed time for the recording session, that person will be notified, and the upload process of the recording will begin. We recommend creating shorter, well-defined, "key tasks" to test. Watching a video much longer than 20 minutes usually ends up in diminishing returns for the test creator.
Q: What happens after a tester completes a test?
A: After a tester is done with the scenario you've created, she needs to click the "stop" button on the launch test window. This will initiate the upload process, and the tester will be asked not to close the browser until the upload is complete. After the upload finishes, you will be notified via email that the upload is complete. You can then log in to your account to view the video.
Q: At the end of a test, can I ask the tester to take a survey or give a gift?
A: Yes! When you create a new scenario, you can enter any text you want in the "Custom thank you message" field. The custom text appears after the tester stops recording and has the opportunity to put in their final comments. While we don't provide the tools to create the survey, you can put in a link to a survey and/or a link to a place where they can redeem a gift/giftcard. There are many tools out there such as SurveyMonkey that allow you to create extensive surveys, and many of the these tools are free.
Q: Once recorded, can I share links to the video with others without them having an account?
A: Yes! We provide a link that you can share with anyone you want. You'll find a link immediately below the video player under a tab labeled "Share."
Q: What browsers are supported by OpenHallway?
A: On the Mac, Firefox and Safari. On the PC, Internet Explorer 7 and Firefox. We recommend using Firefox for pretty much everything, but especially recommend it when using OpenHallway. We don't support IE6 for Windows and wish that browser would just go away. We apologize in advance if not supporting IE6 causes an inconvenience for any of your testers.
Q: Why are there differences in readability of recorded videos?
A: We capture the entire screen, even if the tester is using a wide screen monitor (like a big external monitor). For best results, we recommend the tester use a screen resolution between 1024x768 and 1680x1050. We recommend that your testers have a computer with a microphone. This allows you to get the most out of your testing and allow the tester to "think aloud" as they test. When you record a monitor with a lot of screen real estate, words on the screen become harder to read in the video.
Q: What if my tester doesn't have a microphone?
A: We believe there's value in just the screen capture of a usability test. If the tester doesn't have a mic, he or she can still take the test, we just won't be able to record the sound. Before the tester begins the testing scenario, we'll let the tester know that the screen and voice are about to be captured. If there is no microphone, we'll let the tester know that just the screen will be captured.
Q: What if the tester has dual monitors?
A: Right now we just support your main monitor. If you know your tester has dual monitors, it's a good idea in your invitation email to ask to use just one monitor during the test, otherwise you may just hear the tester's voice and see one screen image.
Q: What happens when I reach my storage limit?
A: As you reach your video storage limit, you'll see a warning in the storage limit indicator in your account. This indicates that you'll need to delete some videos out of your account. If you do exceed the limit, don't worry, we won't start deleting your stuff right away. Right now, we'll just send you a friendly email notifying you that you've reached your limit. You can then delete some of your videos or upgrade to a higher plan.
Q: How long will the videos be stored?
A: As long as your account is active, your videos will be available. You can decide which ones to delete and which ones to keep within the space limitations of your plan.
Q: What do you do about privacy and storage of video files?
By default, all accounts upload video via SSL. OpenHallway also uses a proprietary method for encoding videos for an added layer of security making SSL almost unneeded, but we did it anyway. For Premium users, videos are also stored in the database encrypted using a unique encryption key per OpenHallway account. Delivery of the video back to the account is also done in a secure way with this encryption technique. For premium accounts, the Enterprise Security feature is on by default. If you want to turn it off, go to the “settings” link (part of our UI updates). Enterprise security is a great feature if you are planning to do a lot of live testing of ecommerce sites with real credit card data being recorded or other sensitive personal information.
Q: Can I download the recorded videos? What format will they be in?
A: If you have a premium plan, you can download the recorded videos in MP4 format. Most video editing software now lets you work with and edit MP4 (H.264 encoded and AAC Audio).
Q: The volume is low on the video I'm watching, what's up?
A: Sometimes a tester's mic is turned down and she don't know it. If you have concern that a tester isn't familiar with the microphone settings, it might be a good idea to call or email the tester and let that person know how to change it. Mac microphone settings (System preferences - Sound - Input). Windows microphone settings (Control Panel - Sounds and Audio Devices - Sound Recording - Volume).
Q: What if I want to record video playing within the browser (like a YouTube video)?
A: OpenHallway throttles the recording of a video (like a youtube video) in the page. If we captured the full frame-rate of a video playing in the browser, it would be a really large file to upload. If your tester is watching a video within the page, you'll notice a "tiled" effect within the recording of that video. This is because we are slowing the frame rate capture of that video area so as to keep the file size for upload smaller.
If you have questions, feedback, business opportunities, or want to chat about usability, here's how you can get a hold of us.