ResearchExec, owned and operated by the Fairfield, Connecticut based company of the same name, is an entirely web-based survey system that is available either as a hosted solution or as a package you can install on your own server. It offers a tremendous amount of customizability as well as a number of advanced features such as the ability to set quotas for individual questions and advanced survey logic. It has integrated panel management that allows you to send surveys to specific members based on their responses to previous surveys. The survey development system, while elegantly designed and extremely flexible, is tedious to use and does not play well with FireFox 1.5 or Internet Explorer 7 RC1. It offers very little in terms of a reporting system, and you should expect to do most of your analysis in a separate program such as Excel or SPSS. Because of the focus on advanced features, the sharper learning curve and the lack of a reporting engine, ResearchExec is more likely to appeal to users at a professional research firm and not those looking for a quick easy way to produce and report on internet research.
Survey Capabilities
ResearchExec offers a strong, unique combination of basic and advanced surveying features that will meet both the needs of the basic user and the advanced user. Some of the features I have yet to see anywhere else.
ResearchExec offers the traditional variety of question types plus a few extras. Specifically, it offers both drop-down and radio button single response questions; multiple response checkbox style questions; single and multiple line open-ended questions and multi-line rating questions. The system also offers several "special" open-ended questions where it will validate the responses to be e-mail addresses, zip codes, or phone numbers. Plus it offers specialized date and time fields where respondents can use drop down boxes to select month, days, and times. You can also include constant sum questions (where users are given a certain number of points they can allocate among different choices).
ResearchExec offers smart response validation. In other words, date fields allow you to specify a range of dates that may be entered; numeric fields allow you to select a range of dates. All questions can require or not require a response of some kind. For most surveys, this is all the validation you will need. ResearchExec also allows programmers to embed additional validation through client-side scripting, samples of which are available from the company.
ResearchExec offers unique implementations of both branching and piping — plus it includes something called "linking" which might prove very handy for certain types of surveys. Branching is what allows you to skip questions or even entire pages based on responses to questions answered earlier in the survey. What makes ResearchExec’ implementation particularly interesting is the ability to set quotas — example: if you only needed 100 men to answer a particular question, you would first ask if they were men and then use the ResearchExec logic to only allow the first 100 men to branch to the question you needed answered. If you met the quota, future men would be directed to another question or a different page. Pretty cool. In addition, ResearchExec can also be set to allow questions and/or answers to appear only when respondents meet specific criteria.
In that same vein, ResearchExec also allows you to set basic quotas for each question — so instead of forcing everyone to answer every question — even after you’ve met your quota — you can automatically pull a question from the survey once the specified number of people have answered it.
Another fancy feature is called "linking" (commonly referred to as "looping"). This is a function that allows you to repeat a question multiple times based on responses to previous question — for example, if I asked you to indicate your three favorite colors, I could then ask — for each color you select — to describe a fruit that matches that color. Although I’ve never had the opportunity to use this feature in practice, it sounds like it could save you a lot of development time.
ResearchExec also allows you to randomize both question and answers. It does this in a way that probably provides more flexibility and control in this regard than any system I’ve seen, but I also have to say that it can be kind of time consuming to implement.
ResearchExec allows you to import respondent answers from either a text file or a URL query string. This can be an extremely useful feature, especially in situations where you are having your respondents respond to information you already know about them (for example, consider a survey that you ask customers to participate in right after they make a purchase from your web site — with this feature, you can import all of the data from the sale such as what they bought, how much they spent, how they paid, etc and then use it in the survey). Imported answers can be used to "fill in the blanks" for guests automatically or even better, they can be used to control branching (so that only respondents who meet certain pre-defined criteria will be asked to answer specific parts of the survey).
Overall, ResearchExec offers some of the most advanced survey features I’ve seen in any of the packages I’ve looked at so far or had the opportunity to use in the past. If you’re interested in doing complex, interesting things with surveys without the need to learn a programming language or dig into a software development kit, ResearchExec may be for you.
Design/Development
ResearchExec offers a sophisticated web-based survey design/management system that provides access to numerous questionnaire and design options. The price of all of these options (complexity?) is that I did run into a few technical issues and found it to be quite time consuming to build and tune a survey.
When you create a new survey in ResearchExec, you’re first decide whether you want to start from a blank survey or from one of sixteen pre-written templates from one four categories (customer satisfaction, demographics, HR surveys, and Internet Related Surveys).
Although I did find the interface to be nicely designed from both an aesthetic and functional standpoint, I did have some technical issues. For example, the WYSIWYG editor (used to add pictures to questions or change fonts) didn’t work so well in Internet Explorer 7 RC1, and I couldn’t get anything to work at all in FireFox 1.5. Whether the problem was with my machine or limitations in the software I do not know, but make sure you test the program on your system before you invest (note: according to the company, ResearchExec does not support FireFox but it will have full support for IE7 by the time the final version is released).
…and while the software gives you access to a lot of advanced features that can be used to control precisely how your survey looks and acts, developing a new survey from scratch can take a very long time. Lots of clicking is required. Lots of screen refreshes are required as well. I’m not sure whether this is simply an unavoidable fact of life that comes from having such a feature-ful survey development system or whether it too closely tries to model the experience of developing a survey in a client-based machine — or if it is simply the result of less than thoughtful design. However, creating and editing a simple question like, "What is your favorite color?" followed by three different randomized options took (in my opinion) far too long. Even worse, try creating a list of list of twenty different colors and manually moving a color at the bottom of the list to the top of the list — it takes forever! (note: the company expects to have this issue resolved within the next couple of weeks).
With that said, ResearchExec has the ability to import MS Word documents that are prepared in a specific format and turn them directly into questions. According to the company, most of their users put surveys together in this way and I expect it speeds things up considerably.
ResearchExec does offer a lot of options to help you configure how your survey looks and feels. It allows you to pick from 27 pre-configured design themes or create and save your own — which means that after you come up with a look that you like it is fairly easy to re-use it over and over again. This is a nice feature — and, given the time that this will save you it does make up for to some degree some of the issues described in the previous paragraph (sometimes it can take a lot of time to get your survey to look just right). You can set the default fonts (face, size and colors) used for questions, answers, buttons, headers, and errors — plus you can add a header and/or a footer to each page.
In addition to controlling the default design, you also have a lot of flexibility when it comes to designing individual questions. For example, in addition to changing the font, color, and size of the text of each question (or answer!) you can also insert images in just about any text box. This can be an extremely useful featured if you’re doing new product development research, branding research or just about any kind of research where a picture is more meaningful than words. To that end, because you do have precise control over the HTML used in each question and answer, you should have no problem including Flash, video, sounds, animations, etc in your surveys.
As one would expect from a system that allows branching, piping and skipping, ResearchExec does allow for multiple page surveys and you can control which questions appear on each page.
ResearchExec offers a well implemented, easy to use question library that should go a long way towards helping you standardize your question base (another feature that makes up for the slowness of the design process, since once you create a question the way you like it, it becomes very easy to insert the question in any survey you develop). Basically, questions that you add to the library are added to the drop-down list of question-types so they are readily available, right in front of your face during the question-building phase. Very nice.
ResearchExec offers a preview function that allows you to test out your survey at any time. This is particularly important in the case of ResearchExec because after you finish your survey, you have to submit it to be approved by ResearchExec before it is then moved from the "development" server to the "production" server. This "approval" process is not something I have seen from other vendors and is somewhat surprising…it slows down how quickly you can get your surveys into the field (and there have been plenty of times when I have needed to get things up and running immediately in order to meet a deadline. The company says that in actuality, regular customers are allowed to bypass this approval process and that for others the delay can sometimes be as little as a couple of hours. Although beware — according to the web site approval time can take up to 48 hours.
To make it even worse, if you have to make a change to your survey later (because you forgot to add a question, or to fix a typo, or maybe even clean up a bit of branching logic that isn’t working right) you have to resubmit your survey for approval again before the change is made.
User Experience/Response Management
Once your survey is activated (after it has been approved by ResearchExec) you can control when the survey will end or you can set a limiting quota. One neat feature that I’ve found only in ResearchExec is the ability to control what happens to the user if they try to take the survey when it is closed. You can either have it show the default system message, a message of your choosing, or you can redirect them to a web page of your choosing.
ResearchExec allows you to include a progress bar on each page as well as the necessary "continue" button that is used to move on to the next page. It also gives you the option of whether or not you want to include a "Previous" button so that users can go backwards. Surveys can also be configured so that users can save their answers and come back later to finish or make changes.
ResearchExec does offer the ability to monitor your survey’s statistics, such as the number of people who have started the survey, the number of people who have finished the survey, including the number of people who were screened out. You can also monitor the degree to which each of your quotas are being met.
The system will allow you to eliminate specific responses through the reporting system. You can also use the filtering tool within the reporting system to eliminate batches of responses that meet a specific criteria.
Panel Management
One of the elements of ResearchExec that I like most is the panel management functionality. There are many survey systems that will maintain a "panel" of respondents for you — which basically means you can upload your list of e-mail addresses into the software and you can then use it to mail out invitations to possible recipients for multiple surveys. What makes ResearchExec unique is the fact that it will allow you to import your people and then build a history for each of them based on their responses to the surveys you give them. In other words, ResearchExec allows you to choose people to receive specific surveys based on their responses to earlier surveys. That way, once you find out that a guy is a guy, you don’t have to waste his time by sending him surveys about cosmetics and feminine products.
The other panel management feature that ResearchExec includes is the ability to keep your e-mail list fresh by automatically de-activating respondents with non-working e-mail addresses.
ResearchExec also has the ability to send reminder messages to members of your list that have not yet started or completed your survey.
Reporting Capabilities
Based on the extremely limited reporting/charting capabilities of ResearchExec, I’m going to assume that the designers felt that most users would want to analyze their data using professional analysis software rather than in the software itself. Yes, I appreciate the ability to generate frequency tables for each of my questions, but I had kind of expected to be able to do at least a little analysis in the software. ResearchExec does not allow you to generate any charts and graphs that can be exported into presentations; it doesn’t allow you to crosstab the data; it doesn’t allow you to create reports that include only specific questions.
It does allow you to filter the data based on questions in the survey. It also allows you to export the data into CSV and TXT files which can then be imported into other applications. You can also e-mail the basic frequency reports to yourself or others. But that’s it.
If you’re the type of person who doesn’t want to use an outside package to do analysis and simply want to be able to create your reports/charts in the survey software, then you definitely want to find a different survey package. However, if your sole focus on using a package such as this is to collect data for use in other analysis/reporting systems, then the lack of built-in reporting functions probably won’t be a consideration in your decision process.
System Requirements/Security
ResearchExec is available in four flavors: an "Enterprise" version that runs on your own equipment with all of the features the software has to offer; a "Professional" version that also runs on your own equipment but that doesn’t include advanced survey logic, full e-mail management, or panel management features; a "Managed ASP" version which runs on ResearchExec’s equipment, offers a limited number of administrative accounts and "Classic ASP" version that offers limited only one administrative account, limited survey logic, limited e-mail management and no no weighting/scoring capabilities.
ResearchExec does allow you to have multiple accounts and the security of each account can be individually controlled to allow access to only specific objects.
If you want to run either the Enterprise version or the Professional version, you’ll need Windows 2000 Server edition (or greater, I assume); Microsoft Internet Information Server (free) and Microsoft SQL Server 2000 (definitely not free, but you might be able to use an account on an existing server).
Vendor Related Notes
ResearchExec tells me that they’ll be releasing version 7.0 of ResearchExec in the 1st quarter of 2006 which will be completely redesigned and redeveloped,
Worth noting: the most recent press release on the "Current News" page of the ResearchExec web site is dated August, 2003.
To learn more, visit http://www.researchexec.com.