Let’s start at the beginning: the survey design interface. Beginning a new survey is as easy as selecting “Create Survey” from the main screen. You can either start with a blank survey, use one of the nine templates that are available (not a very big selection actually…and they’re pretty simple too), or you can copy one of your existing surveys (SurveyGizmo allows you to create an unlimited number of surveys, even if you’re using the free account).

SurveyGizmo’s survey development environment is clean, easy to navigate, and doesn’t require any training or instruction. For example, consider the blank questionnaire I’ve started above — the page even includes a green arrow telling me exactly what I need to click in order to add my first question. The entire interface is like that.

SurveyGizmo has a number of question types — and as far as I can tell new ones are regularly added. It of course includes all of the traditional questions such as single choice radio buttons, multi-choice checkboxes, drop down menus, text boxes and essays,but it also includes some really fancy question types that I’ve never seen anywhere else. For example, SurveyGizmo offers a star ranking table question (see below) that makes it easy and fun for respondents to rate items. It offers image-based single selection question (and makes them easy to implement). It allows for tables/matrixes of radio buttons, drop down menus and checkboxes and also supports a fairly friendly continuous sum and ranking question. SurveyGizmo also supports custom matrixes (table questions that allow for multiple question types in each row) and custom question “groups”, as well as file uploads — but these three features are only available to paying customers.

SurveyGizmo gives you a number of “advanced” option for each question type without making the process intimidating — thereby ensuring that you’ll have an easy time creating your survey but still have the ability to make it do exactly what you want. For example, I created a simple single-selection radio button question with three options. After I finished entering the basic question, I could then easily change the type of answers (checkbox, radio button or dropdown menus); change the orientation of the answers; set a default value; change the number of columns; randomize the options; indicate whether I wanted the question randomized on the page; hide the question based on criteria; number or not number the question; etc. I could also add a custom CSS class name. That’s a lot of options!

For text questions, validation is available to ensure that the responses fits a certain criteria. It is extremely flexible, and is based on regular expressions (or you can choose an option from the drop down list). It is very easy at any point to mark a question as required or not required. If you indicate that a question is a date, it will pull up a calendar when the respondent clicks on it — if you indicate that the field is an e-mail address, the system will do validate that the e-mail address is valid — not just by checking the format, but by contacting the respondent’s e-mail server to make sure it exists. Numeric fields can be automatically formatted as percentages, currency, or whole numbers (it won’t let the respondent enter decimals).

SurveyGizmo’s development environment is entirely AJAX based, which means that when you click a button or make a change you don’t have to wait for a postback — the change takes place immediately. Not a lot of survey packages offer this and it will greatly speed up your survey programming. Surveys can be previewed at any time.

It is very easy to add images or multimedia to your survey — however, only paid accounts are allowed to upload media for storage on the server, so if you’re using the free account you’re going to have to store your images on an account on a different system. Hard to complain though since it’s free…

SurveyGizmo allows branching, skipping and piping. Skipping can either be managed through a simple drop down box (show this question when some other question equals some value) or using advanced rules which have to be manually entered. Piping can be used to add responses to other questions to either the question text or the response area.

In addition to questions, SurveyGizmo allows for “Actions” to be inserted between questions. Some of these actions are about as advanced as any I’ve ever seen in a survey software package and allow you to do random branching, send mobile messages, allow for subscribers to log in, retrieve web content and send e-mails. Most of the actions are available only in the “Pro” version and the “Enterprise” version, but I think it is fair to say that many of the features available in SurveyGizmo aren’t even available in many of the paid packages I’ve tried.

SurveyGizmo allows you to choose from around 20 different “themes” which let you control the color scheme and the general look and feel of your survey. You can have your questions appear in whatever font you choose, and you can add your logo to the top — even for the free surveys. If that’s not enough customization for you, SurveyGizmo gives you direct access to the survey page’s underlying HTML template and the CSS file used to control how your page looks. Most survey packages make it difficult to get this level of control…

If you sign up for an account, you also get access to response quotas, auto-close on date, vote protection (de-duping), forward-only surveys, URL variable patterns, and IP-based geotracking. With the Enterprise version you can also eliminate the “powered by” icon that appears on the page.

Launching your survey is very easy and the system lets you go back and edit it at any time. Every survey has it’s own custom URL, and you have the option of creating your own custom survey URL as well. Surveys can easily be embedded into your web site and all of the code is provided for you. Special code is provided so that you can embed the entire first page of your survey into a MySpace page, an e-mail blast, or just into a page on your web site.

SurveyGizmo will send out survey invitations for Personal, Por and Enterpise accounts (not for free accounts). Upload a list, and the software makes it fairly easy to track who has taken your survey. You can also send out followup invitations.

Data can be exported in either text format or as a CSV file (no SPSS option except in the Enterprise version). One neat feature is that all data is saved on their server until you delete it (not automatically deleted after a certain period). SurveyGizmo offers an external database sync which allows you to send/receive information to outside websites and databases.

Reporting is extremely flexible — you can create multiple reports for each survey including charts, graphs and crosstabs. All account types can filter reports by date range, although only paid accounts can filder data by responses.

SurveyGizmo Pro and Enterprise allow for multiple user accounts (5 accounts for Pro, 20 accounts for Enterprise). Permissions can be user based, and team-based permissions (where groups of users can be partitioned) is also available. Surveys can also be secured to ensure privacy, although media (such as videos and pictures) cannot be secured using the software.

SurveyGizmo offers a variety of tutorials (some of them as videos) on how to use the software, ranging from how to create a new survey to personalizing your survey to details on some of the more unique advanced features. In addition, SurveyGizmo has a very active support forum with over 2000 posts one of which is dedicated to new feature requests.

SurveyGizmo is available is four flavors: “Free,” which gives you access to all of the basic features, the ability to create an unlimited number of surveys but only allows you to collect 250 responses a month; “Personal” which costs $19 and offers 1,000 responses a month plus some of the more advanced features; “Pro” which comes iwth 5,000 responses a month and “Enterprise” which comes with up to 50,000 responses a month (that’s a lot of responses).

I really like SurveyGizmo. I like the clean interface, the variety of features and the fact that it appears that they are constantly updating it with new capabilities. I can say with a great deal of confidence that I will be using this survey package for my own future projects and may even consider the enterprise version for larger applications. Certainly, if you’re doing a simple project for school or for your internal organization, I would recommend the free version of SurveyGizmo over either SurveyMonkey or any of the other free applications out there.

• SurveyGizmo

Survey Capabilities

SurveyZ includes all of the basic question types and adds a few extra "combo" question types (questions in which multiple basic question types are combined to form more complex elements) such as a "multiple line text" question (which allows you to ask several open-ended questions together through the use of one question, a "spreadsheet" question (which creates a grid of open-ended questions that look a little like a spreadsheet) and semantic differential questions (very much like traditional rating questions, except that you can use different bi-polar adjectives for each scale (see the sample below). Actually, SurveyZ gets a lot of credit for offering semantic differential questions at all — most other packages only offer basic Likert scales (SurveyZ does too) but there are times when a semantic differential question can be extremely useful.

Sample of SurveyZ semantic differential questions

SurveyZ also offers the ability to include matrixes where you can group together multiple questions into one grid. The functionality of these tables are somewhat limited in that you can only include one type of question in each matrix. SurveyZ also supports ranking questions and constant sum questions, which gives SurveyZ about as much variety in the question types that it makes available as most of the other packages that I’ve looked at.

SurveyZ supports some validation of fields: for example, you can require that respondents answer certain questions, and for open-ended questions you can require that responses are either textual, numeric, or even an e-mail address.

SurveyZ makes it easy to execute conjoint studies.

SurveyZ includes a few very special question wizards that help you execute some pretty advanced research studies. For example, the "Random" wizard allows you to randomize any group of questions in your survey (you can have multiple groups in each survey). The "Conjoint" wizard will create and manage a conjoint question set (either full profile, discrete-choice, or self-explicated) which helps you understand the relative utility of various attributes of products and packages. The "Parallel" wizard makes it fairly easy to have two questions appear next to each other.

Conjoin exercises  appear as single questions — easy to edit.

The value of the conjoint wizard shouldn’t be underappreciated — it took me just a few moments to create a basic conjoint exercise. When I was finished, the entire exercise (which consists of several questions) were all rolled up into a neat package that I could edit or move around my survey at will.

SurveyZ supports a variety of advanced survey logic features including skip patterns, branching, piping, and looping. SurveyZ can also randomize questions, groups of questions, and answers within questions. All of these features are fairly straightforward to implement and function as well (if not better) than similarly priced packages offered by competitors.

SurveyZ’s branching setup is easy to use

I think the SurveyZ branching system is about as flexible as any I’ve seen so far — it allows you not only to skip to specific questions or groups of questions based on specific responses, but also to skip to specific questions based on pre-populated data (such as whether the respondent made a particular purchase). SurveyZ can also branch based on whether quotas for specific responses have been met — so, if you’ve already had 100 males fill out certain portions of your survey you can skip male respondent #101 to a different set of questions or even right out of the survey altogether.

SurveyZ also allows for trigger-based responses, which is a special feature I’ve seen in only a couple of packages that will allow the system to send an e-mail or redirect the respondent to another web site (or survey) based on their responses to questions. This might be particularly useful if you wanted to receive an e-mail immediately if a respondent gave you a particularly poor rating or if they indicated they wanted information about a particular type of product.

SurveyZ also includes something called "form grading" which allows you to assign point values to survey questions which can then be totaled at the end of the survey to yield a grade that is reported to the user.

Survey Design/Development

If you’re a non-researcher or simply someone who doesn’t want to spend a lot of time writing survey questions, SurveyZ might just be for you. It offers one of the largest pre-written survey libraries I’ve ever seen and features literally hundreds of surveys with thousands of pre-tested questions on topics that include customer satisfaction, human resources, product development, business to business, public/non-profit associations, academics, human interest ("pop psychology" and "personal interest") and numerous industry specific surveys. SurveyZ makes it easy to either copy and build upon an entire pre-written survey, or you can pick out and use individual questions.

With that said, I really don’t care too much for the SurveyZ design environment. It seems kind of ugly, kind of clunky, and very much like it just stepped out of a 1998 web site. Buttons and boxes are all over the place, the menus aren’t particularly well organized, the text boxes are too small, and the icons look like they came from one of those free icon web sites. I’ve seen much better, which is a shame because there are a lot of less capable survey packages out there that, simply because they look and feel more professional, might steal business away from SurveyZ.

It’ also a shame because after you get past the look, feel, and organization it really isn’t all that hard to create a survey. In fact, the process is pretty fast. All of the options for each new question you create are for the most part right where you would expect them to be (although, once again, it isn’t pretty).

SurveyZ doesn’t offer a lot of options for individuals who care about how their surveys look. You can choose from one of sixteen different color palettes, but otherwise you have relatively little control over fonts, sizes and color unless you use HTML – and even then, you have to change each question separately (so if you wanted to change your entire survey from Arial to Times Roman you would have to individually go in and edit each question and response. Ouch!

Users of the Enterprise edition can add their own style sheets which will allow you to change the overall look and feel of the survey all at once. However, other users are stuck with one basic design template (which in my opinion isn’t a very good one — there are issues with spacing, default fonts, and the overall organization of the page that I just don’t like very much.

Frankly, if I were the boss of SurveyZ, I would say that for their next major release they should hire a designer and a usability expert to revamp the overall look and feel of the program to bring it into line with what most users expect.

And if it sounds like I’m ranting, it is simply because I care — there are so many positive things to say about SurveyZ it disappoints me that the interface is so unnecessarily…unpleasant.

SurveyZ does allow you to produce both single page surveys and multi-page surveys (you can control where the page breaks go). Surveys can’t be imported from text files or MS Word documents, but you can build and maintain your own question and survey library.

As noted above, SurveyZ does allow you to embed your own HTML just about anywhere (headers, questions, answers) which means it is possible to add images and other forms of multimedia to your surveys. One note though: SurveyZ does not come with an interface that lets you upload images and other multimedia to include in your survey — so if you want to add a picture (or a video) you will need to host the images on your own web server (the Enterprise version of SurveyZ does allow for construction of a library of images).

SurveyZ will allow you to preview individual questions as you create them, or at any time you can click one button to see your entire survey previewed in a separate window.

User Experience/Response Management

The SurveyZ respondent interface is straightforward enough and users shouldn’t run into any issues. As the survey designer you have the option of including a progress meter on each page, and you can also allow your users to save their surveys at any time and then come back and finish them later (and here’s an interesting twist: the system will also let you e-mail a link to another user to finish the survey for you, which may be necessary for some kinds of complex corporate surveys where different people need to complete different sections).

The SurveyZ survey completion interface does not appear to allow for a "Previous" button that respondents can use to check/change their answers.

One nice feature is that SurveyZ can be configured to e-mail a confirmation note to the respondent after they complete the survey.

SurveyZ includes a built-in invitation system which you can use to invite respondents to take your survey, or it also provides you with a URL that you can embed on your website or in a newsletter. Even better, SurveyZ provides you with code you can install on your web site so that the survey will pop up in a separate window (either every time or once for each visitor).

SurveyZ also provides you with code you can use to embed your survey as a form in a web page on your site, or in an e-mail newsletter (using this option would undoubtedly give you more control over the look and feel of the survey, although I still think it is a lot of unnecessary work).

The trial account I used to test SurveyZ didn’t give me access to the e-mail system — however, as far as I could tell (based on the description), you can use this system to upload and manage recipient lists; upload and manage custom data fields for tracking purposes; create personalized messages for e-mailing; monitor and manage delivery status; and repeat mailings to individuals who have not yet responded to the survey.

Although you can’t set predetermined start and stop dates for your survey, you can start and stop your survey manually at any time. SurveyZ will allow you to monitor responses as they are collected (you can see how many times your survey has been viewed, started and completed) and the system will allow you to go in and view/edit/delete individual responses.

SurveyZ also includes a cookie-based "anti-ballot box stuffing" feature that allows you to deter individuals from completing your survey multiple times.

Reporting Capabilities

SurveyZ has a very basic built-in reporting engine. It allows you to generate charts and graphs, crosstabs, and frequency counts. It also allows you to filter the data that appears in your report based on complex criteria, and it allows you to export your data into a CSV/TXT file which can then be analyzed in a separate program such as Excel or SPSS.

SurveyZ has the unique ability to generate SPSS command/syntax files that can be used to help you import your data more easily into SPSS. While this is truly more assistance than most other programs provide, it does make you wonder why the publishers wouldn’t just include an SPSS export feature (which would allow you to generate and save SPSS .SAV files).

Reports, such as the sample above, can be made publicly accessible via a URL, and you have the option to protect your results from public consumption by assigning a password.

If you are looking for a system that can generate custom, automatic reports that look nice and feature a variety of different types of analysis, then SurveyZ probably is not for you.

Extensibility

SurveyZ operates as a web-based service provider only — in other words, you can’t buy the software and install it on your own servers. While this makes it difficult to directly integrate SurveyZ into your existing system, it does let you integrate your own data into the survey results (which means that you can include a customer ID in the URL which will ultimately allow you to associate specific surveys with customer accounts).

According to the web site, the SurveyZ architecture is designed to accommodate the "quick and easy" addition of new question types to meet your needs, but you should probably find out about costs before you commit.

Account Management

SurveyZ allows each account to have multiple users, and the account administrator can manage every element of of each user’s access, ranging from their permission to view/edit specific surveys to library access.

SurveyZ also allows users to share resources (such as surveys, questions, images), which is particularly useful in workgroup environments.

System Requirements/Security

From a security standpoint, one element of SurveyZ I don’t like is the fact that all of the surveys in the system (yours and everyone else’s) are sequentially identified (for example, your survey might be #54805 and the next person who creates a survey gets #54806). While internally the system is certainly free to number/identify surveys any way that make sense, using the survey "key" as an external identifier means that other users and respondents can change the number in the URL and thereby view other customer’s surveys.

Other survey systems avoid this problem by giving each survey a unique, alphanumeric public ID that is extremely difficult (if not impossible) to guess.

SurveyZ makes some effort to mitigate this problem by making it possible to password protect your survey so that only those who enter the right code can view/take your survey. However, this is not the default.

The same issue applies if you make the results of your survey publicly viewable. Unless you password protect the results, anyone with enough time on their hands can see your results by simply changing the ID in the URL.

Pricing Structure

SurveyZ offers separate pricing for companies and for academic organizations. Corporate packages range from $49 for 400 responses (12¢ per survey) to $99 for 1000 responses (10¢ per survey) to $7500 for 37,500 responses (20¢ per response). Yes, that seems a little weird — the best price is for 1000 surveys — but there are other differences between the packages that may explain the variance.

For academic users, prices range from $49 for a 500 response "student project" account to a $199, 2000 response "professor research" account to a $5000, 30000 response "department-university" license.

SurveyZ’s corporate pricing structure

Privately branded solutions are also available, and if you’re a non-profit you can even request a "survey research grant" which will give you free use of the service provided that you promote SurveyZ.

Additional Resources

The following resources provide more information about SurveyZ:

• SurveyZ Web Site
• General Product Information
• SurveyZ Enterprise Features
• SurveyZ’s Survey University
• Survey Survey Library
• Corporate Pricing and Academic Pricing

Survey Capabilities

Opinio offers the usual variety of question types, included single/multiple choice questions, drop-down boxes, open-ended questions, essay questions and rating questions. A very flexible "matrix" type question is also available, which allows you to place multiple questions on a line (you might use this, for example, if you wanted to ask customers to rate an attribute and rate the importance of the attribute on the same line).

One neat feature is the ability to include a free text comment below any question you choose. You can also add text both before and/or after questions.

Opinio allows you to include images in questions, and will even let you use images as answer responses (respondents can click the image that represents their response). Opinio does not support ranking questions.

Every question type has its own unique set of validating capabilities (for example, you can say that you want to require respondents to select a minimum of two options, and a maximum of three items, or you can require than an open-ended question be entered as an e-mail address). Opinio makes it easy to include your own custom validation message.

Opinio supports question branching but it doesn’t support answer "piping" or repeating questions. The branching capability is actually quite powerful although in my opinion it is somewhat complicated to implement compared to how some other packages do it.

Opinio does not allow you to pre-populate answers to questions, although it does allow you to rotate the order of the answer choices in a multiple choice question (personally, I prefer to be able to randomize them).

Survey Development

Opinio has a very attractive and functional interface. It also offers lots of configuration options, both for your surveys and for your questions. Based on my experience, web-based systems that allow for a lot of configuration options tend to be somewhat slow and hard to use. Opinio is the exception. Despite the fact that the system lets you control everything about how your survey looks and acts, Opinio does a fairly good job of minimizing the need for lots of postbacks and screen refreshes through the use of smart design and what appears to be a fair helping of JavaScript.

Opinio’s main survey management screen

Opinio does not come with a bunch of pre-written surveys (at least, they weren’t available to my demo account and I didn’t see them mentioned anywhere). I assume that because the package is fairly advanced they’re assuming that most users will want to create their own surveys from scratch?

The implementation of the open-ended/essay question provides an interesting illustration of the flexibility of Opinio. You can insert text and numeric fields anywhere — including in the middle of blocks of text.  And remember: you can use just about any validation scheme you can think of to ensure that respondents insert the type of data you want:

Questions can be inserted in the middle of text.

Part of the control that you get through Opinio’s survey development system comes through the ability to insert HTML whenever and where ever you please. This means that if you want to control the colors, size and look of your fonts it helps to know a certain amount of HTML. And because you can insert HTML any where you choose, it is fairly easy to embed graphics, sounds, animation and videos.

In terms of developmental flow, Opinio makes survey creation relatively quick and painless. It accomplishes this by including all of the most-likely-to-use option for each question on the same page as question you are creating. It also does everything possible to simplify navigation between questions. I did find that it took me a little time to get used to moving about from question without accidentally ending up at the main survey configuration screen, but after I got used to it I didn’t have a problem.

Opinio Question Designer

Rearranging questions in your survey can be kind of a pain. As you develop your survey, all of the questions are displayed in a form that makes it relatively easy to view what your survey is going to look like (which is nice). However, to move questions around you have to use little up/down arrows. If you want to move a question from the top of your twenty question survey to the bottom you need to click the down arrow twenty times (refresh, refresh, refresh, refresh, etc).

In terms of design/organization, Opinio includes a neat feature that allows you group questions into sections. Each section can have its own title, page, and background color and you can even randomize questions within each section.

Opinio does allow you to control much of the look and feel of your surveys through CSS style sheets and templates. CSS style sheets will be of particular use to companies and users who want all of their surveys to look a particular way — once you figure out how to properly set up your own CSS file all you’ll have to do is select it whenever you create a survey and you’ll be good to go.

Templates work differently from the templates you find in other survey systems (where by "template" they really mean "’pre-designed survey". A template in Opinio allows you to set the shell for your survey — basically, so you can embed your survey in the middle of a web page on your site. In other words, all of the navigation that you would find on your regular web site will be available from the survey. Pretty cool, but as far as I can figure, somewhat complicated to set up the first time.

Opinio gives you absolute control over all of the text that appears on your web site — not just questions and the answers, but also all of the buttons and system messages. These can each be customized on a survey to survey basis. Opinio also allows you to create and include a header and footer for each page (you can even save a bunch of headers and footers and reuse them from survey to survey). In fact, Opinio includes an option that allows you to view and edit all of the text used in your survey on one page, making it relatively easy to proofread and edit all of your words without having to navigate the survey question by question.

Opinio offers one of the most integrated systems for dealing with multiple language surveys that I’ve seen. With just a few clicks, you can set up the survey in any language you want, and then have the system send a special link to your translator who merely needs to follow prompts to enter each translated bit of text directly into Opinio. Even the administrative interface can be translated into multiple languages (all character sets are supported) so Opinio might be a good option if your native language is something other than English.

Opinio offers extensive question organization/management features including a library that makes it very easy to save and reuse your favorite questions from survey to survey. The program also offers a unique XML import/export feature that allows you to save questions and entire surveys in an XML format that can then be easily imported into other surveys or even other installations of Opinio.

One more item worthy of note — once responses have been collected it is impossible to edit/adjust the survey without deleting all of the data that has been collected so far. This is a surprisingly inflexible limitation — from my own experience, there have been numerous times when I have needed to go back to a survey to make changes 

User Experience

Although Opinio does not offer a progress bar, it does allow you to include a "previous page" button and you have the option of letting your users finish their surveys later after they’ve already started — in fact, one especially thoughtful feature is that Opinio will e-mail the respondent with a link that they can use to get back to the survey.

Opinio makes it easy to embed your survey in a web site

One of the features that makes Opinio unique is the ability to embed "polls" into your web site and then automatically, immediately display the results to the respondent. While I couldn’t imagine buying Opinio based on this functionality (there are a lot of software packages that do this type of thing for much less money and complexity) it might be an interesting extra feature for someone who already owns the package and has a bunch of extra responses to use up.

Response Management

Opinio allows you to set start and stop dates for your survey, although it doesn’t let you edit and republish your surveys unless you delete any responses that you’ve already collected (very annoying!). Opinio doesn’t allow you to set a quota for your survey, which may be annoying to those of you trying to get the most bang for your buck (fees for use of the ASP service are based on the number of surveys completed).

Opinio has a built-in invitation system, whereby you can provide a list of e-mail addresses (either by entering them manually, by pasting them in from the clipboard, or by uploading them in a file) and it will send out both initial invitations and reminders on a schedule you set. The reminder system is actually pretty straightforward and easy to use.

Opinio does not offer panel management functions.

Opinio will allow you to view and delete individual responses, although you can’t edit/delete responses based on search criteria. On a positive note, you can use this functionality to "reopen" a response and then send a link to a user so that they can finish, although in my experience I’m not really sure how often this will prove to be a meaningful feature.

Opinio has a flexible system to prevent users from completing your survey multiple times, either through the use of cookies, IP address, or invitation codes. Surveys can also be protected by passwords or by unique invitation codes (where a user has to click a link in the invitation they receive in order to access the survey).

The system gets extra points for allowing you to save and report on incomplete responses (not a lot of packages do this, and it can be useful if you design your surveys appropriately).

Reporting Capabilities

Opinio some of the best report customization features I’ve seen. In addition to the basic summary reports (which are created for you), the system allows you to create and save custom reports that let you control every element of what appears, how it is formatted, and how the data is filtered. Although it may take a little longer to build a report than on other systems, the result you end up with will be so much better than just about anything else I’ve seen.

Opinio comes with two pre-configured reports: a comment report and a summary report. These are both great for getting you the "quick and dirty" results that are helpful to view during data collection. You can filter the reports based on the data that the date was collected or by any of the responses to any of the questions, and you can use multiple filters per survey (so that the filtering effect is cumulative).

Opinio allows you to control precisely how your report looks.

Depending on which edition of the software you purchase, you then have the ability to create any number of "custom reports." With a custom report you literally can change any element of how the report looks (using either on-screen controls or CSS), what it contains (you determine which questions are included in the report and how the data is filtered), and how each question is reported on (you can, for example, include summary reports, detailed statistics, cross tabulations, free text comments, and charts for each question if you choose).

You can report on each question in a number of ways.

What I especially like about the Opinio reporting engine is the ability to save and regularly tweak the reports you create. This is particularly useful if you have ongoing surveys (such as customer satisfaction surveys) and you want to be able to run regular, consistent reports on your data without having to download it into an analysis package.

Reports can be published to the internet and set to automatically update on a regular basis — so if you want to make your reports available to respondents or members of your organization, you can.

Opinio offers a variety of data export options.

Data from Opinio can be exported and saved for use in other analysis packages. While the software gives you a lot of control over how your data is exported, there aren’t easy, straightforward options for exporting data into common analysis packages such as SPSS or SAS. Objectplanet does offer an Excel export plugin for Enterprise users.

Extensibility

Opinio has been designed so that programmers can create "plugins" that can be integrated into either the survey experience or the survey management experience. Plugins must be written in Java.

In addition to the plugin interface, Objectplanet publishes on its web site a detailed description of the SQL database used to store the surveys/responses — which makes it relatively easy to suck data into your own applications without having to go through the export interface.

Account Management

Opinio is a multi-user survey system, and depending on the type of license you purchase you can create any number of users. Permissions can be set very granularly and you as the system administrator can decide specifically which surveys and reports you want each of your users to have access to.

The user management system in Opinio is highly flexible, yet relatively easy to manage.

System Requirements/Security

Opinio is available both as a ASP/Hosted service or as software you can put on your own web server. The software is platform independent and runs on all major web servers. Although the system comes with a built in database, Objectplanet recommends that you use a separate database package (any will work, including Oracle, MySql, SQL Server, DB2, etc).

Opinio supports clustering, which means that you can have multiple servers running the same surveys and storing the results in the same database (if you’re planning a big installation). The software is scalable and designed to accommodate high loads.

Pricing Structure

The ASP/Hosted version of Opinio comes in a variety of different levels ranging from "Lite" to "Universal." The ASP version alleviates the hassle of setting up and managing your own server and database, but it is usually likely to cost more.

 
ASP/Hosted Pricing Structure and Options

If you want to install the software on your own server, which in addition to unlimited control over your installation also provides you with unlimited respondents and unlimited reports, is available in two forms: The "Enterprise" edition costs $3,750 and comes with 10 users. The "Corporate" edition starts at $2,000 and is the same as the Enterprise version except it comes with only 5 users and doesn’t allow clustering or multilingual surveys.

Additional Resources

The following resources provide more information about Opinio:

• Opinio Web Site
• Sample Opinio Surveys
• Opinio System Requirements
• Compare Lite, Corporate and Enterprise editions

Survey Capabilities

ResearchExec offers a strong, unique combination of basic and advanced surveying features that will meet both the needs of the basic user and the advanced user. Some of the features I have yet to see anywhere else.

ResearchExec offers the traditional variety of question types plus a few extras. Specifically, it offers both drop-down and radio button single response questions; multiple response checkbox style questions; single and multiple line open-ended questions and multi-line rating questions. The system also offers several "special" open-ended questions where it will validate the responses to be e-mail addresses, zip codes, or phone numbers. Plus it offers specialized date and time fields where respondents can use drop down boxes to select month, days, and times. You can also include constant sum questions (where users are given a certain number of points they can allocate among different choices).

ResearchExec offers smart response validation. In other words, date fields allow you to specify a range of dates that may be entered; numeric fields allow you to select a range of dates. All questions can require or not require a response of  some kind. For most surveys, this is all the validation you will need. ResearchExec also allows programmers to embed additional validation through client-side scripting, samples of which are available from the company.

ResearchExec offers unique implementations of both branching and piping — plus it includes something called "linking" which might prove very handy for certain types of surveys. Branching is what allows you to skip questions or even entire pages based on responses to questions answered earlier in the survey. What makes ResearchExec’ implementation particularly interesting is the ability to set quotas — example: if you only needed 100 men to answer a particular question, you would first ask if they were men and then use the ResearchExec logic to only allow the first 100 men to branch to the question you needed answered. If you met the quota, future men would be directed to another question or a different page. Pretty cool. In addition, ResearchExec can also be set to allow questions and/or answers to appear only when respondents meet specific criteria.

In that same vein, ResearchExec also allows you to set basic quotas for each question — so instead of forcing everyone to answer every question — even after you’ve met your quota — you can automatically pull a question from the survey once the specified number of people have answered it.

Another fancy feature is called "linking" (commonly referred to as "looping"). This is a function that allows you to repeat a question multiple times based on responses to previous question — for example, if I asked you to indicate your three favorite colors, I could then ask — for each color you select — to describe a fruit that matches that color. Although I’ve never had the opportunity to use this feature in practice, it sounds like it could save you a lot of development time.

ResearchExec also allows you to randomize both question and answers. It does this in a way that probably provides more flexibility and control in this regard than any system I’ve seen, but I also have to say that it can be kind of time consuming to implement.

ResearchExec allows you to import respondent answers from either a text file or a URL query string. This can be an extremely useful feature, especially in situations where you are having your respondents respond to information you already know about them (for example, consider a survey that you ask customers to participate in right after they make a purchase from your web site — with this feature, you can import all of the data from the sale such as what they bought, how much they spent, how they paid, etc and then use it in the survey). Imported answers can be used to "fill in the blanks" for guests automatically or even better, they can be used to control branching (so that only respondents who meet certain pre-defined criteria will be asked to answer specific parts of the survey).

Overall, ResearchExec offers some of the most advanced survey features I’ve seen in any of the packages I’ve looked at so far or had the opportunity to use in the past. If you’re interested in doing complex, interesting things with surveys without the need to learn a programming language or dig into a software development kit, ResearchExec may be for you.

Design/Development

ResearchExec offers a sophisticated web-based survey design/management system that provides access to numerous questionnaire and design options. The price of all of these options (complexity?) is that I did run into a few technical issues and found it to be quite time consuming to build and tune a survey.

When you create a new survey in ResearchExec, you’re first decide whether you want to start from a blank survey or from one of sixteen pre-written templates from one four categories (customer satisfaction, demographics, HR surveys, and Internet Related Surveys).

Although I did find the interface to be nicely designed from both an aesthetic and functional standpoint, I did have some technical issues. For example, the WYSIWYG editor (used to add pictures to questions or change fonts) didn’t work so well in Internet Explorer 7 RC1, and I couldn’t get anything to work at all in FireFox 1.5. Whether the problem was with my machine or limitations in the software I do not know, but make sure you test the program on your system before you invest (note: according to the company, ResearchExec does not support FireFox but it will have full support for IE7 by the time the final version is released).

…and while the software gives you access to a lot of advanced features that can be used to control precisely how your survey looks and acts, developing a new survey from scratch can take a very long time. Lots of clicking is required. Lots of screen refreshes are required as well. I’m not sure whether this is simply an unavoidable fact of life that comes from having such a feature-ful survey development system or whether it too closely tries to model the experience of developing a survey in a client-based machine — or if it is simply the result of less than thoughtful design. However, creating and editing a simple question like, "What is your favorite color?" followed by three different randomized options took (in my opinion) far too long. Even worse, try creating a list of list of twenty different colors and manually moving a color at the bottom of the list to the top of the list — it takes forever! (note: the company expects to have this issue resolved within the next couple of weeks).

With that said, ResearchExec has the ability to import MS Word documents that are prepared in a specific format and turn them directly into questions. According to the company, most of their users put surveys together in this way and I expect it speeds things up considerably.

ResearchExec does offer a lot of options to help you configure how your survey looks and feels. It allows you to pick from 27 pre-configured design themes or create and save your own — which means that after you come up with a look that you like it is fairly easy to re-use it over and over again. This is a nice feature — and, given the time that this will save you it does make up for to some degree some of the issues described in the previous paragraph (sometimes it can take a lot of time to get your survey to look just right). You can set the default fonts (face, size and colors) used for questions, answers, buttons, headers, and errors — plus you can add a header and/or a footer to each page.

In addition to controlling the default design, you also have a lot of flexibility when it comes to designing individual questions. For example, in addition to changing the font, color, and size of the text of each question (or answer!) you can also insert images in just about any text box. This can be an extremely useful featured if you’re doing new product development research, branding research or just about any kind of research where a picture is more meaningful than words. To that end, because you do have precise control over the HTML used in each question and answer, you should have no problem including Flash, video, sounds, animations, etc in your surveys.

As one would expect from a system that allows branching, piping and skipping, ResearchExec does allow for multiple page surveys and you can control which questions appear on each page.

ResearchExec offers a well implemented, easy to use question library that should go a long way towards helping you standardize your question base (another feature that makes up for the slowness of the design process, since once you create a question the way you like it, it becomes very easy to insert the question in any survey you develop). Basically, questions that you add to the library are added to the drop-down list of question-types so they are readily available, right in front of your face during the question-building phase. Very nice.

ResearchExec offers a preview function that allows you to test out your survey at any time. This is particularly important in the case of ResearchExec because after you finish your survey, you have to submit it to be approved by ResearchExec before it is then moved from the "development" server to the "production" server. This "approval" process is not something I have seen from other vendors and is somewhat surprising…it slows down how quickly you can get your surveys into the field (and there have been plenty of times when I have needed to get things up and running immediately in order to meet a deadline. The company says that in actuality, regular customers are allowed to bypass this approval process and that for others the delay can sometimes be as little as a couple of hours. Although beware — according to the web site approval time can take up to 48 hours.

To make it even worse, if you have to make a change to your survey later (because you forgot to add a question, or to fix a typo, or maybe even clean up a bit of branching logic that isn’t working right) you have to resubmit your survey for approval again before the change is made.

User Experience/Response Management

Once your survey is activated (after it has been approved by ResearchExec) you can control when the survey will end or you can set a limiting quota. One neat feature that I’ve found only in ResearchExec is the ability to control what happens to the user if they try to take the survey when it is closed. You can either have it show the default system message, a message of your choosing, or you can redirect them to a web page of your choosing.

ResearchExec allows you to include a progress bar on each page as well as the necessary "continue" button that is used to move on to the next page. It also gives you the option of whether or not you want to include a "Previous" button so that users can go backwards. Surveys can also be configured so that users can save their answers and come back later to finish or make changes.

ResearchExec does offer the ability to monitor your survey’s statistics, such as the number of people who have started the survey, the number of people who have finished the survey, including the number of people who were screened out. You can also monitor the degree to which each of your quotas are being met.

The system will allow you to eliminate specific responses through the reporting system. You can also use the filtering tool within the reporting system to eliminate batches of responses that meet a specific criteria.

Panel Management

One of the elements of ResearchExec that I like most is the panel management functionality. There are many survey systems that will maintain a "panel" of respondents for you — which basically means you can upload your list of e-mail addresses into the software and you can then use it to mail out invitations to possible recipients for multiple surveys. What makes ResearchExec unique is the fact that it will allow you to import your people and then build a history for each of them based on their responses to the surveys you give them. In other words, ResearchExec allows you to choose people to receive specific surveys based on their responses to earlier surveys. That way, once you find out that a guy is a guy, you don’t have to waste his time by sending him surveys about cosmetics and feminine products.

The other panel management feature that ResearchExec includes is the ability to keep your e-mail list fresh by automatically de-activating respondents with non-working e-mail addresses.

ResearchExec also has the ability to send reminder messages to members of your list that have not yet started or completed your survey.

Reporting Capabilities

Based on the extremely limited reporting/charting capabilities of ResearchExec, I’m going to assume that the designers felt that most users would want to analyze their data using professional analysis software rather than in the software itself. Yes, I appreciate the ability to generate frequency tables for each of my questions, but I had kind of expected to be able to do at least a little analysis in the software. ResearchExec does not allow you to generate any charts and graphs that can be exported into presentations;  it doesn’t allow you to crosstab the data; it doesn’t allow you to create reports that include only specific questions.

It does allow you to filter the data based on questions in the survey. It also allows you to export the data into CSV and TXT files which can then be imported into other applications. You can also e-mail the basic frequency reports to yourself or others. But that’s it.

If you’re the type of person who doesn’t want to use an outside package to do analysis and simply want to be able to create your reports/charts in the survey software, then you definitely want to find a different survey package. However, if your sole focus on using a package such as this is to collect data for use in other analysis/reporting systems, then the lack of built-in reporting functions probably won’t be a consideration in your decision process.

System Requirements/Security

ResearchExec is available in four flavors: an "Enterprise" version that runs on your own equipment with all of the features the software has to offer; a "Professional" version that also runs on your own equipment but that doesn’t include advanced survey logic, full e-mail management, or panel management features; a "Managed ASP" version which runs on ResearchExec’s equipment, offers a limited number of administrative accounts and "Classic ASP" version that offers limited only one administrative account, limited survey logic, limited e-mail management and no no weighting/scoring capabilities.

ResearchExec does allow you to have multiple accounts and the security of each account can be individually controlled to allow access to only specific objects.

If you want to run either the Enterprise version or the Professional version, you’ll need Windows 2000 Server edition (or greater, I assume); Microsoft Internet Information Server (free) and Microsoft SQL Server 2000 (definitely not free, but you might be able to use an account on an existing server).

Vendor Related Notes

ResearchExec tells me that they’ll be releasing version 7.0 of ResearchExec in the 1st quarter of 2006 which will be completely redesigned and redeveloped,

Worth noting: the most recent press release on the "Current News" page of the ResearchExec web site is dated August, 2003.

To learn more, visit http://www.researchexec.com.

Survey Design/Development

Grapevine users can develop their own surveys from scratch or can start from one of a few dozen pre-written survey "templates" from a variety of categories including "customer," "human resources," "marketing/sales," "products," "retail," "B2B," "board [of director] surveys," "community social," "education," "health care," "web polls," and "web site." Thumbnails of each of these starter templates make it easy to get a sense of both how the questions are organized and the design motif. These templates (of which there appear to be at least a hundred) are in and of itself a reason to explore the Grapevine survey system — especially if you don’t have a lot of experience writing survey questions. Even if you’ve written plenty of your own surveys, you may find some of the questions provided in the templates useful.

Whether you develop your survey on your own from scratch or use one of the template, you will very quickly find yourself in the Grapevine question editor. What I liked about the editor is that it makes it fairly easy to format the text of your questions any way you choose (font, size, style — plus the ability to make questions as long as you need them to be). I also like the fact that it provides a lot of help text and variety of ways you can customize each question (for example, if you want you can display an open-ended "comment" box under whichever questions you choose). Sample questions can be viewed on the Grapevine web site.

Grapevine offers the usual variety of question types, including closed-ended single selection, closed-ended multiple selection, rating scales, open-ended text responses, and even multiple answer grids (which allow you to have several different drop-down or checkbox questions on each line of the matrix). It also offers constant sum questions (where people fill in numbers that have to add up to a number you set) and special date fields (it shows drop down boxes for month, day and year). Grapevine does not offer ranking questions or other advanced question types.

I don’t like the fact that the software has to refresh/reload the page every time I change the question type or even any of the options for each question type. For example, if I try to do something as simple as changing the number of answer options in a question from 3 to 5, I first make the change in the drop down box (from "3" to "9") and then I have to wait a few seconds before the page is redrawn with the nine answer boxes I requested. While a few seconds isn’t a lot, when you consider that I have to change several options for each question (each with its own wait time) and that I have to make lots of questions, you can imagine that the annoyances would start to add up. It breaks your flow — throws off your rhythm…and it seems unnecessary in an day and age where Javascript and AJAX make such waits unnecessary.

The same is true when it comes to question management — yes, the software makes it pretty easy to change the order of questions and to move questions from one page to another — but at the same time, each change requires a wait that could be extremely annoying if you had to rearrange a lot of questions.

Grapevine does allow you to upload images and other documents that can be included with each question (images, such as JPG files or GIF files are displayed instead of the question — other files, such as Word, Excel, or PDF files appear as links that can be opened below the question text) however it only allows you to include one image per question, and you have to upload each image with each of the questions that you add.

Grapevine offers some flexibility in terms of the overall look (design) of your survey by providing you with 38 "graphic skins" (style templates) which are split among several categories ("aggressive", "artistic", "business", "collaboration", "education" and "simplistic"). Almost all of these templates are designed so that you can include your own logo at the top and some of them are quite creative and may just do the trick for those of you who want your surveys to look good but don’t want to spend a lot of time customizing the look and feel.  Graphic skins, incidentally, are not customizable from the interface — if you want something different what what is available you can hire Grapevine to create a customized skin for you.

Grapevine offers both page/question skipping and piping. Both of these functions only work when you use multiple pages (which is fairly normal).

Data Collection & User Experience

The sample surveys on the Grapevine website are for the most part very attractive and illustrate what the user experience can be like. In my experimentation I was unable to create surveys that look as attractive as the sample surveys, although I have to admit I didn’t spend a great deal of time trying. One thing I think that is particularly interesting about the way in which Grapevine asks questions is that it doesn’t use the traditional radio buttons and check  boxes but instead uses a combination of images mixed with custom JavaScript to provide a more unique appearance and apparently allows Grapevine to assert more control in the design of the controls. Whether or not it improves the survey experience is something you will have to decide for yourself.

For multi-page surveys, Grapevine provides progress displays, "previous" and "next" buttons as appropriate. Surveys can include both introductory messages and thank you messages.

Users can save their responses as an Adobe Acrobat PDF file and then print them. A feature apparently unique to Grapevine is the ability of respondents to forward their response to friends and coworkers via a form on the "Thank you" page.

Another feature apparently unique to the Grapevine Survey System is the ability to have the system send out e-mail alerts to specific individuals when respondents answer questions you specify in certain ways. For example, if I asked the question, "do you plan to break a law today?" in my survey, I can set it to automatically e-mail the police if someone answers "Yes."

Grapevine has the capability to save incomplete surveys so that users can come back and finish later provided that you use the respondent account management system.

Grapevine does not feature the ability to maintain a library of questions and answers that you can reuse from survey to survey. Nor does it make it easy to copy questions from one survey to another. It will, however, allow you to make copies of existing surveys and reuse them.

Response/Respondent Management

Grapevine allows you to set up all of your potential respondents with "accounts" which can either be entered individually (ugh!) or uploaded using a CSV file (much easier). You can then assign each of your respondents to groups which can then (as a group) be sent customizable invitations to participate in your survey. You can then at any time you want pull up a list of those who have completed the survey and those who have not.

In some ways, the Grapevine respondent account system functions as a rudimentary panel management system since it allows you to put people into groups, each of which can receive different surveys. If you’re actually looking for a survey system that includes panel management feature then this is not going to be a good alternative for you since each individual must be manually assigned to each group and you don’t have the option of sorting and selecting by demographic or behavioral characteristics.

Grapevine allows you to set a launch date and a finish date for each of your survey, as well as the maximum number of responses you want to collect. You can also set a password that respondents have to know before they can have access to your survey.

Grapevine does not save incomplete survey responses.

Reporting/Exporting Capabilities

There are a lot of positive things to be said about the Grapevine reporting system. I really like      that by pushing the "Overall Results" button you are instantly presented with frequency counts, percentages, and even basic bar graphs for all of your questions. This makes it relatively easy to get an overview of all of your results fairly quickly without having to run a lot of separate reports, and the presentation is thoughtful.

I like the fact that from this same overview report it is possible to generate pie charts, bar charts, column charts and line charts with the push of a button. Granted, the quality of lack of customizability of each of these charts means I would probably never copy them into a report for my boss or for a client, but it is a useful feature nonetheless.

I also like that the software provides you with the ability to easily review individual responses, and that it will let you query/filter your results based on responses to specific questions. Plus, you can even run crosstabs on any closed-response question. It even lets you limit your responses to those that are completed during a specific period.

One specifically unique feature of the Grapevine reporting engine is the capability to create something called a "construct grouping." My understanding of this feature is that you can combine the responses from several rating-type questions together to generate an overall score. So, for example, if you were a restaurant and included several rating-type questions about the quality of the food, you could then have the system automatically combine each of the ratings to create a kind of "overall score."  It’s an interesting idea that some users may find very useful (and interesting). On the other hand, I do have some concerns because it doesn’t let you control how each of the attributes are rated, nor does it really explain how they are combined. A feature that might be interesting in the future is to add the ability to weight the data.

Grapevine does give you the ability to export your results to Excel or a PDF file and the report that it generates looks pretty good. You can even limit which questions appear in the report. Data can be exported in CSV format with a couple of options to help you import it into SPSS or SAS (although note that it will not export your data directly into either of these formats).

Given the scope of the overall package, I don’t have a lot of complaints about the survey engine. I do wish it would remember when I tell it to hide the results of specific questions so that I don’t have to go back and do the hiding manually every time I want to print a report. I also wish it were possible to tell the software to show the verbatim responses when I go to print the report (as it stands now I have to go in and manually tell it to print the results of each open-ended question separately). 

More advanced users will miss the fact that it does not offer any survey stats such as the percentage of users who dropped out of each page, how long it took to answer each page, etc. Also missing is the ability to "publish" your reports so that you can dynamically share the results of the survey with co-workers.

Survey/User Administration

Grapevine allows you to create as many surveys as you want and have them all collect data at the same time (Grapevine charges you based on the number of people who complete your survey and not by the number of surveys you create). Surveys can be edited and re-edited as many times as you like, and you can copy surveys for re-use as "new" surveys. One neat touch is that you can even save your surveys in "folders" to keep them organized.

One especially well-implemented feature of Grapevine is the ability to add multiple "users" (survey administrators) to your account. Each person you give an account to can be allowed to view reports, edit surveys, or even create surveys — depending on their access level. Users can even be given permission to edit or view surveys on a survey by survey basis (which is probably better than giving users unlimited access to every survey on your account).

Help/Technical Support/Services

In addition to the variety of templates and examples provided directly within the context of the survey development system, Grapevine also offers 24 hours e-mail and telephone support. In addition, Grapevine offers survey management services (where they will take your questions and create the surveys for you), question writing services, and training services. Grapevine will also create custom survey designs for you and can even provide you with respondents (using reputable survey panels) if you need them. Grapevine will also provide survey translation services.

Pricing

Pricing for the use of the Grapevine service is based on the number of your surveys that are completed and ranges from $400 for one hundred responses ($4 each) to $80,000 for half a million responses. Fees are charged to your credit card, which they will keep on file. A Grapevine Survey System subscription is twelve months.

Incidentally…at the time of this writing (Sept, ’06) Grapevine is offering a free trial which allows you to play with the design/reporting interface and collect up to 25 responses.

To learn more about Grapevine visit their web site at http://www.grapevinesurveys.com.