Suggestions for evaluation survey design

For an overview of the evaluation stage of a ParEvo exercise, go here.

The purpose of this page is to provide a list of types of questions that might be asked in a survey of people’s views of the storylines generated by a ParEvo exercise. Please treat this list as a flexible menu, not as a complete and necessary package

The respondents

Firstly, bear in mind that there may be at least three different kinds of survey respondents, and the membership of these types may overlap to different degrees in different exercises.

  1. Contributors, people who contributed paragraphs to the storylines
  2. Commentators, people who commented on various paragraphs within the different storylines. These comments were then made visible next to those paragraphs
  3. Observers, people who could view the storylines and comments, but could not leave any comments or contribute any paragraphs

2021 07 15: Up to now most of the questions listed below have been written assuming the respondent was a contributor, clearly there is need for more diversity yet. I also seem to have assumed that an evaluation will be done via an online survey, which is clearly not the only option. More work needs to be done on this menu!

Types of questions

  1. Questions about specific contributions
    1. Which individual contribution did you think was:
      1. : most likely, least likely, most desirable, least desirable,
      2. most equitable, least equitable, most sustainable, least sustainable.
  2. Questions about specific storylines
    1. Closed ended questions
      1. Which storyline do you think was:
        1. most likely, least likely, most desirable, least desirable,
        2. most equitable, least equitable, most sustainable, least sustainable.
      2. Which extinct storyline do you think had the most undeveloped potential?
    1. Open ended questions
      1. What is the most significant difference between the storylines”
      2. What difference has or will this difference make?
  3. Questions abut the whole set of storylines
    1. Surprise
      1. What was most surprising about the content that was present, in any of the storylines?
      2. What was most surprising in terms of its absence from any storylines?
    2. Agency
      1. How much do you think the events in the storyline are likely to affect your life?
      2. How much do think you will be able to affect the events described in the storylines?
  4. Questions about participant’s judgements versus that of others
    1. Other people’s behavior
      1. How did you feel when two or more others added their contributions on to any contribution of yours?
      2. How did you feel if and when nobody added any contributions on to a contribution of yours?
    2. Optimism
      1. How optimistic were your contributions?
      2. How optimistic were the contributions made by others?
  5. Questions about the design of this ParEvo exercise
    1. Were there too many/too few iterations?
    2. Was the time available during each iteration too long/too short?
    3. Was the guidance given at the start of each iteration clear?
    4. Overall, was the whole exercise: enjoyable, hard work, confusing, informative, fun, exciting, depressing…
    5. As a reader was the 150 word limit per contribution too long or too short or neither?
    6. As a writer, was the 150 word limit per contribution too long or too short or neither?
  6. Questions about consequences for what happens next, after the exercise
    1. Prospective
      1. Will you be doing anything about any of the potential risks highlighted by the storylines? (=likely but undesirable events)
      2. Will you be doing anything about any of the potential opportunities highlighted by the storylines? (=unlikely but desirable events)
      3. What sort of changes might there be in the way you will be doing things, that have been prompted in any way by your participation in the exercise?
    2. Retrospective
      1. What has been the most significant change that has taken place since the ParEvo exercise, which was influenced, triggered or caused by your participation in the exercise?

Leave a comment or ask a question

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.