Comments on survey design

Because this survey (on “sexual and asexual relationship dynamics” from Ball State University) did not have any option to leave comments on the design of the survey and what the questions were supposed to mean at the end, I’m going to just leave my comments here. I started copying and pasting questions into Notepad somewhere in the middle of the survey, so these are only some of the issues I had with this survey. I surely have forgotten others. At the end, I will mention the way the survey handled consent, but I’m mostly not focusing on that.

I want to preface this by saying that I am really annoyed by MOST surveys, I just don’t typically have the time to comment on them like this, and when there is an option to share comments about the survey within the survey itself, there is usually no need to share those comments publicly. This survey is not even remotely exceptional or surprising. More discussion of asexuals’ responses to academic surveys can be found in a fairly recent Asexual Agenda question of the week. I hope that people who research asexuality consider these problems when designing surveys in the future. Honestly, these are mostly problems that testing with a focus group could have helped iron out. It is very frustrating that these issues don’t ever seem to be corrected before the surveys are sent out.

Survey asks:

“Please indicate the physical attractiveness of your partner”

How am I supposed to know what THAT means???? It is not specific enough about whether it’s talking about how she is perceived by other people in general, or my personal sense of whether or not she is “physically attractive” (a phrase I am not 100% sure I understand even in the case that it’s about my personal level of attraction to my partner, since it is often used as a euphemism for “sexually attractive” but can also be used to talk about whether you want to cuddle).

Survey asks, about polyamory, for me to rate my level of comfort with this:

“You and your partner may form outside romantic relationships, but they must always be less important than the relationship between the two of you.”

I don’t like this survey’s handling of poly relationships. Why is this ONLY listed with a caveat? Why isn’t JUST the first part listed? There was another similar statement, but that one also had a caveat. My partner and I can form outside romantic relationships. Period. We do not need to set a limit on those relationships and pre-determine that they must “always be less important” than our relationship. So far it has worked out that way—it’s hard to beat a relationship that’s been going for as long as ours, especially when your dating prospects are pretty slim. But there’s no reason that ALWAYS has to be the case, especially since my partner is aromantic and I’m greyromantic (I don’t recall there being space to list grey- or quoiro, so I just said “prefer not to answer” for the romantic orientation question).

Survey asks me to rate level of agreement with these statements:

  1. In most ways my sex life is close to my ideal.
  2. The conditions of my sex life are excellent.
  3. I am satisfied with my sex life.
  4. So far I have gotten the important things I want in sex life.
  5. If I could live my sex life over, I would change almost nothing.

The last 2 questions suddenly switch from current sex life to lifelong sex life, the way that I read them. It’s kind of confusing and awkwardly worded, so I’m not sure that was intended? Also it doesn’t really ask about what my sex life is CURRENTLY like, anywhere in the survey. It only asks in a “have you ever tried…?” sort of way. And I think it asked something about “are you likely to have sex in the next month” or something like that? So… kinda weird to ask about that with no context given for it. I want to note that what my sex life is like has varied DRASTICALLY at different times in my life. So has my “ideal” sex life.

Survey asks me to imagine the following scenario and then rate how likely I would respond with specific emotions to it:

“Imagine that you find out that your partner has fallen in love with another person. Your partner has become deeply emotionally attached to this other person, and thoughts of this other person consume your partner’s every thought”

This is very hard to imagine, since my partner is aromantic. And… how am I supposed to know how I would react to that if such a thing happened? At best, I’m making guesses. And again, I’m annoyed at what seems to be a presumption that her falling in love with someone else is not allowed. This is treated exactly like it’s cheating, but it’s totally okay within my relationship—which I couldn’t actually note anywhere in the survey.

“Imagine that you find out that your partner has been having sex with someone else. Imagine your partner trying—and enjoying—many different sexual positions and activities with this other person. Imagine that most of your partner’s waking thoughts and fantasies are about having sex with this other person.”

Okay, now this implies that my partner has been cheating. Since, you know, in a poly arrangement like ours, you wouldn’t suddenly “find out” that your partner “has been” having sex with someone else—she tells me beforehand if she’s even vaguely considering it, even when it’s extremely unlikely to actually happen. I guess I’m just confused about why it’s not explicit about the “without telling you” part.

And uh… MOST of her waking thoughts and fantasies are about sex with someone else? Why? Why is that the assumption I’m directed to make? It seems uncharacteristic for it to take up THAT much space in her life. And frankly, I’m suspicious whenever anyone makes the assumption that sexual fantasies (about one specific other person) take up such a disproportionate level of thought/fantasy life—for most people, most of the time, I strongly doubt that’s the case. So why, exactly, should I be primed to think of it that way?

And lastly (bolding added):

You indicated: “I am currently in a romantic/sexual relationship ” and you selected the following relationship type: “Polyamorous- A relationship with more than one person and this is an agreed-upon arrangement”

How likely do you think it is that your asexuality played a role in your relationship ending?

What?????? It hasn’t ended, wtf??? I already told you that in an earlier question? There was no “N/A” option for this question, nor was I directed to guess at how likely it might be to play a role in the relationship ending. I had to answer neutrally, since that seemed least likely to throw off the answers.

I just don’t understand why that wasn’t corrected before this survey was sent out.

Next, I’ll talk about consent, so if you’re not up for that discussion, feel free to peace out now.


 

I will say, at least, that this is somehow one of the better surveys I’ve taken when it comes to handling issues of consent (usually surveys are downright abysmal about it, so that isn’t saying much). It asks about how many “voluntary” sexual partners you’ve had in your lifetime, rather than leaving it 100% unspecified as to whether they wanted you to include non-consensual partners or not (although, you know, some people don’t even know whether a given partner counts as a “voluntary” sexual partner or not—gray areas of consent exist and gaslighting makes everything even more confusing). However, there is still (of course) a lot of eliding the full range of issues of consent. This part of the survey happened before I started noting problems, so I don’t recall exactly what the wording was, but among reasons for having sex, it listed something like “physically forced” and “verbally coerced.” There are plenty of non-consensual situations that don’t fall into either of those categories, though. I wish that cultural coercion and… I don’t even know what to call it, but maybe… non-verbal, even non-active (passive), coercion within the context of an abusive relationship, where you’ve been groomed to agree to having sex even if you don’t want to, and to not think of yourself as asexual… I wish that space would be given for those types of experiences. Not just in this survey, but in ANY survey.

At least there was sort of an area where you could provide your own response about reasons that you’ve had sex—but since it was a VERY SMALL text box that could not be resized, it was very difficult to provide a substantive open response. I felt that discussion of other reasons for having sex were structurally discouraged by the tiny box. It should have been a TextArea instead. Seriously, there are hundreds of reasons why people might have sex. Of course a tiny text box isn’t going to be able to cover that.

Sigh. It’s just… Okay, I’m done.

10 thoughts on “Comments on survey design

  1. I don’t get this sort of thing at all. These people are academics, researchers – how can they get things so screwed up and irrelevant and weird? Isn’t it their job to do these things right, or even just with a bit of forethought? I was thinking of doing that survey, but not anymore. Especially the poly-ish questions are just missing the mark entirely.

    Liked by 1 person

  2. The scenarios where your partner falls in love or has sex with another person are not presumed to be cheating. The question asks about which emotions you might feel, and you are free to say that you don’t experience the negative emotions listed, and that you do experience the positive ones.

    Like

    • I think you’re misunderstanding my point? Let me try to explain again. The scenarios are treated the same way as one another in the survey (for those who haven’t taken it, they’re literally asking the exact same questions for both), and I said that the second scenario would be considered cheating in MY relationship. The first one wouldn’t. The difference about whether one is cheating and the other is not is very significant for me, and that they are treated the exact same way plays into narratives of cheating and jealousy in a way that doesn’t really apply to me—and I don’t get a chance to challenge that, so the difference would go unrecognized. I don’t particularly care whether the survey creators thought that both would consist of cheating or not, but that it asks about these negative emotions without asking ME whether I would consider these scenarios cheating or not, uh… just REALLY elides the complexities of relationship dynamics. I do not feel that poly relationship dynamics fit well within the parameters of the questions that were asked. If the goal was, as stated, to examine relationship dynamics, I think they missed a key part of it by ignoring that factor. So what I’m saying is, the context matters, and they didn’t bother to gather any data about it.

      And honestly, I do not understand the point of asking me to predict potential emotional responses to these very specific scenarios, when neither is even remotely likely to ever happen in my case. It is so far removed from my experience, how could I possibly know how I would react? What bearing does any of this have on my ACTUAL relationships’ dynamics? And why this (very specific and kind of… extreme?) way of framing everything? This feels like being forced into talking about a very contrived, speculative scenario based on a monogamous jealousy narrative. That’s what I object to.

      “you are free to say that you don’t experience the negative emotions listed, and that you do experience the positive ones.”

      That… doesn’t really address what I’m talking about though? I’m confused. Whether I would have any negative emotions or not does not indicate whether it’s acceptable or unacceptable behavior within the relationship.

      Like

      • Well I am still missing your point. I mean, if you saw the first and second scenario as very different from each other, doesn’t that only justify asking them as separate questions? If many respondents say they don’t have strong negative emotions, doesn’t that, in fact, overturn narratives about cheating and jealousy? If the question instead asked about whether the behavior was acceptable, *my* responses would have been uninformative, since my relationship is monogamous, but it’s more to do with his feelings than mine.

        I do think it’s weird that they felt the need to make the scenario so strong, and it seems like they should have posited that the partner tells you what they did.

        Like

        • “I mean, if you saw the first and second scenario as very different from each other, doesn’t that only justify asking them as separate questions?”

          Okay, maybe this is the problem? My issue ISN’T that they asked them as separate questions! That’s justified, sure. It’s that they asked questions about just these two scenarios, and nothing else—and the way it was worded was so awkward that it necessarily implied a cheating scenario in one but not the other (at least in my situation). And the way they’re treated, both scenarios have nothing to do with my real relationships.

          “If many respondents say they don’t have strong negative emotions, doesn’t that, in fact, overturn narratives about cheating and jealousy?”

          This strikes me as a misunderstanding of poly and jealousy, here. Poly people often DO get jealous and have negative emotions in situations like the ones described, especially when they’re set up to be that extreme (even if these scenarios are not considered cheating, management of priorities and quality time between multiple partners could be a big issue). I don’t think you would find significant enough differences in the amount of negative emotions poly people describe to challenge the narrative set up by these questions. The difference is more about how we respond to those emotions when they come up, what we consider acceptable or unacceptable limits to put on our partners (which would not be the same for all poly relationships), and just generally how we manage those situations. All of those different dynamics would be completely ignored by this survey.

          I dunno, it’s like they tried to make some sort of attempt to address and include poly people, but they really didn’t understand poly well enough to do it all the way. I think there were like four questions about what kind of poly dynamics might exist in the relationship (I mentioned one of them), and all of them were really limited. The rest seemed to be very biased towards an assumption of monogamy, without really addressing or even considering important poly relationship dynamics. And they directed people to only talk about one “primary” relationship, so they’re excluding a LOT of poly people who don’t have those as well as ALL of the dynamics of other relationships.

          I’m really just not convinced that useful data will come out of this survey because of all of that.

          Like

          • Yes – the whole “If you’re poly you obviously have one single ‘Primary’ partner” thing made things really difficult. I ended up leaving most of those blank. And there were so many questions that just didn’t compute for me, or weren’t coherent in an ace/poly context. My biggest complaint was that they didn’t give me a place to complain (hem – ‘leave constructive criticism’). Having no room for feedback on surveys really annoys me.

            Like

  3. I agree with you wholeheartedly that for a survey that seemed to realize that polyamory exists, it still kind of assumed you weren’t poly.* I, too, felt dissonance in several sections, and I also didn’t know whether those scenarios they presented were supposed to be okay within my relationship or not. Like, are they leaving me? That’s really what I need to know here. This is really surprising to me considering that they clearly know that polyamory might be a big factor in what they’re trying to research.

    I also had a lot of trouble the section that asked me to rate how desirable I was a partner, both in my own opinion and other people’s?? That’s so wildly dependent on the person making the determination? How are you supposed to generalize that?

    Basically, I just 100% agree with you that it was a very difficult survey to navigate.

    *In certain sections it seemed to take your answers into account, and my last relationship was a closed one in practice, so I suppose the questions could have extrapolated from there except–nope.

    Like

    • I also had a lot of trouble the section that asked me to rate how desirable I was a partner, both in my own opinion and other people’s?? That’s so wildly dependent on the person making the determination? How are you supposed to generalize that?

      Omg, yes, I had so much trouble with that part, too. Like, I’m told that I’m a “good catch” sometimes but I don’t feel that way? And whose word exactly am I supposed to be taking? What criteria would you even use to determine that kind of thing? It seems like it would be very different depending on what aspects people value most. I just don’t understand what they were going for with that section.

      Like

      • I thought maybe they were trying to get at the self-confidence of the people taking the survey? Or, at least, that’s the conclusion I came to once they asked if I was a “good catch.” The problem is that desirability is very, very different from being a “good catch.” Which, unsurprisingly, is a tension that is is blindingly apparent to me as an asexual person. If they were trying to get at confidence, they probably would have done better to be a little more direct.

        Like

Comments are closed.