Engagement Survey Results

February 6, 2010    By: Matt W. @ 11:14 am   Category: Life,Mormon Culture/Practices,Uncategorized

Firmographics


First, we ran the survey from February 2-February 6 and had a total of 218 Responses. 4 responses were unusable due to bad data, which left 214 responses. The survey was only presented through this blog, and all respondents came to the survey via this blog, so we can safely say the responses of this survey represent the readership of Newcoolthang.com.

Second, as you can see from the results above, most responses to the survey were within the first 24 hours. This tells us that the majority of the readership of newcoolthang.com probably check their source for this website’s material at least daily. So thanks for your interest!

Next, we can look at where the computer thinks you are based on your IP. While it is flattering that we have readers in Australia, the UK, the Middle East, Europe, Asia, and Canada, the vast majority of the respondents were in the USA, so we have to assume any church engagement findings we discover will be US-centric.


Do you consider yourself a member of the Church of Jesus Christ of Latter-day Saints?

The first question was meant to be a catch-all protection against members of other churches skewing the results of the survey. Perhaps I needn’t have worried, as only 3 respondents answered that they did not consider themselves members of the church, and their responses lead me to believe they still attend church. Also, in hindsight, I realize that wording may have excluded excommunicants, so if I were to do this again, I’d word the question “Have you ever considered yourself a member…” rather than “Do you….” In any case, the readership on NCT is 99% members, and those who do not consider themselves members were baptized, so I chose to include that 1% of respondents as well.


How Active do you consider yourself at church?
This question weighted the response with points, 5 being the most active and 1 being the least. As you can see, NCT readers are very active, with 92% of the readership attending church more than once a month. 4% more attend at least once a month, with the final 4% attending less than that. For the sake of this survey, I am going to consider the top 96% “More Active” and the bottom 4% “Less Active”. More on that later.

How old were you when you were baptized?

86% of the respondents were baptized at age 8. For the sake of this study I am considering all of them (and my two 9 year old baptisands) “Life-Long Members” as opposed to “Converts”. Rather than show the rather boring chart of lifers vs converts here, I thought it would be more interesting to pull out converts by age.

(Perhaps an interesting point of further study would be to go flesh this out a bit in another survey, as it certainly suggests that the largest success area in the US for missionary work are teens aged 16-19. However, since we have no idea of knowing if this is exclusive to “bloggernacle” types, or if this is high success rate in this area is due to more attempts or more opportunities.)

Anyway, with our caveats in place (Mainly US, Mainly Active Members, Mainly Life-Long Members) We can continue.

Results
Since these are weighted results (1 being most negative and 5 being most positive), the easiest way to handle them is to take the average response for each question. For the whole set, this looks like:

I’ve ordered the responses by score. Generally speaking, anything above a 4 is a good score, and anything below a 3.5 needs work. So looking at the above, we can see that the church is doing well in terms of setting expectations, providing materials for callings (this was surprising considering all the fuss about lesson manuals lately), and having leadership show sincere caring.

On the flip side, the church has it’s work cut out for it in working on “Thinking to Thank”, Fitting the Calling to the individual, friendshipping at church (the low score here totally surprised me at first, except then I looked into it, and this is the lowest score for most corporations also. This may be due to a too-literal understanding of “best friend” or thinking of church as local only. I’d love feedback on this). Finally, the absolute lowest score went below 3, which I would consider in the danger zone. As you can see, it was the question of whether someone at church asked you about your progress in the last 6 months. In our local ward, we often see converts drop away around the year mark, and we often see teens drop away shortly after becoming adults. I suspect that this question could have a correlation to those results, in that new converts and teens are highly evaluated, and perhaps going from a period of high attention to low attention connects well to loss of members.


I almost did not include the above, because the sample size for “less active” is so small, but I did want to highlight one result, even if the sample size makes it tentative at best. It is this. For the question “Are your associates (fellow church-members) committed to doing quality service?”, The “Less Active” scored much higher than the “More Active”. This is especially interesting because in all other categories, we find what we would expect, that those “less engaged” in going to church scored lower than those “more engaged”. But for this one, it is the opposite. Perhaps it is true that familiarity does breed contempt.


For this last chart, I simply wanted to highlight the difference between “lifers” and “converts”. Converts, down the line, scored higher than life-long members. While this may seem like a win for converts, ultimately, I want to remind you that converts are supposed to be 60% of the membership of the church, but only represent 13% of the respondents. The best we can infer that to the extent converts are engaged in online discussion regarding the church, they are also more engaged in the church than life-long members.

I regret not having asked for current age and for not asking for gender in this survey, as I would have loved to divide the results along those lines. But for what this survey was, I am pleased with the results, we can easily see areas where we need improvement, and we can see where we are doing well.

When we do these types of studies at work, we have 3 basic follow-ups:

1. Why are we God/Bad at this item? (one of the questions above)

2. What can we as individuals do to improve the score of this item?

3. What does the institution need to do to improve the score of this item?

So what do you think?

17 Comments

  1. I’m glad you’ve posted statistics like this. Data always is interesting.

    Comment by Joseph Smidt — February 6, 2010 @ 12:07 pm

  2. Thoughts on polling and the Bloggernacle:

    We love polls, and judging by the ratio of votes to comments on many blogs, anonymous polls encourage participation by readers who otherwise don’t share their thoughts.

    We love polls, so if you wanted to do a modified version of this poll now or later to add or modify questions or after defining a few terms, we’d no doubt cheerfully take the poll again.

    While there may be some regular readers who catch up on several days’ posts at one sitting rather than checking in every day, my sense is that a blog’s core readers are in the habit of checking daily. If we want a poll or vote to reflect a core community, polls need to close after a couple of days, before there is time for a campaign in any alternate community to take over.

    Thoughts on our engagement with the church:

    I know which points I ranked lower and why, but since taking your poll I haven’t been able to imagine many practical ways to improve. I would like, *in theory,* to have someone take an interest in my progress and to have the opportunity to report in, but as a practical matter I can’t think who that would be — the bishop, perhaps, ideally, but who could add so much to his burden? home or visiting teachers, maybe, again in theory, but the arbitrary assignment of those roles and the haphazard way those callings are often filled would make me exceedingly reluctant to be catechized by them. I guess that’s what spouses and parents are for.

    Just noticed how long this was growing so will stop.

    Comment by Ardis Parshall — February 6, 2010 @ 2:36 pm

  3. Joseph, Thanks.

    Ardis: i’m actually thinking about doing the Poll in my home ward to check for variations, and may tune some of the questions there. Who knows, more surveys could be coming here though.

    As for checking into our progress, I think there are two parts to that, one is setting personal goals and letting someone know about it (what we can do on our own) and the other is the church having some sort of training available to prepare whoever it would be who would handle the leadership end of it. It would of course need to be very sensitively done. I certainly would not respect anyone making Goals for and in my behalf. (Always have chafed at that.)

    Comment by Matt W. — February 6, 2010 @ 3:32 pm

  4. so we can safely say the responses of this survey represent the readership of newcoolthang.com

    … between the dates of Feb 2-6, 2010…

    Comment by Geoff J — February 6, 2010 @ 4:21 pm

  5. I wonder about the best friend question. For instance there was some confusion if a spouse would count as a best friend so perhaps some people who really consider their spouse their best friend and who attend church with the spouse regularly said no anyway. (For married folks who have a best friend other than your spouse — bummer y’all). Also, for those who don’t have a best friend who is also a spouse, the next question is if the BF is in another ward.

    Comment by Geoff J — February 6, 2010 @ 4:27 pm

  6. friendshipping at church (the low score here totally surprised me at first, except then I looked into it, and this is the lowest score for most corporations also. This may be due to a too-literal understanding of “best friend” or thinking of church as local only

    This may be low for bloggernacle types vs other members. I think we tend to self-select for people who don’t have a real kindred spirit in our ward and seek LDS socialization on the web instead. That is certainly the case for me.

    I didn’t consider spouse as being within the intended scope of the question, although maybe I was wrong about that. Certainly having an active member spouse makes church engagement much, much easier than otherwise.

    Comment by sister blah 2 — February 6, 2010 @ 5:32 pm

  7. LOL, I forgot that my husband was my best friend. I think of friends as non-family people that you connect with. I figured I don’t have a “best friend” at church because I sit next to random people in SS, RS and activities and there is no one that I talk to every week.
    I think the question about checking with me about my progress was very oddly phrased to get a decent answer from many.

    Comment by jks — February 6, 2010 @ 5:36 pm

  8. This may be low for bloggernacle types vs other members. I think we tend to self-select for people who don’t have a real kindred spirit in our ward and seek LDS socialization on the web instead. That is certainly the case for me.

    Translation: SB2 lacks social skillz.

    Comment by Scott B. — February 6, 2010 @ 8:25 pm

  9. Scott B.- this from the zeitcast guy…?

    Comment by Matt W. — February 6, 2010 @ 9:11 pm

  10. I wonder about the age at baptism question. I was baptized at 8, but converted at 16. My siblings, although all baptized at 8, were IMHO were never converted and are all inactive. They had periods of activity, but were never really converted.

    Comment by Floyd the Wonderdog — February 7, 2010 @ 6:22 am

  11. Floyd, I mainly wanted to use that question to distinguish those brought into the church by missionary work vs. those who’s parents were members, if that makes sense. Baptismal age seemed the easiest way to do that.

    Comment by Matt W. — February 7, 2010 @ 2:15 pm

  12. I surprised that only NCT readers participated, I know I put the link on MM on the sidebar. I would have thought some people would have come through. Moreover, I am excited that there is at least one other NCT reader in britain.

    On the results:

    I don’ think it is surprising that we know what we are supposed to do but struggle to feel part of the community. GC provides us all with ver clear direction but the expressions of gratitude and friendship are transferable. Thus perhaps there is a disconnect between the people we interpret as the ones giving instructtions (GA’s) and that we can’t then feel their localised support.

    Comment by Rico — February 8, 2010 @ 5:03 am

  13. Thanks for this result Matt.

    I am kind of stingy when it comes to ‘best friend’ status.

    I was also interested in the less active giving their fellow members a decent score on doing quality service. This seems to indicate that most of them probably do not blame their home teachers (for example) for the inactivity. Which is what the church seems to suggest sometimes (where were the home teachers?!)

    Comment by Eric Nielson — February 8, 2010 @ 5:13 am

  14. Rico: Just to be clear, if you linked to the NCT post and not directly to the survey site, all I can see is NCT. And I don’t disagree with your opinion. One of the things that makes our church distinctive is thefeeling of being part of something world-wide, but we do all still feel someone at church cares about us as a person. I think these questions may have an issue of phraseology, as others have suggested.

    Eric: Excellent point on the hometeachers! I think we’ve too long feared going to inactives, feeling they’ve rejected us and not just our faith, but this survey seems to hint at otherwise.

    Comment by Matt W. — February 8, 2010 @ 7:31 am

  15. I think I linked straight to the survey site. Regardless I am sorry it was not more effective.

    I also think it would be useful to carry out in a my local ward.

    Comment by Rico — February 8, 2010 @ 9:22 am

  16. Thanks for posting this, Matt. Fascinating stuff!

    One possible take on why lifelong members generally rate the Church lower than do converts is that lifelong members tend to have more external pressure to keep them in the Church than converts do. Therefore, lifelong members are perhaps more likely to stay in the Church even if it’s not as satisfying. Converts might find it easier to walk away if it doesn’t work out. So I’m guessing it might be an effect of selection.

    Comment by Ziff — February 10, 2010 @ 12:04 am

  17. Exactly so Ziff. I think converts are much less likely not only to not participate in the church when they do not find it satisfying, but they are also much less invested into participating in an online community regarding the church. Selection indeed.

    Comment by Matt W. — February 10, 2010 @ 7:24 am