Survey Results- Part 1 – Who took the Survey

May 4, 2010    By: Matt W. @ 6:17 pm   Category: Survey Results

As a way of introducing the data set and setting out from the onset the natural imperfections inherent in doing data analysis via a web based tool, I thought I would begin with a brief explanation of the intent of the survey and some preliminary charts showing the demographics provided for those who took the survey.

Intent

The survey was initially a test of the Church Engagement Survey promoted by Gallup. After doing my previous post based on the engagement survey we take where I work, ricke mentioned the book “Growing an Engaged Church” and I became very interested in this. The basic premise of the book is that members of a church who are emotionally engaged in the church are also more spiritually committed and are better able to achieve desirable outcomes. I decided to “experiment upon the word” and to run an analysis of this correlation myself. So, while a set of my questions measured basic demographics, there were a set of questions that measured engagement, an additional set which measured what I will call discipleship, and another set which measured desired outcomes. (short spoiler: The book was awesome! I bought one for each member of my bishopric.)

Secondarily, the local philosophy of the church has been greatly shaped by some recent visits by general authorities who have taught the need of the church to focus on what may be Boyd K. Packer’s most famous statement “The study of doctrine will change behavior quicker than the study of behavior” (citation unknown). The basic idea is that the membership should not worry about how the gospel is being applied by others so much, but focus on teaching correct doctrine(The Why), which leads to correct principles (The What), which then leads to correct application (The How). This is all a leads up to me saying that we are all generally well aware of some points that are discussed ad naseum throughout our community, and especially here on the blogs. I added a rather large set of “reductio ad absurdum” doctrinal questions to the analysis as a test case to see if those instances where there was doctrinal certainty (whether in the negative or positive direction) lead to increased engagement one way or another.

While I can now see where the second intent was somewhat misguided (more on that later) and proved to be ultimately untenable in that I would have been better served by only having questions which measured belief in baseline LDS concepts, I was able to gather some very interesting results in any case, which I will share as these posts progress. Further, this was much more fun.

Who took the survey?

First and foremost, 80% of the responses were from regularly attending (more than once monthly) and holding a calling. Another 10% also attend church regularly, but do not currently hold callings. The final 10% either did not consider themselves members of the church, or did not attend church regularly (more than once a month). This lines up pretty much perfectly with my previous demographics on engagement and so, like then, we have to accept that we are primarily dealing with active participants in the church and thus will be biased in that direction.
In the current survey, we attempted to break the barrier which was imposed on the previous survey of “That’s just what bloggers think.” And so we sent the survey not only via NCT, but also via personal connections through email, Facebook, and Twitter. (Anecdotally, while 20 people from Twitter did look at the survey, 0 filled it out. Apparently people who are in the mode for 140 characters or less are not interested in 60 question surveys!). We also asked that those we invited to take the survey spread the word. Here is what we ended up with:

After deleting duplicates, we were left with 1323 responses, 49% female, 51% male, which comes fairly close to the demographical 50/50 split in the church (according to A Statistical Profile of the Mormons ). While NCT was our primary draw for the church, we nearly collected as many responses from “unknown”. Unknown typically has 3 different causes. Either the user is opening the link in a new tab or browser, and thus its source is not detectable, or the url is being copied and pasted into a browser, or finally, an email tool like Outlook is being used. Anecdotally, we have reason to believe the 3rd option to be the most probable for the majority of these unknown responses, and not just any emails, but especially emailed messaged from Facebook. One surprise here was the high response rate from “Cougar Ute Forums” which I had never heard of before, so thanks for who ever linked the survey there.
To further break down how our respondents came in, here is the same data as above but in an area chart with time overlay:

The 4th and 5th of April marked our initial email campaign, post of Facebook, and post on NCT. The second spike was on the 8th, when Rico kindly posted for us at Mormon Matters. The 12th was when we officially started a group on Facebook. (Thus our anecdotal evidence relating unknown directly to Facebook. We have more group members on Facebook than responses marked from Facebook in the survey). We then had a lull until on the 23rd we posted our final request on Facebook and NCT, and were linked at BCC to the NCT post. The 26th was when Cougar Ute Forums picked us up and that basically carried us to the end of the survey on the 30th, at which point we closed the survey.

Most of the responses were married, with a good number of respondents also not having ever been married. Divorced and Widowed were not represented much.

Most of the respondents were the same age as me. This is one of the problems with sending surveys to your personal friends. We did have a very diverse group overall in terms of age though, so it worked out pretty good.

I used age at baptism and current age to calculate how many years each person had been a member. Only 17% of the dataset was considered a convert (person baptized over the age of 8) but the majority of converts had been members 10 or more years, so it is reasonable to assert they know their ropes.

How international is this study?

Well, not as much as we would have liked, but we did ok. World wide, we got quite a few, especially in the UK (Thanks Rico!) and Europe.

(Yellow is Female Responses and Green is Male)

The US was still over 93% of all responses, breaking down into metro areas as follows.

Unsurprisingly, SLC/Provo was our largest contributing area, but did get big pulls from San Antonio, TX (probably due to our being locals), San Diego, NYC, DC, Phoenix and…Kansas?

Two observations from the demographics:

1. When looking at Marital Status in conjunction with Membership Status, I did notice that the divorced members as a block were much less likely to attend church regularly. While I can not pretend to know the cause of this, this does initially indicate to me that the church probably could exert more attention into the status of divorced members.

2. In looking at the most active converts to the church (attending more than once a month and considered self a member), after we ignore the 9 year olds who join the church, it seems that the most active converts long term are between the ages of 18 and 21. So tell your missionaries to go hang out on college campuses.

However, see if you can find the “Rainbow Discussions” they used back in the 70s, as those got the best results.

Next- Measuring Discipleship

Appendix #1- Brian J. challenges observation #2, here is my response.

16 Comments

  1. Dang! Stupid Utah! Texas Mormons baby! Woot!

    Comment by Tirian — May 4, 2010 @ 7:45 pm

  2. Nice beginning.

    he most active converts long term are between the ages of 18 and 21.

    I don’t think I follow. We know that most people who took your survey are 30-34 yrs old and most converts who took your survey have been members 10-14 yrs; 34-14=20.

    Comment by BrianJ — May 4, 2010 @ 8:52 pm

  3. Seriously, Tirian? Seriously??

    Comment by Niarit — May 4, 2010 @ 9:00 pm

  4. No not seriously Niarit. :)

    Comment by Tirian — May 4, 2010 @ 9:06 pm

  5. Brian J-

    see the now added appendix #1

    It’s basically just a cross-generational fact that 19-23 year old missionaries are going to have the best interactions with investigators who are near their same age.

    Comment by Matt W. — May 4, 2010 @ 9:47 pm

  6. One surprising thing is the lack of participants in S. America. Is the internet that scarce in that part of the world or is the language barrier the major issue? I guess that this will have a v. anglophone flavour to it but it looks excellent. Plus I bought the book. Thanks for the recommendation.

    Comment by Aaron R. aka Rico — May 5, 2010 @ 3:34 am

  7. I really appreciate you taking on this project. It’s fascinating so for far and I really look forward to future entries.

    “It’s basically just a cross-generational fact that 19-23 year old missionaries are going to have the best interactions with investigators who are near their same age.”

    This is a factor, but I think it’s also because it’s an age where people are still “finding themselves,” are more malleable and less set in their ways, and haven’t finished forming their brains yet. With the latter, I don’t mean that young people get baptized because they’re “dumb” (I was a convert at age 18 myself), but that with a still developing prefrontal cortex, the teen and early twenty set experiences emotion unfiltered by that boring ration and judgment business, making it easier to have strong spiritual experiences and take a risk like getting baptized. Well, that’s my theory at least.

    Comment by Katie M. — May 5, 2010 @ 5:23 am

  8. Fascinating survey. I’m excited to see the rest of the results.

    I know I voted from Michigan, but it looks like those of us in SE Michigan were well represented in your survey as well!

    Comment by brandt — May 5, 2010 @ 6:18 am

  9. Aaron R.- Hope you enjoy the book. I may “spoil” some of it with these posts, but I think it is worthwhile.

    Katie M.- Good point, I heard once that most major life changes happen between the age of 19-28, due to it being the initial period when youth leave the home and get out on their own, and thus start making their own life decisions. One stat said if a person didn’t smoke before they turned 25, they probably would never smoke. Wish I could remember where I read those…

    brandt- I probably should go back and put some sort of legend on that map…

    Comment by Matt W. — May 5, 2010 @ 6:37 am

  10. Matt,

    So you’re telling me that I was the only one eh? At least Detroit had SOME representation.

    Comment by brandt — May 5, 2010 @ 7:39 am

  11. Glad to see that http://www.cougaruteforum.com/ participated in the survey. We’re all really interested in the results. For those who may be interested in our little forum, stop by and check us out. You may be surprised. It’s much more than a sports board.

    Comment by pellegrino — May 5, 2010 @ 7:46 am

  12. We’d love any new voices on CougarUteForum, but we’re especially in need of more female posters. Any women who are up for interesting religious discussions would be welcomed. We also discuss politics, books, food, running, movies, home improvement and more.

    The best thing about CUF (in my opinion) is that members, non-members and x-members are able to discuss religion together with little moderation. This rather free environment can make for some fascinating conversation.

    Comment by Rosebud — May 5, 2010 @ 1:55 pm

  13. Wow, now there’s a solid example of why serious statisticians never use pie charts! Don’t be afraid of using tables of numbers when things get complicated and you have lots of categories!

    Great project. Love the geographic distribution of responses maps. Wonder how those blips on the US maps line up with major metro areas. Looks like pretty good correspondence.

    Comment by Owen — May 7, 2010 @ 8:23 pm

  14. I’m a woman living in Saudi Arabia who took this survey but I don’t see any women from my area on your location map. Why is that?

    Comment by meems — May 10, 2010 @ 12:19 pm

  15. a bit late here, but just a clarification: I wasn’t “challenging” you in #2, I just wasn’t following the numbers. (I’m still not, but I’m also really tired….)

    Comment by BrianJ — May 15, 2010 @ 4:22 pm

  16. meems- I need to redo that map I think. Dots seem to cover each other on the maps…

    BrianJ- No worries.

    Comment by Matt W. — May 15, 2010 @ 5:22 pm