Survey Says . . .

The 2009-2010 SI Reader Survey ended a few months ago. Much of the data confirmed what we already pretty much knew. But there are some surprises, too. What follows includes a sampling of both the expected and the surprising. For those who would like to look at the results in detail, the attached files provide a summary as well as several “crosstab” reports which group results under various headings.

The survey was divided into three major sections, which we loosely labeled Demographics, SharperIron and Views. 704 people took the survey; 643 of these completed it. While this is a smaller sample than I’d hoped, it’s large enough to be interesting.

Demographics

A little over half of those who took the survey were SI members, with the other two quarters divided about evenly between readers who were once members (but did not re-register in June of 2009 when the site migrated to new software), and readers who have never been been members.

Though it’s difficult to be sure how well that ratio reflects the SI readership as a whole, it’s certainly clear that we have a significant percentage of readers who are not members.

About a third of survey participants said they had been reading SI since it was launched in 2005.

A solid majority of those surveyed are between the ages of 24 and 55. They are almost evenly divided among three age groups, 25-34 year olds (27%), 35-44 year olds (25%) and 45-54 year olds (23%).

Not surprisingly, 84% are male.

Occupations and education

Occupations reflected in the survey vary widely. (Participants were allowed to select multiple occupations, so some were counted toward more than one.) Though the majority of those surveyed were not pastors, “pastor” was the largest single occupational group (35%), with “teacher” a distant second at 12%. Students, homemakers, missionaries and church staff were not far behind.

Most of us were surprised to see that 37% of survey respondents had earned master’s degrees and another 16% had master’s degree work in progress. 7% were working on doctoral degrees of one sort or another.

SharperIron

At 85%, a strong majority of those surveyed said they read front page articles when they visit the site. While any number of these may also be in the habit of reading other SI content, the percentages for Filings, forum content, etc. were significantly lower. Perhaps the most interesting inference here is that if 85% read the front page articles when they visit, 15% do not, heading instead for Filings or the forums primarily but also for the Foundry and the Blogroll.

Question 12 provides more context. There, we asked participants to rate how much they value the different forms of content at the site. Among those who responded, 44% expressed much appreciation for “Front page articles in general,” and 35% much appreciated Filings posts.

A surprise to me was that 33% said they had “very little” appreciation for the conference “Liveblogs” (which we now call Event Blogs). This was the only content category that had more in the “appreciate very little” column than the “appreciate somewhat” column. If we were going to cut something, the numbers would recommend conference blogs. But SharperIron ought to be a place for keeping an eye on what’s happening in Fundamentalism. For that reason alone, we can’t abandon reporting from conferences. Rather, we should probably take that number as an indication that we should for ways to improve the presentation and content of what we post from conferences.

One strong response under “changes you would welcome” (question 15) is noteworthy as well: “structured debates between selected participants” scored 42%, almost twice the votes the second place pick received (“podcast” interviews with leading evangelicals, 23%). This is a long time dream of mine. We just haven’t figured out how to do it yet!

The forums

The survey included several questions aimed at getting an idea of what people think of the forums and why there is less activity in them than previously (though I should note that recent months have evidence a trend of increasing activity there).

Most survey respondents (60%) indicated they don’t post often simply because they don’t have time. It’s hard to find fault with that! Forum interaction is not for everyone and those who cease to find it helpful are wise to find better ways to spend the time.

But two other numbers stand out in that area. 16% indicated that they stay out of forum discussions because they “don’t like to argue,” and a full 26% responded “other” and wrote comments as to why they do not post more often. It’s difficult to see patterns in the “other” comments. The file named “comments.pdf” lists the full text of all the comments with their corresponding questions. If you can discern patterns in them, we’d be happy to see your observations in the thread on this article. But keep in mind that the question is negatively oriented: that is, since it asks why people do not do something, the comments should be expected to be more negative than positive. To properly weigh them, you have to imagine what sorts of comments might be posted if we had also asked “Why do you like to post in the forums?”

The outcome to question 14 was encouraging. 62% indicated that they did not believe the site gives too much influence to ill-informed or ill-mannered people. Though the question is broadly worded, I’m assuming most of the opportunity for this sort of influence to occur would be in the forums, not in the front page articles. The question is important to us since a huge part of the trick we’re trying to pull off here is to allow considerable freedom in posting opinions, yet not become a dumping ground for ignorance and discontent. Though the 62% number is encouraging, the fact that 24% said SI does give too much influence to folks who shouldn’t have it suggests we have more work to do in that area.

But I’d be a fool to miss the opportunity to talk to that 24%. One way you can improve the value of discussion in the forums is to post better mannered and better informed opinions yourself. There will never be a day when we have too many wise, even tempered, articulate posters in the forums! (I’d love to have that problem. We’d post an ad: “More cantankerous know-it-alls wanted in forums.” That’ll be the day!)

Views on issues

Many responses in this area were interesting. In the sub-topic of Fundamentalism, some scores stand out. I was surprised (and I’ll admit, pleased) that 51% of those who responded said they “identify with the best of the Fundamentalist movement.” Another 21% wanted to distance themselves from the movement but still claim the idea. About 9% felt they needed to be more nuanced and offered comments. But a full 18% basically said they don’t quite fit the Fundamentalist profile.

(Keep in mind, that only about half of the survey respondents are members of the site).

Bible translations

The 60% who are multitranslational is a larger group than I expected. Another 20% said that though they use only the KJV, they do not believe it is necessarily wrong for others to use something else.

Calvinism

There’s no disputing that SI has a Calvinist tilt in the forums. I suspect that if such a thing could be measured, Fundamentalist and conservative evangelical blogdom as a whole has a strongly Calvinist tilt. This is currently where a great deal of the pastoral and intellectual energy is in the American Christian landscape.

But the survey results reveal that the Calvinist tilt at SI has been overestimated by some of our critics and is not as large as it may seem when reading randomly in the forums. 35% said they have several beliefs in common with Calvinism but do not claim the name. Another 6% said they reject Calvinism but respect it. Another 10% rejected it and said they see it as serious error.

Reformed theology

About 50% expressed a very positive attitude toward the Reformed tradition but did not claim to be “Reformed.” 16% claimed the label, but specified exceptions in the areas of baptism and eschatology. Only 4% claimed the label without reservation.

Roughly the same number rejected “Reformed teaching” as serious error as rejected Calvinism as serious error: 11%.

Eschatology

The strong representation for pre-tribulational pre-millennialism surprised me. I expected a majority but not 80%. Interestingly though, only 43% of respondents (not all of which were among the pre-millennialists) said they were “very confident” of their beliefs.

Lessons from the process

Next time, we’ll want to promote more aggressively to try to get a larger sample. We’ll also want to have fewer questions that allow more than one answer to be selected. That question format makes interpreting the results much more difficult. Finally, though the comments are interesting, they are almost impossible to actually use except when a large majority are saying essentially the same thing—which is pretty rare. Consequently, though open ended questions may be more satisfying to answer (and more fun to read), they are less useful for making decisions than questions that force respondents into pigeon holes. Probably both are necessary to round each other out.

A word about the crosstab reports: these are easy to misread. For example, if you look at the Calvinism Crosstab report, the first question could easily be misread to mean that 53% of SI’s members are five point Calvinists. But what it actually means is that 53% of those who said they were five point Calvinists are members. To find out how many of the members (who took the survey) are five point Calvinists, you have to look at the Membership Crosstab report, question 19 (the answer is 22%).

If you have questions about other relationships between survey items or have any other questions, don’t hesitate to post them in the thread below. We only ran a few select crosstab reports, then terminated the account at Survey Monkey (fees are required to keep it open). But we have all the data and can run our own reports to glean additional “crosstab” type information (like, how many homemakers were premillennialists, if you want to know). We’re also aware that a couple of the crosstab reports are incomplete (something went wrong in the printing-to-pdf process).

Aaron Blumer, SI’s site publisher, is a native of lower Michigan and a graduate of Bob Jones University (Greenville, SC) and Central Baptist Theological Seminary (Plymouth, MN). He, his wife, and their two children live in a small town in western Wisconsin, where he has pastored Grace Baptist Church since 2000. Prior to serving as a pastor, Aaron taught school in Stone Mountain, GA, and served in customer service and technical support for Unisys Corporation (Eagan, MN). He enjoys writing, science fiction, music, and dabbling in software development.