This is part 2 of my response to a reader question regarding how I came up with the idea for Vjournal and how I tested the edginess tone.  Part 1 can be found here.

Phase 2: Using focus groups to test your business idea

My focus groups were admittedly less than formal.  I do have a background (way, way back) in research methodology but for Vjournal, I took off that hat and approached it from an entrepreneurial (read: fast and cheap) perspective.  Even so, because of my analytical training,I was well-aware of the rules I was bending and their likely effect on my data. 

First, I created a Power Point mock-up of the homepage with 3 versions to test the brand name and the department names to get an early gauge on going more edgy:

• The first is the one you can see on Vjournal.com because it ultimately passed my tests.  This tested Vjournal and the edgy department names like "Women on Top" and "Ample Assets".
  
• The second had the Vjournal name but "safe" department names, like leadership, entrepreneurs, and health & beauty. 
• The third contained a safe name (LipstickJournal) and safe department names. 

All of the article titles and content as well as the color scheme were constant so I was testing only the name and the department names - the "edgy" vs. "safe" debate.  I held 3 focus groups, totaling 37 business women ages 25-55.  Each focus group received only 1 version above. (I would have done more but scheduling was proving to be a problem.  So after the 3, I supplemented by sending the questionnaire out via email to those that couldn't make them.  This added another 33 replies.) 

The first part of the focus group was each woman filling out a brief questionnaire consisting of their age and zip code, their title, industry, and size of company, followed by questions about where they got information about leadership, business management, health and personal finance today.  They were asked to list all print subscriptions and websites they went to and how often they read or visited them.  They were also asked to list membership in any business and networking organizations, and how many times a month or a year they attended these events. 

The questionnaire then asked them to respond to questions about the PowerPoint site mock-up, the name, the department names, style etc. from totally dislike to totally like.  They were also shown a list of 40 words and asked to circle the words they felt when looking at the site, negative words like "offensive" and "sexist" to terms like "empowered" and "funny".  This part of the test was actually the most important to me as far as testing edginess.   

With the focus groups, the second part was a discussion of the information sources they had mentioned in the survey, as well as female leadership in general, their jobs, and areas they felt they struggled.  The website was intentionally not discussed.

At the end of the focus group, which lasted 45 min to an hour, they were asked to write the name of the website down on a blank piece of paper.  This was for me to get a rough idea of how well they recalled the name after an hour conversation.  In a formal test a researcher would have totally changed the subject rather than talked about related topics, but I had limited access to these folks and wanted to get the most out of them that I could.  Vjournal had strong brand recall in this informal "don't bet the house" test.  Once they heard it, they remembered it an hour later enough to write it down.  (If you are "betting the house" by all means do a reliable brand recall test!)

What I learned:

With basic demographic information, I did a quick analysis on the circled words.  Clearly if too many women were put off by the edgy tone, I'd have thought twice about it.  But over 50% felt "motivated" and "empowered" by the concept.  I did get an 11% response to "offended" and related negative words. Frankly, I was expecting negative reactions to be more like 20% because I knew this tone couldn't please everyone.  So the issue was how big was the "negative" group, and who were they?    Of the ones that might be offended, they tended to be older (majority over 49).  Only 6% of those under age 45 circled "offended" or related negative words.  This dissipated to 3% by the time I got to under 35.  (Of course my sample size was very small, and I broke a number of research rules (see end of article).  But directionally I had enough to move to the "real-world" testing.)

As far as content, I gleaned the following from the questionnaires and discussions:

• None of the women listed a print or online source for ongoing leadership or business management advice that they regularly engaged with. 
• Their largest source of credible information was company-provided training or company-paid leadership classes, and only 8% felt they had access to these forums (likely because few worked in large corporations).
  
• Several had bought one or multiple books, so they were interested in improving themselves in this area.
  
• Most had a variety of sources for personal finance, but they found it overwhelming and therefore rarely made changes to their portfolios. 
• The entrepreneurs felt that most of the information geared towards women didn't go into enough actionable advice.  Others found the information too simplistic, such as "if you dream it, it will become a reality" drivel - so it wasn't "real" enough for them. 
  

The one missing critical piece of research is how "large" my demographic segment is.  Upwardly mobile business women are a growing worldwide demographic.  But unfortunately the data isn't as reliable as other census data because when women opt out of a company, it gets hazy - they seem to disappear, and there is an assumption that many just become full-time moms.  But this doesn't jive with the fact that women are starting new businesses at 3-times the national average.  

From a variety of sources, I can determine the segment to be north of 10 million when you take into account women in executive positions, women in management that hope to rise to executive positions, women-owned businesses, and executive women who are "on-ramping" after being home for awhile.  How much north I'm not sure right now, but even a smaller number would make me feel comfortable that there is a niche market, and I'm aware of a print magazine targeting business women that claims 500,000 subscribers.  So since I'm not at the point of developing a pitch for investors, I am comfortable that I'm not wasting my time and will continue to look into it to further the business case development as time permits.

Rules I bent (ok, broke):

To be clear, this testing was not done to rigorous research method standards, but it gave me a very rough gauge - enough to feel comfortable building the website. In fact I broke a number of research methodology rules for the sake of cost and expediency.  But I had a few things in my favor: a) There was a slightly lower risk to me breaking the rules because I've taken research methods classes and taught statistics 20 years ago.  So while I'm very rusty, I have a better background to to proceed this way than most;  b) Because of that background, I had a sense for which rules I was breaking and the likely cost to the accuracy of my data;  c)  For this early phase I really wanted directional information so I'd know whether women in general were offended or curiously interested in the "tone" or "mood" of the site and the brand name.  I didn't expect to rely on these results alone so detailed accuracy wasn't that important - it would get more accurate with the launch of a real website.   I did it to raise my comfort level that I wasn't throwing money away in site development.  Here is a short-list of the rules I broke:

1. As owner of the company, I should not have been in the room with the focus group attendees because I could have acted in such a manner as to impact their responses. I was not introduced as the owner but rather as a consultant doing research for a client, so the issue is how well I was able to maintain my research 'persona' for my own project.
   
2. The focus groups were held in living rooms - not behind the standard 2-way mirror. In the most formal of research, people wouldn't be allowed to see or hear each other but be in separate rooms while filling out their questionnaires.  Although they were separated enough not to see each other's answers, there was a loud laugh from one that likely skewed some of the results for that group.  Women in that group were probably less likely to say they were offended when hearing a peer laugh so this made my data from that group unreliable for the "edgy" test.  I marked every questionnaire from these focus groups so I'd know that this data should be taken with a grain of salt.
   
3. The questions themselves should have been tested for reliability, validity, and bias.  Instead, given I've written enough professional questionnaires in my day, I skipped the question testing.  But as entrepreneurs you must be extremely careful, as most of the questions I've seen are poorly worded such that they make one answer more likely than another answer, introducing bias.   It's common sense that every question should be neutral so there is an equal likelihood of getting the opposite answer from what you want to hear.   But writing questions to this standard is not as easy as you may think, because bias can be subtle in the word choice, phrasing of the question, and in combination with other questions.  That's why in formal statistically valid research, you really want to test the questions themselves. 
   
4. The focus groups only took place in 2 west-coast cities.  Likely I may have gotten different responses in the mid-west and east coast.  Possibly the number of "offended" people might have been higher in some areas.  Would it have exceeded my projection of 20%?  Don't know.  But my "real world" test would be more accurate on this anyway.
5. If you recall I had each focus group attendee list all print subscriptions and websites they went to and how often they read or visited them, as well as memberships in business and networking organizations.  These lists were likely  incomplete as they didn't have anything around them to jog their memory.  Those that sent in their responses via email had longer lists, so they may have seen a magazine around their home or office or checked their email for newsletters or event announcements, but this is conjecture.  Either way, none had an information source that they relied on consistently for leadership advice and managerial expertise, my 2 planned dominant areas of planned coverage.
   

Bottom line, I got enough feedback that the possibility of a niche publication to power women with an edgy tone appeared to exist that I was willing to build out the real thing. 

My first decision was not to go with a "blog" platform like Wordpress or TypePad, but to instead use Drupal, which is the platform used by most major media websites.  It is more flexible and makes my site a "grown-up" professional news site.  Building it on a blog platform would have taken very little capital, but building it on a professional platform required a few thousand dollars.  I made the decision to "dress for success" because after the testing I believed enough in what I was doing not to want to have to move the site to a heartier platform later, as that is quite painful and would have increased the cost to be on the right platform by a lot.  So I chose to invest in the right one up-front, and now that it's gaining some traction I know I made the right choice.



Have a question?  Ask it at Natalee Roan [at] yahoo.