Statistics and Skewed Samples

An unsophisticated forecaster uses statistics as a drunken man uses lamp-posts - for support rather than for illumination.

- Andrew Lang

I've recently been doing statistical tests on coauthorship data we collected through our NSF Advance grant at New Jersey Institute of Technology. We're looking at questions such as gender differences and models for success in terms of promotion and rank.

The problem is that we have very few female faculty at NJIT. This is a problem endemic to Science, Technology, Engineering, and Math (what many call STEM). Women either choose not to go into those fields, or once in them, they leave. Often they leave because of harsh or hostile environments from colleagues, which is why we are hoping to see if women are excluded from the critical collegial social networks within the technological university setting.

Statistical tests rely on random samples of a population. In this case, however, we are sampling the entire population, but a population that is very heavily skewed towards male faculty. In looking at Associate Professors over a 10 year period, we do not have enough female faculty to even begin to assume a normal distribution. That then begs the question of how we can test our hypotheses regarding the exclusion of women when so few women even exist in our sample?

It's a quandary we're still trying to solve. We're trying various means (permutation tests, bootstrapping) but we may have to resort to simple qualitative descriptions. We're also using social network analysis methods to look for homophily in coauthorships - that is, do men prefer to coauthor with men and women prefer to coauthor with women, or is there also ample inter-gender collaboration? Results will hopefully be forthcoming.

Measuring Influence on Social Media

The brain is a commodity used to fertilize ideas.

- Elbert Hubbard

Social network analysis is a hot field these days. Exploring people's connections can tell us a lot, not just about those people but about the networks within which they live and work.

Recent research is focusing on how to measure the influence of individuals through social network analysis methods. I invite you to take a look at this upcoming conference:

Measuring Influence on Social Media

The conference is currently soliciting position papers and research-in-progress posters, due by June 15.

I encourage anyone interested in this exciting field to submit a paper or poster or attend the two-day conference in beautiful Halifax, Nova Scotia.

You could win a $50 gift card

As part of my PhD studies, a fellow student and I are conducting a survey of Facebook users. One out of every 100 respondents will win a $50 gift card. To participate, please click on the link below.

Facebook Privacy Survey

The survey should take approximately 15 minutes to complete. Please pass this information along to any of your fellow Facebook users. Below is our official text requesting participation.

Please participate in an online survey conducted by two PhD students (Regina Collins and Harshada Shrivastav, under the direction of Profs. Roxanne Hiltz and Cathy Dwyer) for their research project. The purpose is to study the concerns of Facebook users related to recent changes in privacy policies and features. We think you will find the survey itself to be interesting and thought-provoking.

We estimate that it will take you approximately 15 minutes to complete the survey. All information will be kept confidential and will be used only for research purposes. If you are a Facebook user and 18 years of age or over, please click on the link below, or copy and paste the entire URL into your browser to access the survey:

Facebook Privacy Survey

The deadline to complete this survey is December 1, 2011. Please note that you may win a $50 gift card from Amazon.com or Barnes and Noble.com as we will draw one person at random for each 100 people who complete it.

Please also pass on this survey link to all your friends. We highly appreciate your time and efforts.

_________________
Information Systems, New Jersey Institute of Technology

323 Martin Luther King Boulevard

Newark, NJ

Success - and Learning from Failure

The greatest barrier to success is the fear of failure.

- Sven Goran Eriksson

Well, what a ride! I started out in January of this year proposing to do a Master's thesis as part of my degree in Professional and Technical Communication. As part of my thesis, I developed a web-based application for the assessment of student portfolios by a community of readers. Keep in mind that I hadn't programmed since my undergrad days at NJIT (25 years ago!) and so I was faced with the daunting task of becoming proficient at HTML, learning PHP, and learning MySQL. All this in the course of a few weeks so that I could actually develop the application and have it finished in time to perform usability testing and complete my thesis.

This wasn't just a theoretical application, mind you. I had to ensure that the application actually worked because it was going to be put to full use on May 13th of 2010 when the freshmen-level writing professors would be using it to score approximately 80 student portfolios.

I was terrified the night before the actual portfolio assessment session. I had visions of the application crashing and burning in all its glory. I had tested it as much as possible during development, but that's nowhere near the same as putting it out there for 20 or more professors to use in real time.

So was it a success? Yes! Did it have problems? Yes! I realized that the way I had implemented adjudications did not work based on the way the application was actually used, so I have already started to redesign that part of the application. But in general, the application performed beautifully.

So what does this all mean? It means that perseverance and the development of knowledge are just as important to defining success as a positive outcome. The application did not work flawlessly, but there were no recriminations from the professors - there were suggestions of how to make the application better, there was increased knowledge and understanding on my part, and all those factors together point to the fact that the entire experience was a success.

In fact, it was such a success that I have now graduated from NJIT with my Master of Science in Professional and Technical Communication, and I am subsequently applying for admission to NJIT's PhD program in Information Systems. I'm sure that experience will generate even more posts to this blog, so wish me luck!

Designing an Assessment Application

Don't make something unless it is both necessary and useful; but if it is both necessary and useful, don't hesitate to make it beautiful.

- Shaker Philosophy

I've been fairly quiet again, but it's not because I don't care about you, my readers. I've just been really busy trying to work on my thesis proposal. Now that the proposal is in good shape, I'm doing rapid prototyping for my application to work on the HTML screens. I'm going to post them here (there are quite a few) in the hopes that I can get some feedback on usability design. These are non-functioning HTML pages - there are no active links. And I know the design is very simple, but my development cycle is quite compressed so I don't have a lot of time to make it "pretty" right now.

So here are the pages - let me know what you think.

This is the Welcome Page that users will see when they start the web-based application.

They then move on to the Login page, where both raters and administrators can log into the application.

From here the screens divide into two components: screens for the raters and screens for the administrators. I'll handle the screens for the raters first.

The first thing a rater does once he or she has logged in is to select a student to rate using this page.

Depending on the section being rated, the rater will then be presented with one of three scoring rubrics: Humanities 101, Humanities 352, and graduate level MSPTC.

The rater must submit a score for each variable on the rubric unless the rater is adjudicating, but more on that in a minute.

When the rater submits the completed rubric, she is presented with choices for the next action, as shown on this page.

If the rater chooses to log out, she is presented with a screen asking if she is sure she wants to log out. If she answers yes, she is logged out and thanked for her time and efforts on a final screen.

If the rater chooses to rate another student, he goes back to the page to select the next student to rate.

Now, from the administrator side of the application, there are more functions available. When an administrator logs on, she is presented with the administrator screen listing possible actions. I'll step through each of these actions in order.

The Begin Assessment screen lets the administrator start the "official" rating period. Prior to that, any scores entered are not stored in the database. This is useful for the purposes of training the raters prior to starting the actual assessment.

If a faculty member walks into the assessment room and asks to be a rater, the administrator can use the Add New Rater screen to add that faculty member to the list of raters. Similarly, if a student knocks on the door and hands the administrator a last-minute portfolio for scoring, the admin can use the Add New Student screen to add that student to the list to be rated.

Each portfolio is rated by two raters. But if there's scores don't match or aren't adjacent (within one point of each other), we assign a third reader, called an adjudicator, to break the tie, so to speak. The administrator can find the portfolios that need an adjudicator and then assign them using this screen. Note that the adjudicator only provides scores for the variables requiring adjudication.

The administrator can also monitor scores to evaluate the ongoing assessment process. Using the View Reports screen, the admin can select different types of reports to view.

• View Report by Student lets the admin look at the scores from all raters for a particular student.


• View Report by Rater lets the admin look at the scores being assigned by a particular rater.


• View Report by Variable lets the admin see a bar chart of the distribution of scores for any particular variable from the rubric.


• View Records Requiring Adjudication lets the admin see all records that needed adjudication, whether the adjudication was already completed or not. This is useful for identifying any raters who are consistently requiring adjudication.

Once the assessment is over, the administrator can export the data for analysis using either the SPSS or SAS software package by making the appropriate selection on the Export Data page.

That's the design of the application as it currently stands, which is still in the early stages. I look forward to your feedback.

Stereotype Threat - What Every Minority Should Know

The emotional, sexual, and psychological stereotyping of females begins when the doctors says: "It's a girl."

- Shirley Chisholm

I was doing a literature search for a research proposal and I came across a most fascinating study. Published in 1999 in the Journal of Experimental Social Psychology (volume 35, pages 4-28), it documents research performed by Steven Spencer, Claude Steele and Diane Quinn regarding something they call stereotype threat. They define stereotype threat as "the experience of being in a situation where one faces judgment based on societal stereotypes about one's group..."

It's really a fascinating study - one that every minority should read. To summarize what was a very involved study, the researchers found that when men and women took an easy math test, they scored about the same. But when the test was difficult, women scored significantly lower than men.

This might make you think, "Aha - so it's true that men are better at math than women!" But you'd be wrong. Because in a second portion of the study, the researchers showed that stereotype threat was the real culprit behind the significant difference in performance. In this second study, all groups were given the difficult math test. But some groups were told ahead of time that previous tests had shown gender differences (thereby reinforcing the stereotype threat), and other groups were told that the test had never shown any gender differences in the past.

• The groups in which the stereotype threat was reinforced showed a significant difference (almost 20%) in scores between men and women.


• In the groups who were told no gender differences existed, the scores for men and women were almost identical.

Fascinating, isn't it? So the next time you hear someone tell you men are simply better at math, or some other ignorant stereotype, don't believe them, and certainly don't let it impact your own performance!

Life Happens...

Change is inevitable - except from a vending machine.

- Robert C. Gallagher

My post on December 4th talked about my evolution as a blogger. But what it really looked at was the visual evolution of the design of my blog. Although I talked a little bit in there about how my tone has gotten more conversational, really my main focus was to show how I've tried out different blog templates (kind of how teenagers try on different personalities). I still don't think I'm done changing the way my blog looks, so don't be surprised if you check back in and it looks different again...

What about the main topic of my blog? When I first started, I envisioned I'd be writing mostly about women in technology because that's kind of a pet project of mine. Well, if you read my posts, I've tried to go back to that topic every so often, but somehow, I keep getting sidetracked. And isn't that kind of how life works? We start out down one road or one thought process and we find ourselves on some detour.

That's not to say that the detours aren't beneficial. Sometimes you discover new places (or new things about yourself) by taking a detour. In my last posting, I described stepping outside of my comfort zone to design a social media strategy for defero. When I started this blog in September, I never dreamed I'd be blogging about something like that! Or about the Z pattern in eye movements, for that matter. But even though I've wandered around a bit in terms of topics, you can still get a good sense of who I am by reading my posts.

When I first started writing my blog, I tended to write in a formal tone because that's what I was used to doing in my job. But I've tried to relax a bit, and now I try to write my postings as if I'm writing an e-mail to a friend, not posting thoughts to a blog for public consumption. I hope you've noticed and enjoyed the difference.

I've also gone back and forth with being worried about the length of my blog postings. At first, I tried to make sure I covered every point I wanted to make, but then my postings got a little long. So the next few postings got short - maybe even too short - as I tried to really cut back on what I wrote to keep the postings to a reasonable length. Now I've gotten to the point where I'm confident enough to write what I want to say and not worry so much about the length of the posting.

So where do I (and my blog) go from here? I hope to get back more to my topic of interest: trying to encourage girls to pursue careers in science and technology and discussing why women so frequently drop out of those careers. But like my title says, life happens, and who knows where the next detour will lead me? Or, as one of my favorite poems from The Lord of the Rings says:

The Road goes ever on and on

Down from the door where it began.

Now far ahead the Road has gone,

And I must follow, if I can,

Pursuing it with eager feet,

Until it joins some larger way

Where many paths and errands meet.

And whither then? I cannot say.

- J. R. R. Tolkien

Illustration copied without permission from www.coldal.org/quotes.html