- Shaker Philosophy
I've been fairly quiet again, but it's not because I don't care about you, my readers. I've just been really busy trying to work on my thesis proposal. Now that the proposal is in good shape, I'm doing rapid prototyping for my application to work on the HTML screens. I'm going to post them here (there are quite a few) in the hopes that I can get some feedback on usability design. These are non-functioning HTML pages - there are no active links. And I know the design is very simple, but my development cycle is quite compressed so I don't have a lot of time to make it "pretty" right now.
So here are the pages - let me know what you think.
This is the Welcome Page that users will see when they start the web-based application.
They then move on to the Login page, where both raters and administrators can log into the application.
From here the screens divide into two components: screens for the raters and screens for the administrators. I'll handle the screens for the raters first.
The first thing a rater does once he or she has logged in is to select a student to rate using this page.
Depending on the section being rated, the rater will then be presented with one of three scoring rubrics: Humanities 101, Humanities 352, and graduate level MSPTC.
The rater must submit a score for each variable on the rubric unless the rater is adjudicating, but more on that in a minute.
When the rater submits the completed rubric, she is presented with choices for the next action, as shown on this page.
If the rater chooses to log out, she is presented with a screen asking if she is sure she wants to log out. If she answers yes, she is logged out and thanked for her time and efforts on a final screen.
If the rater chooses to rate another student, he goes back to the page to select the next student to rate.
Now, from the administrator side of the application, there are more functions available. When an administrator logs on, she is presented with the administrator screen listing possible actions. I'll step through each of these actions in order.
The Begin Assessment screen lets the administrator start the "official" rating period. Prior to that, any scores entered are not stored in the database. This is useful for the purposes of training the raters prior to starting the actual assessment.
If a faculty member walks into the assessment room and asks to be a rater, the administrator can use the Add New Rater screen to add that faculty member to the list of raters. Similarly, if a student knocks on the door and hands the administrator a last-minute portfolio for scoring, the admin can use the Add New Student screen to add that student to the list to be rated.
Each portfolio is rated by two raters. But if there's scores don't match or aren't adjacent (within one point of each other), we assign a third reader, called an adjudicator, to break the tie, so to speak. The administrator can find the portfolios that need an adjudicator and then assign them using this screen. Note that the adjudicator only provides scores for the variables requiring adjudication.
The administrator can also monitor scores to evaluate the ongoing assessment process. Using the View Reports screen, the admin can select different types of reports to view.
- View Report by Student lets the admin look at the scores from all raters for a particular student.
- View Report by Rater lets the admin look at the scores being assigned by a particular rater.
- View Report by Variable lets the admin see a bar chart of the distribution of scores for any particular variable from the rubric.
- View Records Requiring Adjudication lets the admin see all records that needed adjudication, whether the adjudication was already completed or not. This is useful for identifying any raters who are consistently requiring adjudication.
Once the assessment is over, the administrator can export the data for analysis using either the SPSS or SAS software package by making the appropriate selection on the Export Data page.
That's the design of the application as it currently stands, which is still in the early stages. I look forward to your feedback.
