Online Assessment Series

Online community, 

I wanted to take the opportunity to share with you some online assessment resources that can be helpful while we transition your courses online. Creating meaningful assessments that are authentic and provide a high degree of validity and reliability can be a challenge, but it can be done in every discipline. We have resources that can help you get started transforming a traditional exam over to a more active learning experience for students. As an integral part, assessment provides an opportunity for teachers and students alike to identify areas of understanding and misunderstanding. With this knowledge, students and teachers can build on the understanding and seek to transform misunderstanding into significant learning.Assessments indicate to students what they should learn and they give concrete meaning to valued learning goals. Take a look at your course syllabus and determine what course learning goals have not been met yet, those are the assessments we need to focus on and prioritize. When students are encouraged to assess their own learning, they become more aware of what they know, how they learn, and what resources they are using.

"Conscious knowledge about the resources available to them and the ability to engage in self-monitoring and self-regulation are important characteristics of self-assessment that successful learners use to promote ownership of learning and independence of thought." (Kenny, Silver, 1993)

Some quick tips to keep in mind when developing quality assessments include:

• Provide Clear Instructions. Include any resources (such as readings or media), examples, and your grading standards.
• Set a Point Value. If an Assignment is worth "zero points", the Gradebook can't calculate properly, and students will get an inaccurate report of their grades.
• Set a Submission Type. If you leave the default "No submission", students will have nowhere to upload their work - not the best choice for an online course
• Set a Due Date. This helps students manage time effectively; it also ensures they'll get automatic reminders from Canvas a week ahead.
• Attach a Rubric. A Canvas Rubric is not just text: it's a functional object that can reduce your grading time (and clarify expectations for students).

In addition, reflect on your assessments and ask yourself:

• Engagement: how are your Assignments designed to encourage students to participate actively, rather than read passively?
• Community: how do they elicit and/or build upon a sense of shared intellectual inquiry?
• Accessibility: how do they avoid presenting undue obstacles to differently-abled students, and take advantage of available tools and features to maximize accessibility?
• Ethics, Safety, Privacy: how do they address relevant privacy concerns? do they provide instructions and resources to help students avoid unsafe or unethical online activities?
• Communication/Feedback: do they include specific and relevant feedback channels and expectations (e.g. rubrics and annotations, media comments, etc.)?
• Online Presence: how do they effectively incorporate your online presence, and/or encourage students to do likewise?

Todd Gilman writes “Your top priority in designing effective online assignments should be to make sure they work can be completed by each student on his or her own. Students should not be left to wonder whether they are proceeding correctly at any point along their path to completion of the work. That means that your instructions on how to do the assignments have to be as explicit as they are flawless.” (Gilman, 2002)

There are four categories of assessment as described by Maeder (2017) including standardized assessment, simple completion, complex completion and presentation assessment. There are some ways that you can manipulate each category to get the highest validity possible. For example, items in standardized assessment require the learner to choose a single answer from a  small set of response items (a-b-c-d-none/all of the above). These are the lowest order cognition strategies. Often, the learner can guess the correct answer and they are not recommended for assessment, Rather, a better option would be to use a "Selection" whereas, this item type is characterized by choosing one or more responses from a stimulus array. In contrast to multiple-choice, the number of possible choices is typically large enough to limit drastically the chances of guessing the correct answer. Ideally, the response is mentally constructed and not simply recognized. Examples include keylists, cloze elide (deleting extraneous text), and, via touch screen, tracing orally presented directions on a computer-generated map. Other interactive examples include activity-based assessments, including word-seeks.

For example:

Example. Delete the unnecessary or redundant words from the following paragraph.

Andy Razaf is not a quickly recognizable name that is familiar to most people. Yet Razaf wrote the lyrics to at least 500 or more songs, including the words to the popular “Ain’t Misbehavin’,” “Honeysuckle Rose,” and “Stompin’ at the Savoy” as well. The American-born son of an upper class African nobleman, he still continues to be overshadowed by his composer-collaborators who worked with him, Fats Waller and Eubie Blake.

Another fantastic way to modify standardized assessment is to use rearrangement techniques.  

Although responses are chosen from a stimulus array, the task here is to place them in the correct sequence. Examples include constructing anagrams, ordering a list of sentences to reflect a logical sequence, categorizing elements in a list, arranging a series of mathematical expressions to form a correct proof, arranging a series of pictures in sequence, and putting together a puzzle. 

For Example:

Example: Rearrange the following group of words into a complete and meaningful sentence capitalize the first word and end with a period. No other marks of punctuation should be needed.

...a and be both can comedy enlightening entertaining good...

Another form of rearrangement can be accomplished using a tool known as an Ordering Activity.

There are many other examples provided in the Online Assessment Course - COVID-19, that you can participate in, now. All faculty have been enrolled in this course which will help you migrate your assessments. Other resources in the course include links to the skillscommons.org website, an open educational repository for STEM faculty as well as the National Institute for Learning Outcomes Assessment site; great resource for authentic, meaningful assessments for all disciplines.

I appreciate your willingness to keep an open mind and try a different assessment technique that may not be as comfortable as it was prior to COVID-19. However, designing high-validity, reliable exams is something the online realm has been researching for twenty years. There are some great articles that I'm happy to share with you, that are discipline specific.

Remember, I'm always willing to help you navigate these assessments in Canvas or contact a Canvas Champ to help you get started. I'll post other online assessments resources weekly.

Take care,
Anna


Resources:
Gilman, T. (2002, 03 22). Designing Effective Online Assignments. Retrieved from The Chronicle of Higher Education: https://www.chronicle.com/article/Designing-Effective-Online/64772

Quality Matters. (2018, 03 26). Quality Matters. Retrieved from Quality Matters: https://www.qualitymatters.org/

Wiggins, G., & McTighe, J. (2018, 03 27). Backward Design. Retrieved from State University of NY: https://www.fitnyc.edu/files/pdfs/Backward_design.pdf

R. L. Bangert-Drowns et al., "The instructional Effect of Feedback in Test-Like Events," Review of Educational Research, 61:2 (1991), 213-238. This study reported a metaanalysis of 40 studies that showed that (a) immediate feedback is more effective than feedback that is delayed a day or more after a test, and (b) providing guidance about correct answers is more effective than feedback that merely informs students whether their answers were correct or not.

California Assessment Program, A Question of Thinking: A First Look at Students' Performance on Open-Ended Questions in Mathematics (Sacramento, CA: California State Department of Education, 1989), 6.