@ONE Designing Effective Online Assessments Course Week 1

This is some of the important content from this week:

Assessment in higher education involves four important processes:

• identifying clear, valid, and appropriate student learning outcomes
• collecting evidence that those outcomes are being addressed
• dialogue to attain a collective interpretation of the data
• using data to improve both teaching and learning

Assessment is not an event. Assessment is not separate from instruction; it is embedded in instruction. This is a fundamental concept.


Brief Overview of Student Learning Outcomes
(SLOs) are statements which define what a student should be able to do after the completion of a course or program. The SLO defines what will be measured and dictates what assessment tool is appropriate. SLOs represent both the target for our service or teaching and the expectation for student achievement as a result of our effort. Assessment information tells us what students can do and how well they have learned as a result of a course or program. It informs us about the effectiveness of our pedagogy. Assessment data provides a culture of evidence which is the foundation for modification in teaching or service and further revisions to SLOs.

The Logic of Assessment

Assessment is really a way of thinking and an approach to learning.
Good assessment follows a specific set of steps; these steps answer the following questions:

• What do you want to measure?
• Why do you want to measure it?
• How will you measure it?
• When and under what circumstances will you measure it?
• What did you find?
• What does it mean?
• What’s next (action)?

Why Assessment?

Overview

 

The literature describes three main purposes for assessment.

• Assessment for accountability
• Assessment to improve learning
• Assessment for authentication

Improve Learning

Assessment is the means to improve curriculum, make pedagogy more effective, challenge students to take ownership of their own learning, and produce deeper learning. New research in cognitive science (how people know things) is rapidly expanding; assessment helps implement these principles into classroom pedagogy. In addition, many fields of study, such is Biochemistry and Microbiology, are literally exploding with new information. Assessment provides a tool to incorporate current aspects of the discipline yet keep teaching focused.

At first glance, assessing outcomes may appear threatening, because we are not miracle workers: not all students will succeed. Some students do not care and some do not try. However, we know that many students do learn, do care, and do pass our courses and complete programs. These are the students we want to engage most effectively. How can we improve the quality of learning that occurs for the majority of students in our programs and courses? It begins by using evidence (data) to make those improvements. The process of articulating desired outcomes, assessing those outcomes, and using the data for improvement, is called the assessment loop.

Assessment can, in both settings:

• Guide teaching that targets appropriate levels of Bloom's taxonomy and deep versus superficial learning.
• Provide immediate feedback, the most powerful method known for improving learning.
• Develop a conduit for diagnostic feedback to adjusting pedagogy effectiveness.
• Motivate faculty and students and invigorate professional dialogue.
• Link educational tracks and goals into a cohesive pathway for students.
• Move the institution towards the learning paradigm.
•  Assessment practices specifically target appropriate learning.

Assessment creates diagnostic formative feedback to improve student learning. The diagnostic information produced through assessment creates a conduit to significantly improve learning through corrective feedback.

Faculty-driven, classroom-based assessment focused on learning, produces a dynamic feedback loop between faculty and students that energizes the learning environment and produces tangible improvements in education.

When instructors embed formative assessment techniques into online lessons, they can obtain information to evaluate how well students are learning concepts and make adjustments to teaching plans as needed.

Academic Integrity

Studies report that 50-75% of students self-report cheating (Burrus, McGoldrick, & Schuhmann, 2007). Although it may be intuitive to think that students who are distant from instructors in online courses cheat more often than students in face-to-face classes do, research indicates the opposite.

Instructors can promote honesty with written assignments by following several principles: (a) make clear to students what plagiarism is, (b) require students to complete the writing assignment over a semester, and (c) require documentation of originality (Fain & Bates, 2005).

Promoting honesty with high stakes tests. When a test contributes more than 20% of the course grade, instructors should consider giving the test in a proctored environment.

Other methods to promote academic honesty with non-proctored tests are to create the test as open-book/notes. The instructor can make it “legal” for students to use the Internet, textbooks, and notes, but state what is off-limits, such as discussing items with other students. Instructors can use test items that require higher-order thinking so that answers are not found on the page of a textbook or in students’ notes. These strategies can support students’ synthesis of concepts, analysis of problems, and development of solutions.

Assessment for the purpose of academic integrity provides another opportunity to weave performance-based assessment into our instruction.

SLOs in a nutshell: Define what we want students to be able to do, agree on how to assess whether they can do these things, make sure our instruction is tailored to fulfill that, gather the data on the results produced, and make changes to improve students’ success in attaining these outcomes.

How this translates to online courses:

Since the objectives and SLO’s have already been set for your course, the task at hand is to look at the measures identified in the SLO’s and adjust where needed to ensure the correct results are being measured and captured.

While this is not a course on SLOs or how to write them, rather we discuss it here given that you will need to be aware of the SLOs for your course and will need to look at the measures and assessment tool to ensure that they are appropriate. In week two of this course, we will dig deep into the myriad of tools that will be available to you in the online space. In case you might want a refresher however, we have included some examples and pointers below.

 

Pointers for Writing SLOs

Student learning outcomes (SLOs) are the specific observable or measurable results that are expected subsequent to a learning experience. These outcomes may involve knowledge (cognitive), skills (behavioral), or attitudes (affective) that provide evidence that learning has occurred as a result of a specified course, program activity, or process. An SLO refers to an overarching outcome for a course, program, degree or certificate, or student services area (such as the library).

SLOs describes a student’s ability to synthesize many discreet skills using higher level thinking skills and to produce something that asks them to apply what they’ve learned. SLOs usually encompasses a gathering together of smaller discrete objectives through analysis, evaluation and synthesis into more sophisticated skills and abilities. (ASCCC, 2010, p. 13)

The integrity and value of the assessment process depends on the correct articulation of Student Learning Outcomes (SLOs). To do this, one should follow the principles below:

• SLOs use action verbs from Bloom’s Taxonomy with an emphasis on higher-order thinking skills. There should be 3 to 8 SLOs for each class or program. When in doubt, fewer is better.
• SLOs should be included in course syllabus.
• SLOs should be the same for all sections of a course. However each instructor may include additional outcomes and/or course expectations for his/her section
• SLOs should be written in language that students (and those outside the field) can easily understand.
• SLOs are typically not content-specific.
• SLOs should focus on big-picture, over-arching concepts, skills, or attitudes.
• SLOs ask students to apply what they have learned.
• SLOs must be measurable and should suggest or imply an assessment. If they do include the method of assessment, it should not be too specific, that is, a given SLO for a course should be appropriate for anyone teaching the course.

Other tips:

• Avoid starting SLOs with the words such as understand, learn, know, etc., since these indicate internal mental processes. Focus instead on what students will be able to do, produce, or demonstrate.
• Ideally, each course or program should SLOs from more than one domain (cognitive, psychomotor, and affective).
• When writing SLOs, think about how you will assess each one.

More Pointers for SLOs

Focus on what the student can do. Don't address what was taught or presented, but address the observable outcome you expect to see in the student.)

Use active verbs. Active verbs are easier to measure. For instance, if you want the students to understand how to correctly use a microscope - using the word understand is not measurable. Can you measure understanding? Instead try to imagine the outcome - Students will focus and display an image on the microscope. For this I can both develop criteria and measure ability.

Include an assessable expectation. It helps if you have clearly defined expectations concerning the criteria related to that outcome. In the above example, some of the criteria related to using the microscope would include:

• a clearly focused image
• correct lighting adjustment of the diaphragm and condenser
• appropriate magnification for the object
• an evenly distributed specimen field
• clearly located object identified by the pointer
• a written identification

Share the outcomes with faculty from other disciplines and within your own discipline. This helps focus the meaning of the statements. For instance in the above criteria the faculty may ask for clarification of "appropriate magnification."

Share the outcomes with your students. Students need to clearly understand what is expected, they are unfamiliar with the discipline specific language. This helps focus the clarity of the statements.

Modify as you learn from experience. Leave the word "DRAFT" at the top of your SLOs to remind yourself and communicate to others that you are actively improving them.

Materials and Resources: Videos

Rubrics - an Introduction (5:32 min YouTube video)

Assessment Quickies (4:24 min YouTube video)

Rubrics Presentation (6:09 min YouTube video)

Examples and information about creating rubrics

RubiStar - for creating rubrics

Links to Educational Resources about Rubrics

Authentic Assessment Toolbox

Creating Effective Rubrics

 
This Bloom's Digital Taxonomy shared with me by a classmate

Assignment: SLO Assessment Plan

Course Title: Noncredit ESL 45, English as a Second Language, Level 7 

Description of course: Level 7 ESL prepares noncredit ESL students for credit academic and/or vocational courses.  The course emphasizes fluency and communication by integrating language functions and forms with appropriate information, sources, skills, and topics. The course promotes the continued development of reading and writing skills through the writing process. (from Course Outline)

Student Learning Outcomes:

Students will be able to….

• Identify key information in a variety of formal and informal media presentations.
• Demonstrate the ability to communicate fluently in informal and formal conversations and presentations.
• Interpret meaning from a variety of authentic readings in identified areas of interest.
• Compose a well-organized, coherent paragraph with appropriate detail and sentence variety.

Origination of the SLOs: These SLOs were written by the department faculty who were teaching the course at the time, are assessed twice per year, and reviewed annually as part of program review.