OER Quality Evaluation Criteria
by Megan Simmons 4 years, 6 months agoThis week we shared what high-quality resources look like in our work, and different tools that have been developed to evaluate quality. Please share what your quality evaluation criteria is when you are assessing resources to use by replying below.
For me, a resource is quality when it is:
What is my evaluation criteria when assessing resources to use? I appreciate you asking this question. It certainly gave me a pause. I guess my answer is: it depends. The criteria needs to align with the intent of the learning activity/object. So, without knowing the intent of the resource, it is hard to pinpoint what my criteria will be. I appreciated the Achieve rubrics you shared with us. Rather than a fixed set of criteria, these rubrics represent a system of evaluation that underscore the intent piece. I see that there are 8 of them and they were created or refined in 2011. I wonder if there is any ongoing effort to expand the list.
I would use a quality evaluation criteria that is based on clear standards, I would like to use a well-designed rubric that has content validity. To me, a rubric would not be helpful if the scoring solely used subjective decriptors such as "some", "many' or "satisfactory". Instead, I would have more faith to judge the quality with detailed descriptors that objectively list degrees of quality evidence criteria. Concerning OER evaluation criteria, the Acheive Standards rubric appears to be the closest to what i would like to use.
For me, the resource is quality if it is (1) relevant to the current practices in education, (2) grounded in research that can be verified, and (3) provides for the opportunity to incorporate cultural diversity. These are important factors for all work in education.
I agree with the criteria in the rubrics shared through our hub. I think the Achieve rubric is clearly the most comprehensive, but the downside of this is that it could be rather daunting to apply on a regular basis--for every piece of content created or resource selected. If you work in a setting where you are a one-person or small-size department, using such a tool all the time could be prohibitive in being able to make any materials available.
I also agree that it may depend on specifics of the resource that the rubric will be applied to--so applying a particular rubric that evaluates interactivity and technical quality of the graphics, etc., wouldn't make sense to some types of more limited resources.
I recently developed a rubric for my Library's Faculty OER Review Honorarium, where the whole goal was to have faculty members review existing OER materials--in this case textbooks. I used a rubric by MERLOT plus another by BC Open Campus to create one for our faculty to use. The adapted rubric I created had its shortfalls too, so no tool is going to be perfect; however paying attention to Learning Outcomes, "Quality" (maybe Accuracy is a better word) of Content, Effectiveness as a Teaching Tool, Adaptability, and Inclusivity/Cultural Responsiveness will certainly be important.
I definitely like the Achieve Rubric, but I found myself wanting to add the culturally responsive elements too. Ideally, a combination would be beneficial. Since the achieve instrument is dated 2011, I think it may need revisions that reflect current needs.
I was talking with some colleagues this week about OER and also wondered how P-12 students and college students would evaluate OER?
I wonder what would be important to them? I think they may be interested in aspects that I have not considered.
Just a thought....
Good evening,
As I think about quality evaluation criteria my role as a NAEYC Commissioner for Higher Education Accrediatation and the training that I recieved to work in this capacity. A huge part of the work involves me assessing the program effectiveness based on their objectives/outcomes. In addition, how the goals are met and the efficiency in which this is done. With this frame of mind I believe a high quality evaluation criteria would include some of the same information.
When looking at the Achieve Rubric I believe that it definitely was the strongest rubric. While it woould need to personalized and updated but would be a good start.
As Veleisha stated in her post I too am curious about student use. How do we get students engaged? I also submitted a grant on Monday and I indicated that we would be using OER as a part of the prgoram. So, I am hoping that they don't ask me more until I get more informaiton. LOL
Ty