Effectively Assessing Digital Learning

“I’m working on a technology grant. We have to assess students in grades 5-8. I need a free, cheap digital literacy assessment for 5-8th grade,” Janet said. “Do you have any ideas?” The question made me pause for a moment, casting back over time. Where had I seen such an assessment before? Mankato Public Schools had one, a simple listing of technology skills for students. That had been over twelve years ago. Learning.com has such an assessment, but it costs money. Then, I had a flash of insight. Was a list of digital skills the best way to approach this?

In this blog entry, we’ll explore a few of the challenges with digital skill lists. The road to digital assessments is fraught with good intentions.

Good Intentions

The reason why is simple. The question make some suppositions. It suggests that student technology assessments center around digital SKILLS. Instead, digital learning is the desired focus.  I’m tempted to offer a list of digital skills/tools that students need to be familiar with. We achieve effective digital learning when students exhibit control. That is, control over time, place, path, and/or pace (Source: Digital Now report as cited by Dr. Chris Moersch). This suggests that students must have control over their own learning.

How Students Learn Best

When you consider how students learn best, you are not alone. We know how they learn best. This Adobe Education Report highlights best learning strategies. It suggests that students learn best when teachers:

• Use digital media (videos/podcasts)
• Engage learners with hands-on experiences
• Scaffold team-building skills
• Pose real-world problems for solution development

Assessing Student Learning

As you might imagine, assessing student learning is not a spectator sport. Rather than administer a survey, walk the campus to see what is happening. Use an observational framework, like HEAT or Triple E, to gain insight. You can’t assess students’ digital learning without considering how teachers teach. You could plan a survey with simple list of technology skills, but how effective is that? This will gain you a self-reported, invalid, and perhaps unreliable instrument.

Instead, use a research-based instrument such as the two observational frameworks mentioned. Still want something your students can pass out? Brightbytes’ Clarity has a student survey that provides some insight. Or, if you are gathering information, consider using the Speak Up Survey questions. These would provide some information, and you might mix-and-match the following:

• Students Grades K-2
• Students Grades 3-5
• Students Grades 6-12

You will want to check with the Speak Up Survey developers due to copyright.

Want to try something else?

Create Your Own Survey Instrument

If you want to create an instrument that is not researched (valid or reliable), you can try this approach for fun. Take the HEAT Lesson Scoring Guide. Develop questions based on that framework, which will result in a shorter survey. This may focus less on digital skills and more on digital learning taking place.

You can use a tool like Microsoft or Google Forms to collect data. This would capture the results and then summarize them in graphical format. In the next section, you will find sample questions you can customize for your own use.

Sample Student Assessment

Below, please find an unscientific assessment keen on measuring digital learning.

Question #1 : What kinds of things do you create with technology? (Check all that apply)

• Videos
• Audio/podcasts
• Slideshows with audio or video
• Typed documents I share online via the web (e.g. blogs, wikis)
• Typed documents I turn in to my teacher or are printed for sharing
• Information with numbers and graphs
• I don’t use technology to create anything

Question #2: How do you show what you know?

1. I decide what problem I’m going to solve and I work with others (outside my classroom).
2. I decide what I am going to do when working on a project with others (inside my classroom).
3. I decide what and how I’m going to work on something using technology.
4. I do what the teacher tells me to do.

Question #3 – Does the work you do as a student impact a real life issue, theme, or problem?

1. Yes, I use technology to create solutions for real problems.
2. Yes, I use technology to explore real-life problems to solve with my teacher’s help.
3. My teacher tells me what real-life problem I will solve.
4. My teacher tells me what to do; I don’t know if it’s a real-life problem or not.

Question #4 – How do you decide what technology to use?

1. I decide what technology I will use to learn something.
2. I decide what things (not technology) I will use to learn about something and do it.
3. My teacher decides what technology and/or things I will use to learn something.
4. Technology and/or things are not needed to learn something, but they are nice to have.
5. Technology and/or things to help me learn are not available or used.

Assessing Digital Learning

Don’t like these questions and responses? They may work for fifth graders and above.  That aside, how are you assessing digital learning for students and staff in your organization?