Hearing Your Student Evaluations Differently? – Faculty Focus

Not long after I had received most of my student’s mid-semester survey results, I came across an AI tool that would create a song. The tool is Suno (suno.com) and it uses AI to create both the music and the lyrics. For some odd reason, I wondered if it might create a song based on by student survey results?  

I needed lyrics for the song, so I took the survey results and placed them into Google’s Gemini to develop a summary of the results. The summary was not poetic nor lyrical, so I asked Gemini to write some song lyrics based on the survey summary. Silly I know, but that is how this story started. I then added the lyrics to Suno and chose ska/reggae as the style of music. It was pretty fun to hear about my class though a happy, upbeat song. The students in the class got a kick out of it too and the faculty I shared it with got a good laugh.  

And there it was, a song about my class and how the students felt about it.  

Having spent the last 20 years as an instructional designer, I have worked with faculty enough to know how difficult institutional surveys can be. Often, the most disgruntled students point to course deficiencies and that is what the faculty remember. Then there are deciles and scores and comparisons and the worry that these numbers will negatively affect the tenure process or rehiring as a part time faculty. In general, there is a negative vibe around these surveys and how they are used. I wondered if those single page pdfs could be shared in a different way? Maybe someone singing the results to them? Maybe just a friendly voice letting them know about the things they did well and the things they might improve upon.  

I thought about how different the experience of a voice might be when delivering the information. Might it “feel” better? Worse? 

I had been playing with Google’s recently released NotebookLM (notebooklm.google.com) and I wondered what raw survey data might sound like in the “podcast” feature. I started with just my self-created mid-course evaluations and it was really interesting to hear two “people” talk about my class. I wondered what other “sources” I might give NotebookLM to fill out the conversation and how I might customize the podcast.  

At first, I added the course goals and objectives. But as you can also add YouTube videos, so I added my seven-minute course introduction from YouTube. The class was online, so I added the QM standards and because the class was about pedagogy and technology, I added the ISTE standards. Adding these other “sources” really started to flesh out the podcast as the successes and challenges were framed against existing standards in teaching and learning. Listening too it kept me present on the topic of improving my class and now I was hearing how my student feedback stacked up against common learning practices.  

Soon it was the end of the course and I had asked the students for final evaluation with a few broad questions. They all wrote quite a bit about their experiences in the course, and I added it to the sources in NotebookLM. I also added a video about open pedagogy by David Wiley and another from Stephen Downes. I included a piece of writing about care and inclusivity in higher education. By adding all of those, the podcast was becoming very detailed and full of ideas for improvement backed up by the solid resources I had provided. 

I added an AI generated summary of John Dewey’s thinking about learning and another about the same from Carl Rogers. In this way, I was getting the information to include my heroes and favorites in the field. The rationale the podcasters presented for course improvement was backed up by a lot of good thinking! While I started with just the mid-quarter evaluations, as of today, the list of sources looks like this: 

Mid-Quarter Student Survey  Course Final Reflections by Students  Institutional Course Evaluation  QM Higher Education Standards  ISTE Standards for Teachers and Students  High Impact Practices in Higher Education  Course Description and Objectives  College Program Goals and Objectives  Care and Inclusion in Higher Education  Carl Rogers Theory of Education  Introduction to Open Pedagogy – Stephen Downes  John Dewey’s Theory of Teaching & Learning  Open Pedagogy – David Wiley  All the Course Announcements  Class Introduction Video  Reflections on Teaching with Google Sites 

Then, perhaps the most deeply impactful thing I did was I customized the prompt to make the podcasters speak directly to me, Todd.  

Up to this point the podcasters were simply talking generically about a class, now they were talking to me directly by name. It made it feel far more personal. It was more personal. And it was kind of creepy, but it made a difference.  

One thing I learned was that even if the speakers are AI generated, it feels good to hear positive things about your course and especially if the kind words are backed up with how some of your heroes would agree with what you are doing. The podcasters always start out with praise and that feels better than the charts and decile numbers on the university student survey.  

I keep trying to get the podcast to really focus on improvement and thus far I have already made a few adjustments to the class based on suggestions they made. Those suggestions were alluded to in the student narratives, but the whole of resources I had included in the NotebookLM made those comments seem different. I don’t know how to explain that. Maybe it was that in the student narrative it was one or two sentences among many and in the podcast, it was worded differently, and more detail was given. The human voice is amazingly powerful, even when it is not actually human.  

Where am I today? Well, I started seeing if I could transform the two podcasters text into a single person giving me the same advice. I am interested to see if a single voice feels different? It has met with limited success as all of this has been done with free tools. There are time limits and character limits on the free versions of many of the tools.  

Google’s NotebookLM has a beta feature that allows you to ask the podcasters questions. I have learned that if you state your name, they refer to you directly. I have also learned that the answers they provide are pretty amazing and that you can really drill down to pretty specific detail around assignments and course structure.  

I have also been sharing this with other faculty, and I have had nothing but good feedback about the experience. One noted that they thought the podcast would make a great advertisement for their course. They also said they had wished that they had the tool when they were writing their tenure package as it combined a good deal of disparate pieces of information into a single space. Another said that hearing the student feedback was a more positive experience compared to reading the data. I also heard about the concerns they have. Concerns like privacy and the AI tools making mistakes. To that end I would say, don’t put student names in the AI tools and make sure to listen carefully to how the sources are expressed.   

And now, NotebookLM allows you to create narrated PowerPoint type presentations based on your notebook resources! 

If you think this might be a way to improve your teaching, NotebookLM is a free tool and you can give this a try right now. https://notebooklm.google.com/ 

Listen to the evaluations discussed above:

Ska/Reggae song about the class: https://suno.com/s/xLwnhEqgRuahr9I6 

Most recent version of podcast about the class: https://on.soundcloud.com/KhLdMLmifT1OIgYlyG 

A video from NotebookLM about same topic: