VOTF Episode 3: Schoolhouse rock: Are captions critical to classroom comprehension?
About this episode
COVID-19 forced a lot of social adaptations, some good, some bad, some ugly. One of the hardest was the move to fully distanced or remote learning environments, especially for disadvantaged or differently-abled students. But one positive outcome is that many new accommodations have sprung up as COVID changed the educational landscape and virtual classrooms were forced to become as friendly, accessible and compliant as possible. As virtual classrooms have become more of the norm can we now consider captioning a critical component of student comprehension?
Learn more about Habitat Learn here!
Transcript
Sam: Welcome to Deepgram’s Voice of the Future podcast, aka Our Favorite Nerds. At Deepgram, we’re obsessed with voice, and this podcast is our exploration of the exciting emerging world of voice technology. I’m your host today, Sam Zegas, VP of Operations at Deepgram. And our guest today is Dan Goerz, CEO of Habitat Learn. Dan, thanks for being here.rnrnDan: Thanks for having me.rnrnSam: Great. Well, Dan, this is our favorite nerds after all. So why don’t you tell us a bit about yourself and specifically what kind of a nerd are you?rnrnDan: Yeah. I’d have to say I’ve thought about this for a bit. So I think I would be considered an antique and an innovation nerd, I’m kind of obsessed with both things. Yeah. I really like antiques and learning from the past, and how they used to design things, and how in their own way, they’re innovative products and services even if we don’t consider them that now. So, yeah, I’d I’d consider myself an antique nerd.rnrnSam: That’s interesting. It’s a cool way to think about innovation by studying innovations that are no longer current, but which were important evolutions of some past technology at the time.rnrnDan: Yeah. Yeah. For sure. Yeah. Definitely. We forget how we got here in some ways in the world of technology. And Yeah. Even equipment is is is technology before things got digitalized. So I think that’s, when you think about antiques, there’s the evidence of craftsmanship, which is so front and center in the way that you evaluate an antique and think about the innovation that they were making.rnrnSam: Yeah. Definitely.rnrnDan: I’ve got a drink cart at home, which are popular now. but they have little locks on it. So even back then, they were hiding their booze.rnrnSam: Great. So you’re CEO of Habitat Learn, tell us a little bit about what you do at Habitat Learn and what your vision is.rnrnDan: Yeah. So Habitat Learn, obviously based on the learn part, we are working with higher education. And so we develop our products with our clients. So we like to consider ourselves special that way, is that we like to build solutions with schools that we work with as partners instead of kinda thinking we know everything and gonna build them something and then try to convince them by it later. So we really enjoy working with students and different stakeholders. and to make sure we’re building good products for them. Higher education is going through a big transformation. Obviously, COVID helped with that to make it more digital, but even just looking at how can they give access to all learners. And so we use the universal design for learning principal framework So all of our our products are designed for all users in mind, not just the majority. And so we have build products and services. So we do note taking to have a note taking application. and we have note takers that provide that service as well. But we also use captioning. So we have a program called message driven that uses AI speech to text captioning, and then a human just corrects it so that anyone needs in a percent accuracy or close to that can get that live. And then the other co product that we have is we we promote digitalized learning as well. So we have a smart camera that goes in to a classroom and can broadcast to a wider audience in our platform called Coral. So, yeah, it connects the physical classroom to a digital audience. So yeah. So we’re watching a few products and services that all have cool names that relate to habitat. so people can buy the individually. But, of course, the Habitat works better if, you know, everyone’s there.rnrnSam: That’s great. So you’re really building products to enable the next generation of learning, in particular, digital learning with an eye toward making it equitable for people who have different ability levels.rnrnDan: Yeah. No. Exactly.rnrnSam: Yeah. You’ve probably seen the evolution of the classroom environment change even more rapidly during COVID. Tell us a little bit about what the emerging needs are and how you responded to them?rnrnDan: Yeah. No. Exactly. So a good example of that would be – So before, people would go to the classroom or not. So let’s say they were late or not there, there’d be different reasons people could assume why. Right? And then when we went digital, the the attendance increased a lot. Right? And so they’re finding that when things went back, there’d be a need to stay digital to give more flexibility. And so, for example, schools are basically to learn more about their students. Right? So at first, there’s a still stereotypical version of a student. Right? but then they started realizing, oh, you know, they’re students where maybe they’re from a big family. And when they went digital, they could, you know, you could hear their their younger sibling something screaming in the background of, you know, when they’re asking a question. And so our technology, for example, uses AI for, like, a voice focus. And so it takes out distracting noises like fans and dog barking and ambulances so that others so that person feels that if you can ask a question about other noises being heard, And also, the audio quality is is good for the other students that are are there for the lecture. So that’s probably the biggest example of what we couldn’t expect until things went digital.rnrnSam: It’s interesting. I I’ve read a lot of articles recently about the participation in some demographic groups in learning as as going down in the age of digital learning. What do you think are the challenges of learning in that sort of digital environment? And how do you address them.rnrnDan: Yeah. Good point. Yeah. So for some people, yeah, there’ll always be a need for in person, and then for ones that remain digital. I think the biggest pressures going forward is if you’re one instructor who do you you give your attention to. Right? You have a a physical audience, you know, a digital one. And so it can be a real divide. And so going a step further than having, you know, a device in the classroom where these students remote being there’s one. It’s smart devices adding more technology, so it’s easier for instructors to engage with both students. So, for example, be like, hey, camera. What’s you know, is there any online questions right now? Things like that. to make sure that those students and then through the device can speak through ask questions and be interactive live in the in the lecture itself. So those options the other option too is especially for learning that if you missed the lecture, you you can be able to watch it later. a good life happens. Right? And sometimes you miss and yeah. So it it it can really help. having those options.rnrnSam: Yeah. Yeah. Thinking about all the different aspects of a classroom environment, like whiteboards and projectors and and the different things that people might recognize from being in classrooms twenty years ago, you know, higher education is is changing rapidly itself and the tools really need to evolve along with it. So it’s an interesting challenge.rnrnDan: Yeah. For sure. Yeah. It it’s kind of always been an irony with universities and colleges because the first computer was at the university. Obviously, they invented all that. But like you say, for a long time, classrooms, the most technology they had was a projector. So yeah. So, yeah. No. It’s it’s definitely changed a lot recently.rnrnSam: Yeah. What’s the hardest part about the work that you’re doing?rnrnDan: I would say our client has a lot of stakeholders from government grants and involvement of massive refunding post secondary. So their concerns have to be met, which might seem like, a broad statement. But I’ll give an example would be as we’re streaming video and stuff, you know, governments have a concern and and there’s laws around where the streaming can be housed. And so a lot of our techs don’t then maybe know that. And then by the time they get through the vendor process, their product can’t be bought because it’s, you know, the servers in a wrong country to to you know, their services are paid by a grant or their technology is gonna be paid by a grant, but the grant has rules that you didn’t know about, and so it might not be covered. And so the school’s gonna go with maybe similar technology, but covered under a grant. And so I think that part is probably shock you when people get into it is they think, oh, it’s cool enough product. It should sell, but there’s these little things in the vendor process that can slip you up if you’re not recognizing the amount of different stakeholders. Also, faculty have valid concerns, right, about, you know, their lecture content being safe and secure. And so if we wanna address those, same you know, everyone’s got concerns and needs to be listened to versus some of one client type concept you almost have multiple.rnrnSam: Yeah. Definitely. So there’s there’s a a procurement challenge and there’s an implementation challenge, and you’re dealing with these big institutions. I imagine there’s a lot that you have to consider there. Tell me a little bit about where artificial intelligence and speech technology fit into the thing that you’re building.rnrnDan: Yeah. No. So it’s a big part of what we do. And so in the accessibility market, speech to text is is massively important to make it a world more accessible.rnrnSam: Mhmm.rnrnDan: And with kind of coupa technologies, computers, have disabilities. So, basically, a computer is deaf and blind. They can’t see or hear. So AI technology have made them smarter to speech, ten x is is one of the biggest ones. right, for it to be able to understand, predict, and and and help people learn through massive amounts of information. So for us very specifically, we use the most for captioning for, obviously, students that are deaf and hard of hearing, but also for just anyone who wants to read the captions. A lot of people, if they have the sound off, you know, for example, for a lecture, but they want to read it instead. or in the English versus second language. But those type of speech modules really do have those users in the end. Right? Be able to read the content instead of just having audio only options. And then later on, of course, once somebody’s put into text, computers can do a lot of cool things with that. So being able to search or past lectures, things like that become easier because they’re they’re captions and and they’re searchable. And so, yeah, So they’re successful to the computer to do the more cool things with it. So that’s what’s, I think, really cool about the speech to text technology that exists Yep.rnrnSam: Yeah. It’s it’s a cornerstone of making information available and also making accessibility sort of an a a a feature of the of an equitable product that you’re trying to build if, you know, people have different understanding levels or they maybe hearing impaired or something like that. Just making it available to them is is key.rnrnDan: Yep. Yeah. For sure.rnrnSam: Yeah. I I imagine in the context that you are operating in in education, there’s a difficulty around accuracy on long tail vocabulary, you know, in higher education there’s probably lectures about scare topics where they’re talking about words or maybe, you know, names of individuals on a history lesson or something like that, where it’s difficult for a speech solution to get all of that accurate. What how do you approach that?rnrnDan: Yeah. Good point. So it becomes very important for higher education for that accuracy to be at a pretty high level. And so what we do is we kinda combine the best of both worlds, so especially in the industry, these are things, okay, while AI is good, right, for speech to text, but then, like you mentioned, there’s these potential holes, right? And so but then you would have human transcribers who are basically just typing as fast as as they’re hearing it. But that’s a very expensive pool, which may you know, once somebody’s more expensive, that’s very limited for kind of worst case scenarios of when they would use it. So what we’ve done is speech to text has made by having a human element to it is it kinda de-skilled that job, so they’re not having to do all the hard lifting and typing everything. Right? They can just do the corrections. So when things are misheard, they could do that correction or add this kind of custom vocabulary, which I think is really good because I think higher education almost, like you say, has its own own language to enter the courses. Right? Biology or even the culinary schools, right, of this, you know, french words after things. So yeah. So, you know, it’s it’s very important to have still have some human element to improving the end service.rnrnSam: That makes sense. And this is obviously one of the ways that have a top line, a Deepgram partner with each other. Deepgram are always thinking about what new use case models we need to build and over time thinking about how we can make affordable, scalable, but highly accurate models that can identify the long tail vocabulary in a number of different areas, whether it’s a scientific topic or it is a specific industry use case or the ability to just recognize names that are set. These are really relevant issues in the way that we wanna consume the product to speech technology that are technically quite hard, but it’s something that we love to think about.rnrnDan: Yeah. No. Yeah. It’s a great partnership that way. Yeah. Very aligned. Especially, yeah, college grads. It’s just a list of names at some point. Yeah. It’s very tough. Very important.rnrnSam: Tell me a little bit about where you see the risks for higher education not adopting these sorts of technologies. If if when do you when do you see five to ten years down the road for a university that isn’t making these investments in speech tech?rnrnDan: Yeah. So for yeah, for school that don’t take these next steps basically it impedes them from competing with other schools. As you probably know, there’s thousands of schools in Canada and the U. S. And so students have lots of options, and so they’re going to go to a school where that suits their needs for learning and that’s modern. That can offer not only digital education, but but like we’re saying before something that has multiple means to access it. So even if it’s just agreeing actions, architectural parts of it, just helps them learn, and it’s a it’s a higher level of education, to be honest. And so it really helps schools brands and be able to compete locally and nationally because students, and we’ve already started to see that actually in our markets that we’ve been doing this in, is that students will change programs to go to the school with the similar program that has our technology in it.rnrnSam: Wow really?rnrnDan: Yeah.rnrnSam: I imagine that so you’re talking about people selecting into institutions that have adopted a certain kind of technology. I wonder if you have much data at this point or if universities have much data about the quality of learning itself like this does it really make an impact? And is that something that you’re interested in studying going forward?rnrnDan: Yeah. Very good point. We have actually a pilot that we’re just finish formalizing, and we’ll be rolling out this summer with a university. And I don’t know if I can to the name, but in the States. And so, yeah, we’ll definitely be publishing that later. But, basically, all of our tech all of our ecosystem is going in to courses and classrooms where students are doing well, basically. And there’s you know, a need to kind of improve student grades and stuff and see if see if it makes a difference for them. And then they’ll they’ll they’ll do a wider you know, implementation after their cross campus, but start in kind of the worst case scenarios and see if it helps.rnrnSam: Yeah. I know that your focus right now is on higher education. Do you see yourself starting to work on secondary education or or primary education at some point? And how are those needs different?rnrnDan: Yeah. Yeah. Like, we we we’re always kinda looking at different verticals like that. the needs are are the same, but probably different, I think. So, obviously, things for younger students would be slightly different in a sense of especially even for speech to text, there is you wanna make sure that you’re not catching things that are inappropriate in class. And even there’s different states and laws about, you know, having images of, you know, people by a certain age and stuff like that. So just following those types of rules. So it’s a little more complex, I think. COVID’s probably helped with that. It probably modernized some of those laws. But I know, for example, when I was in school, in the in the region that I was in, if someone’s parent, let’s say, took their family, I mean, a class photo of us, they would have to get permission by each parent. So So you think of, you know, digital learning, if you’re having remote students, you know, is all these things to consider. But I think, you know, COVID has definitely helped with with changing people’s perception of digital learning.rnrnSam: Yeah. You know, most of the work that Habitat Learn and Deepgram are doing together today is, you know, Deepgram provides fundamentally speech to text. And so we’re hoping to generate things like captions, but to tie into what to the point you were just making, the next frontier in speech tech for Deepgram is really about understanding features. It’s being able to understand the meta of what’s going on in the conversation and then, like, okay, this might be a sensitive word or topic. We need to treat it differently or okay, you know, this is the opener for a lecture. Great. We’re gonna mark it as such, and then we’re gonna say this is, you know, one section of the lecture that was being received and we’ll sort of market as such. And that is a kind that speech understanding is really a a brand new frontier in speech tech that a lot of companies, but especially Deepgram are really leaning into now. And I I’m very excited about the future of where that will head as that technology matures. Over the next five years, it’s gonna be a very exciting space.rnrnDan: Yeah. For sure. Yeah. It’s very needed. I agree. I was just gonna say, I think, the it’s understanding after that because at the end of the day, it’s all just raw text. which is great, live in the moment, but understanding it after even, you know, what, you know, even things like especially if you’re doing digital learning and you have online content, what you wanna be able to do is kind of like the Netflix of education after. Right? So you may not wanna watch the whole lecture again, you may wanna just go to points that seem and most interesting, and you kind of decide that it learns your traits, but all that’s gonna be made possible, like like you’re saying, by having more of an understanding of that transcript.rnrnSam: Definitely. You know, we the trend that we see at Deepgram is toward a an ever increasing humanization of speech technology. And what we mean by that is that when a human listens to a conversation, we’re listening to the words themselves, which is the speech to text and and from a technical standpoint. But we’re also creating a narrative a meta narrative in our mind of like, you know, this is why this person is telling me this thing or here’s where I should bucket this piece of information in my head. And so as we teach machines how to do that better and better, products like habitat learns will get stronger and stronger and helping people to to manage the content that is being delivered to them by teachers.rnrnDan: Yeah. For sure. Yep.rnrnSam: Great. It’s been great talking to you. Something that we always do toward the end of one of our episodes. is to remind our listeners just how far technology has come even in our own lifetimes. So I’m gonna ask you to explain a piece of outdated technology and to explain it like you would to a kid who was born after two thousand ten. So this person is about ten years old. They have lived their whole life on computers and I’d like you to explain to them how a typewriter works.rnrnDan: Okay. Awesome. So being an antique nerd, I have an old typewriter. Yeah. And well, I haven’t it’s it’s printed, but I’ll show this to you. So if you can see here, this says noiseless. So this was this was technology back in the day. And this was this was really state of the art. So before this, people would write with pens. Maybe even feather pens. So you needed ink, feathers, anyways, this replaced that. It’s pretty cool. They’re pretty heavy. And this was not noiseless because it’s like a keyboard, like your computers, that sounds too strange. So it started, you know, the idea of keyboards. And then as you pressed it, you don’t need a pen and name arc because there’s ink ink ribbon inside of it. So that replaced that, and then it would just hit the paper. If you made a mistake, then you would lose a lot of paper. But and then yeah. So don’t make mistakes. That that’s that wasn’t possible. There was no control v or control c or any of that that second there. So, basically, yeah, you you’d stop that button. ink would go on the paper. So if you got that stuff, you’d have to change that. So they innovated it because it was they were really loud back in the day. So but I’m not sure if this how loud this one was, and and this is what they were doing to innovate. This would sound like. Not not, you know, not too bad. Alright. Still pretty loud, to be honest. Way louder than just your keyboard.rnrnSam: Yeah. I remember when I was on a typewriter, and it was incredibly not only was it loud, it it was loud because you really had to, like, mash the keys. It was it’s it’s not easy to type on one.rnrnDan: Yeah. It’s true. It’s true. So so they’re thinking about it, you know, it’s annoying. too loud. So that’s probably, you know, what people would complain about. But but it didn’t evolve much after this. They had electric ones, same type of principle. But then after that, it scared the world when computers can help. because computers eventually replace these things. And, of course, now your keyboard’s a lot quieter than that. and computers. Do crazy cool things now. So but this is part of the evolution of the computer. Yeah. Great. Now we live in the age of silicon.rnrnSam: Yeah. Thanks for playing along, Dan. And thanks for being one of our favorite nerds. been awesome talking to you. To all our listeners out there, thanks for tuning in, come check us out for more information about Deepgram or for Habitat learn, that’s habitat learn dot com. And, of course, you can always find us at deepgram dot com and at deepgram AI at all of our socials. So with that, we’re out. I’ll catch you next time.
