Ok y’all, this week I personally believe has been one of the most productive and successful week so far. The game plan for this week was to really go to battle with WebRTC and finally win this time. Guess what? We did! Well, so far but we’re still winning. :)

To open the week, we met to discuss the week’s game plan aka our battle plan for WebRTC. As I stated briefly in week 4 blog, the battle plan was to get GCP Web Speech API and MS Azure to work in WebRTC in Norman’s coded script that he had already wrote for the RTT. Initially, we predicted this to be quite challenging because when we first tried a couple of weeks ago, we hit a doubled, no tripled layered brick wall. However, this time it was different. We had more tools in our toolbox and we were able to break through the hard wall.

To break my side the wall, I stayed working with MS Azure as I have been the previous weeks and getting that specific ASR up. To get it connected, I really had to first understand WebRTC and how Norman’s code to capture the keycodes and prints to the textarea for the local and remote users to see. Once I was able to do so, I created a new script that would use MS Azure’s ASR and print its STT service to the textarea Norman created. But of course, we had some problems; there were “undefined” messages that kept popping up, the text that the users typed would disappear and be replaced by the streaming STT sevice, and the last one was on the local user’s end - when they delete any of their texts, it would show up in the lastRecognized var that would print when the script switches to the ASR. I with enormous help from Norman, was able to solve all of them with adding codes and functions to remove the undefined messages, keep the text that were typed, and remove the texts that were deleted by the user. That took us a couple of days but we were able to get it to successfully work with switching between user RTT and ASR in the textarea with no errors. First wall down!

Though, there were some suggestions to implement a style to change the color of the font or the weight of the RTT or ASR to show the difference between the two when it streams to the textarea but as a team, we agreed to hold that to next week when we got everything else down. Because the next thing on our list was going to take some time and had higher priority.

Now for the next thing, it was to add an option in the hamburger menu to show the entire transcript of the chat in a div or textarea in front of the page. This feature is important because users can refer to previous conversations or go back to a part they missed. It is also critical for saving the transcript in a file after the chat is over for later references. To start, the main goal was to have the transcript of the local user’s ASR shown in the div in front of the page. Hopefully it could be live as well. With luck (and skill), I was able to tackle that wall and have an option that would show the transcript of the local user and have it also stream live. So when talking - the chat area as well as the transcript div will stream the ASR. I also had the transcript div print the timestamp of each ASR so that when needed, you could refer to the transcript and find exactly where you missed or need to review.

So great, right, we had the transcript div up and it prints accurately. Ha nope, our battle for the transcript div is far from over. Now we need to get the ASR from all users of the chat and have it print in the transcript div. Meaning, I would need to look into WebRTC peer to peer connection and have a function that would listen to the remote user’s data streaming to extract the text. But good news (for me), it’s Friday so we can take a bit of a breather from trying to figure out this code and script and come back with a fresh mindset on Monday.

Now, you can see why I opened this blog cheerfully, this week was one of the best weeks in the past 5 weeks. We got a part of one of our biggest purpose of this project - to test ASRs with WebRTC completed. I was able to connect one of the ASRs to WebRTC - MS Azure. So you can bet I’ll have a great weekend! See you guys next week, hopefully we’ll be just as successful as this week.