progress on the digital-whiteboard-text-markup thing

After quickly (and I mean QUICKLY – it took less than half an hour’s worth of fiddling around with code while sitting on the couch watching garbage TV) building a way for people to markup chunks of text using the -powered digital whiteboards, I wanted to test it in action. It’s one thing to try it on my laptop or iPad, but digital ink is a different experience on a 50″ display. The classrooms have been so busy this semester, I didn’t get a chance to try it until this morning.

The good news: It works really, really well.

Marking up some of Bill’s stuff, on love and loss and beginnings and ends…

The drawing tools work perfectly. But as I was testing this, I realized – using the pen tool to write (as one does with a whiteboard) kind of sucks. It’s laggy, which makes it hard to write legibly. You have to write really slowly and carefully, and even then the smoothing kind of messes things up.

I had a theory that the lag was part of the system design of the collaboration carts – the display is hooked up via USB to a USB-ethernet converter, then pushed over the network to the server room where it’s converted from ethernet back to USB and then into the appliance PC to do stuff, then out via HDMI and converted to ethernet to be pushed back downstairs where it’s converted from ethernet back to HDMI to be displayed. Whew. Lots of points where things could be slowing down.

Rough sketch of the hardware that runs the Collaboration Carts™

I timed it, using the super-high-tech method of “point my fancy phone’s slow motion video camera at the screen to record 240fps of me drawing, and count the number of frames between a touch and when the ink shows up.” On the 50″ displays, it’s 33 frames. Which converts to 138ms lag between input and draw. Research suggests ~40ms would be optimal, so we’re looking at over a 3x factor. Not ideal.

So I wanted to test a “best case” scenario – running TIDraw on my iPad Pro, using the Pencil for input. This is supposed to be essentially “best in class” input, so I wanted to time how long it takes to respond to input there as well. It was 28-31 frames. 112-129ms. So, close enough to the larger 50″ display lag that it suggests the lag may not be in the hardware design after all. TIDraw could just be laggy enough that it becomes uncomfortable on larger displays (the lag wasn’t really noticeable on the iPad – it felt instantaneous – but on the larger display, you write larger, with broader strokes, moving the pen more quickly. So a 138ms lag means the ink could be several cm behind the pen, which causes all kinds of dissonance.

Next up – I’ll be tearing TIDraw apart a bit, to see if I can remove the smoothing. I think that’s what’s causing some delay there – it buffers points until it has enough to smooth, then starts drawing. Works great on smaller screens, but on larger screens… yikes. We saw a similar lag with Limnu, and I’d just chalked that up to the hardware design. Looks like there may be a flaw with the way these tools do smoothing?

Text annotation via digital whiteboards

We hosted a “TI Instructors Gathering” this morning, where we invite folks who are teaching in the Taylor Institute to come together to share their experiences and we can learn from them about how they use the spaces and technologies. This gathering was predominantly Languages profs – french/spanish/russian/french – and we got to talking about how they’d like to be able to have their students break into groups and use the “Collaboration Carts” to mark up passages of text. That’s not a thing that’s readily done – there are web-based text annotation tools, but not ones that are based on ink. Having students sketch on text is a useful activity, pedagogically, and so I got to thinking…

And a couple of hours of work later, I’ve gotten that feature working in (which is what powers the digital whiteboards in the Taylor Institute).

Digital whiteboard markup of a bit of Shakespeare…

TIDraw works with images, so text needs to be converted before loading it for markup. There’s a utility for that (in the “Texts” button in the top menubar), and converted texts are available in a Gallery – using the same code as the Save and Template features.

This could be useful for visually marking up short passages – it won’t handle pagination, so longer passages won’t work. But – it’d be handy for marking up stanzas or short passages.

I’ll also work on a feature that will let people upload images – imagine being able to use the whiteboard tools to markup a painting, or an electron microscope image, or an x-ray image, etc…

Design thinking, with giant lobsters

Robert Kelly has been hosting his Design Thinking course in the TI for the past several years.

I was away over the summer, so missed the latest instance of the course. Giant lobsters! Thankfully, they documented the course and published a video of the shenanigans.

Giant lobsters invade the TI. And other interesting things.

It’s a bit of a challenge when the course is going on, because it’s so profoundly unconventional. Classroom? Classrooms? Nah. Buildings! But – it’s been amazing to see what the participants do, how they work together, and what they build.