Thoughts on the mythical School Aggregator (EduGlu?)


I've been giving some thought to the "school aggregator" that grew out of the discussions around Northern Voice. What kinds of things will it have to be able to do? Types of interfaces? Explicit and implicit data and metadata? How to manage caching of items, and manage displaying the potentially hundreds of thousands of bits of content that will be pulled into the system over the course of a year? And how to present cohorts/classes/years within this? How to allow students to add multiple data sources, and tag it for use in whatever class context(s)? How to let students and teachers mine the aggregated data to get what they need/want? Lots of stuff to chew on here.

Brian's students have been off to a great start in their AggRSSive project - and they have plans to make it even more kick-ass. Tyler's described plans were pretty much spot-on to what is needed. Not complete, but a darned good start. Can't wait to get my hands on that...

At Northern Voice, someone from a genetic engineering organization (didn't catch the name of the guy or the agency - I was sure it was Genentech, but others were sure it wasn't) was describing a mind-blowing RSS-based workflow that he's using to tie research, automated lab results, individual publishing, and lots of other sources together into one interface. They have some AI ninjas crunching everything to make sure it gets to where it needs to go, and to start making connections between stuff. He mentioned that as a result of this AI-based aggregation, they were able to make a completely new discovery that linked two previously unrelated topics (proteomics and something else...) Very cool stuff. We should see if they can give a tour, and if they'd be willing to share with the rest of the class.

In the meantime, I just took a quick romp through SourceForge to see what else has been done in the area. Not a lot, unfortunately. But, I did come across a rather cool server-side aggregator that I hadn't heard of before. sux0r appears initially to behave like others (Feed on Feeds, etc...) but - it doesn't have categories. Well, it does, but not in the traditional sense. You create a set of tags, and then proceed to flag aggregated posts as belonging to any of these tags. After a while, the bayesian magic has enough to chew on, and it begins to automatically tag incoming posts. Latent semantic analysis to apply folksonomies?

Anyway, although the concept is cool, and will form an important part of EduGlu, the current incarnation in sux0r won't scale to thousands of feeds in thousands of categories, over dozens of years.

OK. Enough thinking about this stuff for now. Back to work...


See Also