Drupal + Aggregator2 isn’t quite EduGlu
I just took a few minutes to install a fresh copy of Drupal 4.6.5 along with the Aggregator2 module, which provides Glu functionality to Drupal. You can provide a bunch of feeds, and it sucks them in as if they are native content to that instance of Drupal. It tries to assign taxonomy/categories based on the data in the RSS feed, and provides some interfaces to view the feeds and items. I set up a prototype on my desktop box (so it might not be running all the time, and may disappear if my new box ever shows up), but it’s there to play with. I’ve pre-seeded it with my blog, flickr, and del.icio.us feeds, and took the liberty of adding the same for another couple of folks (Brian and Alan) to see how it handles multiple users adding data.
My initial reaction is that it does the Glu stuff really well. But the query interface isn’t nearly sophisticated enough to manage a full-on stream of hundreds (or thousands) of students pumping content into the system. Also, the Autotaxonomy module (the one that converts RSS categories into Drupal taxonomies) doesn’t break the categories into individual items. If one post has categories of “eduglu prototype” and another has “prototype eduglu” - they are stored as separate distinct items in the taxonomy. It should break them into “eduglu” and “prototype” so I could find relevant items without forcing all users to use identical category strings. Which would fall apart if someone wants to add an additional category, like “eduglu prototype drupal” since it wouldn’t match either of the other two literal string taxonomies.
It’s possible that Drupal (and the Aggregator2 module) could be tweaked to do what EduGlu would require. Might be easier to do the work there, rather than starting from scratch…
Update: Interesting - Aggregator2 is properly understanding the individual categories on items published by my blog, but not from other tools like Flickr or del.icio.us - maybe Aggregator2 is right, and the others are wrong in how they’re storing/sharing categories?
Thanks for this post D’Arcy.. very helpful! I have been dying for this type of functionality in Drupal for awhile now and it’s great to be able to read what other education tech geeks are doing with it.
Thanks!
Jo McFetridge
27 Feb 06 at 9:52 am
geek being a compliment, that is
Jo McFetridge
27 Feb 06 at 9:54 am
taken as a complement… It’s in the subtitle of the blog, after all…
D'Arcy
27 Feb 06 at 11:10 am
Agg2 reads categories correctly for feeds that properly use RSS 2.0 categories. Del.icio.us in particular fscks this up pretty badly. You could add some code to Agg2 that dedicated the upstream URL (e.g. http://www.flickr.com/xxx, http://del.icio.us/xxx) and made corrections.
What you want next of course is solutions for categorizing/viewing/rating a ton of content…it comes in as native Drupal content, so you use traditional “node” techniques to work with them. One of the voting/rating APIs would be interesting. I would also NOT promote all content, but rather let people rate/promote to show the stuff that is valuable for everyone. Yes, this is like building your own digg — see the vote_up_down module for some ideas: http://cvs.drupal.org/viewcvs/drupal/contributions/modules/vote_up_down/
And no, it’s not quite EduGlu, but it’s pretty damn good for out of the box functionality. Some UI work and work on filtering mechanisms and it might be interesting.
Boris Mann
27 Feb 06 at 1:27 pm
Great to hear that you think Drupal could be tweaked for your EduGlu initiative. I would suggest doing development in 4.7. There has been many performance enhancements under the hood since 4.6. As far as enterprise level performance have you checked out the notes from the Enterprise-wide session at the recent DrupalCon?
http://drupal.org/node/49717 Many of these issues will undoubtedly be dealt with by 4.8
travischristopher
27 Feb 06 at 1:35 pm
Boris - I was thinking the same thing. Playing with the Drupal/Agg2 mockup, I said “Damn! That’s exactly how that should work!” several times. It’s close - just needs some more flexible views on the data. Cool!
Travis: I used 4.6 only because that’s what the Agg2 plugn was rated for. I’m using 4.7.beta4 on a few projects and am comparitively loving the hell out of it. The caching etc. will really help as well.
D'Arcy
27 Feb 06 at 1:38 pm
Have you tried out the views module yet? Great stuff but not really 4.6 friendly… I wonder how close Agg2 is to being ported to 4.7
travischristopher
27 Feb 06 at 1:48 pm
Haven’t tried the Views module (yet) - the project description sounds juicy. I’ll try installing Agg2 on 4.7B4 with Views to see what blows up. Thanks for the tip!
D'Arcy
27 Feb 06 at 3:16 pm
Agg2 is not ported at all AFAIK.
I have a long wish list of items of “doing Agg2 right”. We’ll see if some funding crawls out of the woodwork somewhere.
Oh, and I forgot about the personal aggregator/syndicator/blog reader that Prometheus6 is working on (just on his own site, not on Drupal.org). See http://code.prometheus6.org/node/67
Boris Mann
27 Feb 06 at 5:51 pm
Feel free to keep an eye on my work…it’s GPL, so you can do whatever with it. Right now, it’s running at http://www.intrapolitics.org/syndicator. The current module is more focused on feed management than presentation, so I’m testing it with 800+ feeds.
The biggest obstacle I find to repurposing feeds is inconsistent XML. Like Blogger send its Atom feed items out as application/xhtml+xml if there’s an img tag in it and text/html otherwise…in the same feed. Then there’s just bad xml, bad html, posts composed in a iso8859 character set (Windows) and stored in utf8 tables…
The personal feed subscriptions and tagging are next, but I should be working on a parser.
Prometheus 6
27 Feb 06 at 8:09 pm
Boris and Prometheus - thanks for the link to Syndicator. I’ll check it out.
As for funding for Agg2-done-right, I’ll check with my account manager [rummages through pockets] - I have about 5mg of pocket lint to contribute at the moment. Might be able to justify some time, though…
D'Arcy
28 Feb 06 at 7:30 am
Slightly OT, but I have to object to Boris - del.icio.us does not get categories wrong. It uses RSS 1.0 and uses dc:subject in a perfectly acceptable manner, and which I’ve also seen used in RSS 0.91, RSS 2.0 and Atom feeds.
Phil Wilson
1 Mar 06 at 1:32 pm
Just to let you guys know, I updated Autotaxonomy contrib module for Aggregator2 so that the del.icio.us style categories are supported correctly. Enjoy!
Also, as Aggregator2 never quite seems to be stable 100% of the time for me, I wrote a small module which wraps existing aggregator.module feed items in to Drupal nodes. You can download it from http://drupal.org/node/59712
Budda
22 Apr 06 at 8:32 pm
Thanks, Budda. I’ll check it out ASAP!
D'Arcy
23 Apr 06 at 9:21 pm