Jeroens blog

Yesterday I quickly wrote up a simple (but awesome) MediaWiki extension that allows you to make use of the Spark library in your wiki.

Spark as described on the Spark website:

The web is not only growing in sheer size, but it also grows in how much it is interconnected. Where once the Web was a set of more or less separated sites, today sites are more and more being connected. More and more data is being offered on the Web in a way that can be further processed, and more and more sites and applications are using external data. More and more mashups are created, where data from different sources is integrated and displayed with novel visualisations.

Spark is a library that enables HTML authors to create mashups more easily than ever before. Using standard Web technologies like SPARQL, RDF, HTML5, and JavaScript, Spark can query external knowledge sources (so called triple stores or SPARQL endpoints), and then visualise the results.

With Spark, website developers can create visually appealing mashups without having to write a single line of JavaScript, but merely using some markup elements describing the source of the data that is to be shown, a query to select the appropriate data, and selecting one from an expandable set of visualisations and their parameters.

Spark is developed by Denny Vrandečić and Andreas Harth.

This MediaWiki extension, unsurprisingly titled Spark, adds a <spark> tag to MediaWiki which is equivalent to <div class=”spark”> as described in the spark library documentation. All parameters (except the class=”spark” one) can just be copied over between spark divs in web pages, and the <spark> tag in MediaWiki. It is currently at version 0.1, which is a beta release. It includes a still experimental version of the Spark library, so you should probably not use this extension on production websites just yet. The Spark people are looking for developers to help out, so if you want to play around with SPARQL a bit, like I basically did with this extension, be sure to poke them

The extension required MediaWiki 1.17 or above (as it makes use of the new Resource Loader) and PHP 5.2 or later.

Download

• The current Spark (MediaWiki extension) release (zip)
• svn co http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/Spark/REL_0_1 (tag for 0.1, including Spark lib trunk)
• svn co http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/Spark (trunk, including Spark lib trunk)
• svn co http://rdf-spark.googlecode.com/svn/trunk/src/ (Spark lib trunk)

Further possibilities

Right now you can embed mashups with SPARQL queries that get their data from some SPARQL endpoint. This opens up a whole bunch of possibilities, but is a bit silly when you are running your own Semantic MediaWiki instance and want to visualize structured data stored by it using Spark. A possible addition to the Spark MediaWiki extension therefore is having support for Spark as a so called SMW result format. For this translation from the SMW ask query language to SPARQL is needed, which is some work. I might implement this at some future point, but have several other things I want to poke at, so it won’t be soonish

After doing a pile of research on how to best create an extension management platform for MediaWiki as my Google Summer of Code project, I realized that a lot of the work that I wanted to do was already done in some form or another, and decided to somewhat augment my goals. Since I can build upon Configure, and the Deployment Framework of Ontoprise, it should now be possible to also take care of the MediaWiki and extension configuration that I put as optional in my original proposal. To have some transparency here, and not to cause misconceptions, I created an awesome new roadmap.

I did an attempt to get some feedback by posting my roadmap on wikitech-l, but apparently everyone is either happy with it, or more likely, Parkinson’s Law of Triviality is in play here. So feedback and suggestions are definitely welcome, you can post them on the discussion page.

As Configure is doing an awesome job already without making significant changes to any existing code, I decided to start off with the Deployment Framework. I’m currently in the process of figuring out how it works exactly, so I’m able to extend it’s features, and build a GUI in the form of several MediaWiki special pages on top of it. This will have some immediate pay-off as Ontoprise will be able to use this improved version directly.

Both my presentations for Wikimania 2010 have been accepted, of which one is about my project, which will give me a change to explain to people what I’m doing and why it’s so awesome

Like last year, I’ve been accepted for GSoC 2010 – yay!

The Google Summer of Code (GSoC) is an annual program, first held from May to August 2005, in which Google awards stipends to hundreds of students who successfully complete a requested free software / open-source coding project during the summer.

My project is creating an awesome extension management platform for MediaWiki, facilitating the installation, updating, removal and configuration of extensions. I’ll get mentored by Brion Vibber, which is probably the best known MediaWiki developer there is. The underneath paragraphs are out-takes of my actual proposal.

Project summary

The goal of this project is to create an administration panel from where wiki administrators can update, install and remove extensions. A second goal would be to allow management of the installed extensions.

A panel where wiki administrators can install, update and remove extensions would have huge benefits. First of all, people would not have to manually download an extension and put an includes in LocalSettings, neither would they need to worry about compatibility and dependencies. Hitting an update button also takes considerably less time then doing the whole download routine again, and will cause people to run more up to date extensions. Another important advantage is that people will get extensions recommended, and can easily browse them. This way people will find extensions that do something they wanted but did not know about, and in general have extensions that better suit their needs. A third advantage is that extension developers won’t need to do extreme efforts to let people know there is a new version (and probably still only reach part of the relevant public). This is inspired on the way Word-press does things.

The second goal of this project is to add setting management for individual extensions. Currently extension settings are managed via LocalSettings. The aim here is to completely remove the need of editing any file directly by storing the configuration the the MediaWiki database, and creating a GUI to modify these settings. This would involve creating API modules so extensions can add and update their own settings. Work on this will only be started after the first goal is completed, and is seen as an “if time permits to-do”.

A third, also optional, goal would be to create a management interface for the wiki’s configuration itself. This is very similar to the second goal, and should be kept in mind while creating the management for extension settings. I do not expect to complete this to-do during GSoC, but want to provide the foundations for it, so this can be completed after the project itself is finished.

Deliverables

Note: the beneath list is a guideline only.

Things the administration panel should be capable of:

Required deliverables

• Automatic checking for extension updates (and updates to MW itself)

• Download and install option for extension updates

• Download and install option for extensions that have not yet been installed

• Removal of installed extensions

• Dependency and compatibility checking (both for installation, removal and updating)

• Extension browsing
  • Recommendation of similar extensions
  • Showing a list of most used and most recommended extensions
  • Finding extensions by category or keywords

• Enabling and disabling of installed extensions (so without removing them)

If time permits

• Management of settings specific to each extension.

• Management of MediaWiki configuration.

Project schedule

I’m for a loose schedule, since I believe this is the most efficient. I have no doubt that the to-do list will change a lot during the project, items will be changed, moved, multiple new ones will be added, and some might be removed. A fixes schedule would take away flexibility and stand in the way of efficiency. One of the main reasons to have a schedule is to ensure the student does not take the project to lightly, end ends up making insufficient progress. I like to believe I have clearly demonstrated that I will put considerable effort in such projects, even without any schedule, during last yeas GSoC, and with all the commits I’ve made since then.

This list contains some loose planning without any dates:

• Discuss the best way to structure the platform with mentor, and other relevant people.

• Investigate how similar functionality works with other software, and which aspects of these implementations can be used.

• Get the requited knowledge of MW to be able to create the platform in an efficient and modular fashion.

• Create the panel with the most basic features, then release and document it.

• Add the other features, possibly spread over several releases.

• (Start on the extension setting management functionality.)

• (Start on the MediaWiki configuration management functionality.)

• (Add other awesome things to the platform.)

• Take over the worlds and make everyone use this platform.

My current planning

I have to finish up the work I’m currently doing for the Wikimedia Foundation before I can fully start on this, and also like to release Maps and Semantic Maps 0.6 before then. This will probably be in two to tree weeks from now.

Also see the Wikimedia tech blog post which links to the other accepted projects.

As of today I’m working for the Wikimedia Foundation as developer. I have a 3 month contract, after which I’ll participate in GSoC 2010, and do a project for … well… Wikimedia Foundation (I’ll post more about GSoC 2010 later on, I have a really awesome project planned :d) Although the payment is pretty regular for this type of dev, I think it’s completely awesome to work for a mission based company with as objective to make knowledge freely available to everyone, create completely open source software and work with really enthusiastic and talented people.

The project I’m going to start with is the Wikimedia Storyboard, which will take the form of an extension. The idea is to develop a fundraising landing page on http://wikimediafoundation.org/ with moderated but up-to-date stories from users and donors explaining the role Wikimedia has played in their lives. The motivation of building such a landing page is that donor and supporter stories can help to convince and persuade people of the impact an organization is having. It’s essentially an extended, moderated version of the real-time contribution history.

I’m hoping to get this extension finished in a bunch less then 3 months, so I can also do other stuff before GSoC. I’ll only be working part time on this though (~3hours/day), and will also be doing work on Semantic MediaWiki for KIT as well as other projects, such as Maps and Semantic Maps

I did a thorough overhaul of my wiki. I cleaned out the mess, put in some nice demo data, installed the latest version of Semantic Bundle, upgraded to MediaWiki 1.16alpha, put in a new logo and switched to the new (and awesome) vector skin. I also finally created decent templates and semantic data structures for my events and Semantic Maps demo’s.