It’s been a while since I last wrote a blog post, and this is in part due to the last extension I wrote sadly enough not being available under an open license due to client demands :/ This blog post is about what I’m currently working on, new infrastructure for a project that I hope you’ll agree upon is totally awesome.
Open knowledge and open learning tools go hand in hand, thus Wikipedia can be an amazing resource for education further then simply looking up things. The Wikipedia Education Program strives to involve university students with the Wikipedia community, by creating and editing articles as part of their coursework. This is a real win-win for everyone involved. The students get to do something which is actually useful, as their work won’t disappear into some dusty archive of their university, and they learn working collaboratively with other people in the Wikipedia community. The gains for Wikipedia, and it’s community, are probably pretty obvious; more contributors, more content, more people to help increase the quality of the encyclopaedia as a whole.
If you are interested in bringing this program to your university, definitely check out the Wikipedia Education Program (WEP). Also, you can find more comprehensive posts about the program itself, as well as many success stories, on the Wikimedia blog.
The WEP started several years ago with a handful of universities participating. Such participation and enrolment was kept track of manually on wiki pages. Obviously, this does not scale well, and now that the program has two or three orders of magnitude more students, new tools are urgently needed. The main goal is having a way for students to enrol themselves, and then be tracked by their mentors and people of the community wanting to provide assistance where needed. Further a lot of nice things can be done, such as keeping track of contributions, drop-out rates, ect, so that problems can be spotted, and measures can be taken to avoid these in the future. You can find an initial draft of the requirements here.
I’m implementing this as a new MediaWiki extension, which will when completed, be placed onto Wikipedia. The extension is, probably unsurprisingly, called Education Program. It’s currently in early alpha stage, so not much to see there yet. However, a draft of it’s functionality can be found here. Management wise it’ll be somewhat similar to Contest, but more extensive (since there is more stuff to manage), and hopefully improved along multiple dimensions. I still need to put more thoughts into the exact flow for students though, and discuss this with the other people involved.
This is a screenshot of one of the many special pages making up the management interface. It lists all courses, allows you to browse through them (paged) and filter on criteria. Program administrators and mentors also get to see a control to add new courses, which then takes them to a new page with a form. Very similar pages exist for other types of objects, such as Institutions, Terms, Mentors and Students (although the later two are somewhat different, since they cannot be modified by people other then the students or mentors themselves).
I will likely be working on this for another 2 months, after which I will start work on an even more awesome project (don’t get me wrong, the WEP is definitely awesome), on which I’ll post more later on.
As it’s been 2 months since my last blog post, I figured it was time for another one. Quite a few things I could write about (SMWCon, my new awesome laptop, Stanfords AI and ML classes, me moving to Berlin, …), but I decided to give some introduction to my most recent MediaWiki extension: Contest.
Contest extension that allows users to participate in admin defined contest challenges. Via a judging interface, judges can discuss and vote on submissions. I created it for the Wikimedia October coding challenge, so it got a nice amount of review, uncovering some minor misconceptions I had about some core MW code, and it got deployed on MediaWiki.org. The coding challenge is quite awesome, but I won’t discuss it any further here, so check out the linked blog post if you’re curious/interested.
Feature overview:
Requirements
Some screenshots
Some background
When starting with this extension, it was clear pretty quickly that it could really use the awesome DBObject class I’ve been incrementally creating over my last few extensions and mostly finished in Survey. This class is a wrapper for objects of a certain type, which is equivalent to a row in some db table. Even though it’s in essence very simple – it just has a field that is an associative array with field => value – it’s also very powerful and flexible. When I started with this, I had no idea it would turn out to be so neat. The bad news was that I could not use PHP 5.3 or later for Contest, while the DBObject class uses late static binding, which was introduces in PHP 5.3. I came up with a simple hack: all static methods in the base class have been made-non static, but are marked as “should be static”. Then, every deriving class has a public static function s(), which returns a (cached) instance of the class. So then for every “static” method you need to call, instead of ClassName::methodName(), you do ClassName::s()->methodName(). If you know this, and do not misuse the non-static-but-should-be-static methods in the base class, it retains all it’s niceness, at the cost of something that’s pretty much a tiny bit of syntactic sugar. And it’s quite obvious how easy it will be to replace this with actual LSB usage once this becomes possible
Download
What’s next?
There are various small additions that could be made, but one things really stands out: contest configuration with version history. Right now, you can create contests and challenges, modify them and delete them. But once you make a change, the previous version is lost. You cannot revert. You cannot compare. You can’t even see who made the change or when. Implementing such a thing is not trivial, especially if you want to have a generic system that can be used by any extension that wants to store data and have version history for it. And if you think about it, quite a few extensions could use this. Let’s have a look at my extensions, latest first, and see if they can use it:
That’s 6 out of 10 just for the extensions I wrote. What it comes down to is that pretty much any extension that has some sort of settings interface that is not user-specific, could use this. And maybe even the user specific ones, which would obviously include the user preferences in MediaWiki core. So why not store this data in wiki pages, such as done by Maps for layers? You could even store it as JSON or serialized PHP objects if you need more complexity… The things with this is that it only works for simple use cases (such as the layers in Maps), and even then is limited. You cannot query over the data as you do not have it in relational form. And you cannot have fine grained access and write rights control over the data, which in a lot of cases is quite important. So a generic solution here would be an awesome addition to MediaWiki if you ask me.
More info on Contest can be found on it’s documentation page.
As this particular project is coming to and end, I figured I’d do a quick blog post on it.
Wiki Loves Monuments (WLM) is a photo contest for European monuments, organized by Wikimedia this September. Last year some JavaScrip hacks on the regular Wikimedia Commons (the media repository for Wikipedia and other Wikimedia Foundation projects) upload interface where used for this contest. This year the new and completely awesome Upload Wizard (UW) will be used, with configuration optimized for WLM. I created a campaign-based configuration system from the UW and also added a bunch of new settings.
2 new special pages where added. One listing all campaigns, their status, and edit and delete links. This is at Special:UploadCampaigns.
The other special page handles the edit action and displays a list of all available settings that can be modified for the campaign. This is at Special:UploadCampaign/name.
A campaign can be applied to the UW by adding the “campaign” url parameter with as value the campaign name, ie ?campaign=wlm-be.
One fun thing about the architecture of the campaign system is that the setting support is very generic. I created a new settings class that pulls in the default settings, overrides these with the wikis config (ie PHP vars in LocalSettings.php), passed URL arguments and finally the upload campaign settings if a campaign is specified. I like this kind of setup, as it’s a lot nicer then dealing with over 9000 global variables, and in the meanwhile already applied some variation of it in Semantic Signup and in my new Surveys extension. And I wrote up a more general and powerful version of such setting handling in the Maps extension. Unfortunately this code uses late static bindings and thus requires PHP 5.3, making it not usable in actual code for quite a while
Another neat thing is that the upload campaign class only specifies a lift of settings that should be configurable for upload campaigns, together with what kind of HTML form input they should be displayed. That info is then merged with the settings obtained from the settings class and put into a FormSpecialPage, which uses HTMLForm to display anything without any further hassle
Not used the Upload Wizard before and curious how it works? Go upload some nice stuff to commons then
Today I released the first version of a brand new MediaWiki extension titled Semantic Watchlist. It extends Semantic MediaWiki by adding the capability to watch/follow sets of properties for groups of pages (that can be specified with categories and namespaces). You can view changes to these properties via Special:SemanticWatchlist, which works similar to the regular MediaWiki watchlist. And you can even request to be notified via email when a change is made!
Feature overview:
Let’s have a look at the different parts of the interface:
The watchlist
Each user can view changes to properties they watch on Special:SemanticWatchlist, which looks and works similar to the regular watchlist. Items that have not been viewed yet on the watchlist will be indicated as ‘NEW’.
Watchlist preferences
Each user can manage which watchlist groups they follow via their user preferences. They can also choose if they want to receive email notifications or not. These preferences can be found on Special:Preferences, which is linked at the right top of the page in most skins for logged in users.
Watchlist groups
The watchlist groups can be managed via the Special:WatchlistConditions page by people that have the ‘semanticwatchgroups’ right, by default only administrators. Each group has a name, which allows users to easily recognize the groups in their preferences, and a single category, namespace or concept it covers. Only changes to properties on pages in this category, namespace or concept will be shown to users watching this group. Each group also has a list of properties, which further restricts what property changes should be shown to the user.
Email notifications
When you choose to receive email notifications for changes to semantic properties covered by watchlist groups you watch, they will appear both in your watchlist and your inbox. The below screenshot is a simple example of a notification email.
Extending Semantic Watchlist
Semantic Watchlist is in part a workflow extension, which makes it important for other extensions and tools to interact with it. This is possible via the hooks and API modules Semantic Watchlist provides.
API modules:
Hooks:
Further plans
I think this extension opens up a lot of new possibilities for the SMW platform in the area of workflow. Due to it’s API modules and hooks, it’s very easy for other extensions to build on top of the watchlist functionality, so I’m curious as to what will happen there.
This first release comes with core functionality, but at places lacks polish. And since it’s the first release of a somewhat complex extension, I do expect issues to show up in use cases not tested for yet. Both these factors make it likely that a second release will be made relatively soonish.
Since I think this extension is such a great addition to the already awesome SMW environment, I’m going to give a talk about it at the upcoming SMWCon. That’s bound to yield some interesting feedback
Requirements
Download
Some background
I developed the Semantic Watchlist extension as WikiWorks consultant for the IEEE, with some help from Yaron Koren.
Also check out the wiki version of this post.
Version 0.7 of both the Maps and Semantic Maps extensions is now available for download. This release is made after 3 beta’s and a release candidate, so should be stable.
The most notable new features in this release are tag extension support for all the Maps parser functions, more consistent error handling via Validator 0.4 and compatibility with the upcoming MediaWiki 1.17. Most changes are internal improvements, but there are also several bug fixes and minor new features.
Whats next?
There only big new feature on the roadmap for 0.7.x is the infamous images-as-layers one. This feature was supported by the now long obsolete Semantic Layers extension (example), and has been requested by dozens of people since the first releases of Maps. Next to fixing the bugs that show up, the focus will mainly be on adding new functionality. Feel free to propose new features, or help out creating them.
Just under a month after the 0.6.5 release of both mapping extensions, the next minor update, 0.6.6, is available for download. No spectacular new features, but several important bugfixes. Several issues with coordinate parsing have been fixed, you can now using geocoding when behind a proxy, and wikitext should finally(!) behave correctly in marker pop-ups. Some internal changes have also been made, mainly rounding off the many changes I made in the 0.6.x branch. I expect this release to be the most stable one to date, and have therefore changed the extensions status from ‘beta’ to ‘stable’ on the documentation pages.
A lot of improvements have been made to the documentation as well. Both the Maps examples and Semantic Maps examples are now comprehensive and complete. There now are finally examples of using query templates, of distance queries and of some nice compound queries. Some more work is needed though, a lot of which is explaining basic functionality and fixing minor issues all over the place. I’ll be taking care of the most important things, but I’d be great if people using the extensions could help me out improving the documentation
This release is probably the last one before 0.7, in which I expect to be focusing on new functionality. I’m looking for people that want to fund the development of new features, so please contact me if you are such a person
Downloads:
As Google Summer of Code (GSoC) 2010 has ended, I’m writing this blog post to outline what I’ve done during the coding period and what the results are. Thanks go to the Wikimedia Foundation and Google for providing the opportunity to do this project, Brion Vibber, who mentored me, and to all other people who helped me out, especially Yaron Koren who I bugged the most
What I did during GSoC
My initial proposal was to create an awesome extension management platform for MediaWiki that would allow for functionality similar to what you have in the WordPress admin panel. After doing some research I realized this would require significant effort in two areas: configuration and deployment. After looking at some already existing tools such as the Configure extension and the Deployment Framework of Ontoprise, I decided to completely drop the configuration part and concentrate on the deployment work.
I started with porting the filesystem abstraction classes from WordPress, which are needed for doing any upgrade or installation operations that include changes to the codebase. (The current MediaWiki installer can do upgrades, but only to the database.) I created a new extension called Deployment, where I put in this code, and which was intended as a place to experiment with all the MediaWiki-installation side deployment stuff. As You obviously want this functionality to be part of MediaWiki itself, I wrote it with the idea of moving over the code to MediaWiki core once it was finished. It turned out that doing filesystem upgrades securely is not an easy task though, so after finishing the port, I quitted work on this as I decided to do other functionality first.
I then poked somewhat at the new MediaWiki installer, which is a complete rewrite of the current installer with a lot of new cool stuff and a totally more awesome interface. I made some minor imrpovements there, and split the Installer class, which held core installer functionality, into a more generic Installer class and a CoreInstaller. This allows for creating an ExtensionInstaller that uses the same base code, such as database, filesystem and LocalsSttings manipulation.
After this I started thinking about how to best structure a package repository for MediaWiki and extensions to get updates and new extensions from. I had a look at PEAR and CPAN, as well as WordPress, although I don’t learn a lot about the later. Apparently their repository code is not freely available
After discussion with Brion I decided to just create the repository from scratch, and started working on another extension, titled Distribution, for this purpose. I merged it together with a rewritten version of the MWReleases extension written by Chad, which already had core update detection functionality.
After the Distribution API’s where working decently I started work on the Special pages in Distribution that would serve as the equivalent of the WordPress admin panel. As I put of the configuration work, and also the file-system manipulation for the initial version, this came down to simply listing currently installed software, update detection and browsing through extensions available in the repository.
On top of my GSoC project itself, I did quite some other MediaWiki work in “my free time”. I released 5 new versions of Maps and Semantic Maps, starting with 0.6 and ending at 0.6.5. As I finally got core commit access, I also poked at some other things, such as Special:Version, which now will automatically put all extensions of unknown type in the “other” category, and will display this category as the last one. Plus misc minor improvements to a verity of extensions. This all amounts into a little over 550 commits to the MediaWiki SVN repository during the GSoC coding period.
State of the code
The Distribution extension has the infrastructure for storing and providing extension and core data via the MediaWiki API basically ready for use. It adds 4 database tables to MediaWiki:
The API modules added are:
To populate the database with existing extension info I wrote a maintenance script “getSvnMetadata”, which goes through a local checkout of the MediaWiki extensions directory and get’s the names from the extensions. I haven’t found a good way yet to also get other extension data though.
The Deployment extension contains an abstraction layer for repository interaction and several interfaces that use this. The abstraction layer allows for supporting different kinds of repositories. The only implementation it currently has is for interaction with repositories provided by the Distribution extension. It’s also a convenient point to implement caching, as you probably don’t want to send the requests for available updates every time you view a page on the admin panel, and allows for changes to the format the repository uses without any effects in other parts of Deployment. The interfaces that are finished to some extend are:
The extension allows you to configure several aspects of the repository interaction:
What’s next?
Although some very basic functionality is working, quite some work still needs to be done to get this to the WordPress-awesomeness level. Let’s first have a look at Distribution and then Deployment:
The most basic issue with Distribution currently is that there is no script yet that allows collecting all current extension data, which is needed for it to be of any use. I’m not sure how gathering current data can be properly automated, which is the main reason the script doesn’t exist yet. Any suggestions here are very welcome! After the initial version it should be possible for extension authors to edit their extensions data, and create new releases. For this we’ll need some new special pages. The data itself can then be used to populate the extension pages on mediawiki.org, and some new magic words such as “current MediaWiki version”, can automate a bunch of stuff. After these things new features can be added, such as the management of packages, and more detailed extension information, including things such as dependencies and compatibility info.
Deployment mainly needs interface work, and will need additions to support any new information provided by the Distirbution repository. A cool feature that could be added is supplying the repository with installation information (obviously optionally), which would allow the developers to get an idea of which versions of MediaWiki core and extensions people are using. After the whole MediaWiki deployment model has been revised and is up and running, it’s configuration can similarly be reinvented. The interfaces added by Deployment can then be adapted to allow contain extension configuration.
GSoC 2011?
This was my last GSoC as a student, as I no longer qualify, since I quitted my official studies. If I’m still doing MediaWiki development next year, which I guess is pretty likely, there is a lot of change I’ll be signing up as a mentor though
If you are interested in being a student in 2011, you can already put your name on the 2011 GSoC page
Some useful links
Just a few quick screenshots of Special:Extensions, on which I’ve been working today. The first screenshot shows Special:Extensions page displaying a list of all the extensions I have installed on my local wiki:
As you can see, you can now filter on extension type with the control right below the “Installed extensions” title. Here I have filtered on the SEMANTIC extensions:
An interesting change I made is that you can now access this page without having the siteadmin permission. Doing this will get the above, but without the add new button and administration controls (currently only “Deactivate” which is there only for show so far). This way this page will be a nice addition to Special:Version.
Tomorrow is the last coding day in Google Summer of Code 2010, during which I’m planning to focus on the update detection functionality, or rather creating the interface for it, as the plumbing for it is all but done. I also want to move several classes from Deployment over to MediaWiki core, as they make more sense to have there, and would allow for some nice improvements.
Since my last blog post about my GSoC project, which aims to bring more awesome deployment capabilities to MediaWiki, I’ve been putting my time into both the Distribution and Deployment extensions. I was pleased to find a bunch of stuff was easier to do then I had imagined, and now I finally have some functionality you can actually see working – yay
It’s not a lot, as this is just very rudimentary and even uses demo data at places. Still it’s very nice to be able to post some screenshots after months of doing research and poking at code. I also got a new crappy diagram (although I think I’m succeeding in getting them less crappy each iteration) that shows the architecture of the initial versions I’m working towards.
The following screenshots show an import script running. This script is part of Distribution and is meant to get data from a checked out copy of the extensions directory and store it into the database tables provided by Distribution. I haven’t found a good way to actually get the extension data other then their path names (for example ‘SemanticMediaWiki’), so have a function that’s just returning some demo data.
Via Deployment you can search for extensions by keyword, author or tag. This is done by making a request to the API provided by Distribution which serves the data collected by the script. This screenshot shows the interface I have created so far:
And after clicking the button, you get:
As I’m doing with all special pages in Deployment, Special:Install’s layout and functionality is based on what WordPress has in it’s admin panel.
I also put some work in Special:Extensions already, although it’s basically just Special:Version with just the extensions for now.
I’m not wring a real comprehensive overview of what features there are already, which are planned, and how I plan to create them for now, as there is little time left in GSoC (only 3 days!), and I want to get some more work finished before then
With only 2 days till the suggested Google Summer of Code ‘pencils down’ date, and a week longer until the firm one, I’m using my remaining time to get some basic functionality working for my GSoC project. I’ve started creating a new extension called Distribution that will provide an API module which can be used to query extension meta-data. Later on it should also be able to do this for core, and provide archived packages that can be downloaded to your MediaWiki server, and directly installed. Initially I’ll probably simply be pointing to the ExtensionDistributor extension on MediaWiki.org, which can create archives for extensions on request. The Deployment extension which I started on earlier on will use data obtained via the API Distribution provides to display available extensions on your local installation. Right now I’m attempting to get the ExtensionDistributor working locally, so I can develop the Distribution extension alongside it, and confirm everything works. This will take some time as it uses Linux/UNIX specific commands, forcing me to use my Kubuntu install, on which my dev environment is only partially set up at the moment.

Categories
Tag Cloud
Blog RSS
Comments RSS
Last 50 Posts
Back
Void « Default
Life
Earth
Wind
Water
Fire
Light 