It’s been a while since I last wrote a blog post, and this is in part due to the last extension I wrote sadly enough not being available under an open license due to client demands :/ This blog post is about what I’m currently working on, new infrastructure for a project that I hope you’ll agree upon is totally awesome.
Open knowledge and open learning tools go hand in hand, thus Wikipedia can be an amazing resource for education further then simply looking up things. The Wikipedia Education Program strives to involve university students with the Wikipedia community, by creating and editing articles as part of their coursework. This is a real win-win for everyone involved. The students get to do something which is actually useful, as their work won’t disappear into some dusty archive of their university, and they learn working collaboratively with other people in the Wikipedia community. The gains for Wikipedia, and it’s community, are probably pretty obvious; more contributors, more content, more people to help increase the quality of the encyclopaedia as a whole.
If you are interested in bringing this program to your university, definitely check out the Wikipedia Education Program (WEP). Also, you can find more comprehensive posts about the program itself, as well as many success stories, on the Wikimedia blog.
The WEP started several years ago with a handful of universities participating. Such participation and enrolment was kept track of manually on wiki pages. Obviously, this does not scale well, and now that the program has two or three orders of magnitude more students, new tools are urgently needed. The main goal is having a way for students to enrol themselves, and then be tracked by their mentors and people of the community wanting to provide assistance where needed. Further a lot of nice things can be done, such as keeping track of contributions, drop-out rates, ect, so that problems can be spotted, and measures can be taken to avoid these in the future. You can find an initial draft of the requirements here.
I’m implementing this as a new MediaWiki extension, which will when completed, be placed onto Wikipedia. The extension is, probably unsurprisingly, called Education Program. It’s currently in early alpha stage, so not much to see there yet. However, a draft of it’s functionality can be found here. Management wise it’ll be somewhat similar to Contest, but more extensive (since there is more stuff to manage), and hopefully improved along multiple dimensions. I still need to put more thoughts into the exact flow for students though, and discuss this with the other people involved.
This is a screenshot of one of the many special pages making up the management interface. It lists all courses, allows you to browse through them (paged) and filter on criteria. Program administrators and mentors also get to see a control to add new courses, which then takes them to a new page with a form. Very similar pages exist for other types of objects, such as Institutions, Terms, Mentors and Students (although the later two are somewhat different, since they cannot be modified by people other then the students or mentors themselves).
I will likely be working on this for another 2 months, after which I will start work on an even more awesome project (don’t get me wrong, the WEP is definitely awesome), on which I’ll post more later on.
As it’s been 2 months since my last blog post, I figured it was time for another one. Quite a few things I could write about (SMWCon, my new awesome laptop, Stanfords AI and ML classes, me moving to Berlin, …), but I decided to give some introduction to my most recent MediaWiki extension: Contest.
Contest extension that allows users to participate in admin defined contest challenges. Via a judging interface, judges can discuss and vote on submissions. I created it for the Wikimedia October coding challenge, so it got a nice amount of review, uncovering some minor misconceptions I had about some core MW code, and it got deployed on MediaWiki.org. The coding challenge is quite awesome, but I won’t discuss it any further here, so check out the linked blog post if you’re curious/interested.
Feature overview:
- Admin interface for managing contests and their challenges.
- Landing and signup pages for each contest.
- Personal contest list and submission interface for each user.
- Summary pages per contest listing contestants, which can be filtered and sorted.
- Judging interface that allows for rating and commenting on each participant.
- All contests, challenges, contestants, comments and votes can be queried and exported via the API.
- Signup and reminder emails.
Requirements
- MediaWiki 1.18 or above – yeah! my first awesome MediaWiki 1.18 or later extension
- PHP 5.2 or above
- MySQL
Some screenshots
Contest welcome interface
Contest submission and management interface for a single user
Contest administration and judging interface listing all contests
Contest administration and judging interface for a single contest
Contest editing interface
Judging interface for a single submission
Some background
When starting with this extension, it was clear pretty quickly that it could really use the awesome DBObject class I’ve been incrementally creating over my last few extensions and mostly finished in Survey. This class is a wrapper for objects of a certain type, which is equivalent to a row in some db table. Even though it’s in essence very simple – it just has a field that is an associative array with field => value – it’s also very powerful and flexible. When I started with this, I had no idea it would turn out to be so neat. The bad news was that I could not use PHP 5.3 or later for Contest, while the DBObject class uses late static binding, which was introduces in PHP 5.3. I came up with a simple hack: all static methods in the base class have been made-non static, but are marked as “should be static”. Then, every deriving class has a public static function s(), which returns a (cached) instance of the class. So then for every “static” method you need to call, instead of ClassName::methodName(), you do ClassName::s()->methodName(). If you know this, and do not misuse the non-static-but-should-be-static methods in the base class, it retains all it’s niceness, at the cost of something that’s pretty much a tiny bit of syntactic sugar. And it’s quite obvious how easy it will be to replace this with actual LSB usage once this becomes possible
Download
What’s next?
There are various small additions that could be made, but one things really stands out: contest configuration with version history. Right now, you can create contests and challenges, modify them and delete them. But once you make a change, the previous version is lost. You cannot revert. You cannot compare. You can’t even see who made the change or when. Implementing such a thing is not trivial, especially if you want to have a generic system that can be used by any extension that wants to store data and have version history for it. And if you think about it, quite a few extensions could use this. Let’s have a look at my extensions, latest first, and see if they can use it:
- Contest – well yeah, obviously
- Survey – yes, for the surveys, their configuration, and their questions
- Semantic Watchlist – yeah, for the watchlist groups
- IncludeWP – no
- SubPageList – no
- Live Translate – yeah, for the translation memory admin
- Push – no, although it could, if it had an admin interface for the push targets
- Validator – no
- Maps and SM – yeah, for the map layer definitions currently done as key=value pairs, one per line, in the Layer namespace
That’s 6 out of 10 just for the extensions I wrote. What it comes down to is that pretty much any extension that has some sort of settings interface that is not user-specific, could use this. And maybe even the user specific ones, which would obviously include the user preferences in MediaWiki core. So why not store this data in wiki pages, such as done by Maps for layers? You could even store it as JSON or serialized PHP objects if you need more complexity… The things with this is that it only works for simple use cases (such as the layers in Maps), and even then is limited. You cannot query over the data as you do not have it in relational form. And you cannot have fine grained access and write rights control over the data, which in a lot of cases is quite important. So a generic solution here would be an awesome addition to MediaWiki if you ask me.
More info on Contest can be found on it’s documentation page.
Categories: Programming, Software
Over the past 3 weeks I’ve been working on a new MediaWiki extension that allows creation of on-wiki surveys by admins. It comes with a whole bunch of neat features, and is the most awesome (code wise) extensions I’ve created so far.It’s aptly titled Survey.
Feature overview
- Surveys get displayed to wiki users using a nice pop-up form.
- Surveys can be created and modified via an admin panel (Special:Surveys).
- Surveys can be shown to specific types of users (logged in, anonymous, editor, confirmed, …)
- Surveys can be shown for a specified percentage of users, and only after a certain amount of page views.
- Survey questions can have different types of inputs, of which some allow restricting answers.
- Surveys can be tested by admins via Special:TakeSurvey without them being live on the wiki.
- Summaries of survey submissions can be found on Special:SurveyStats.
- The survey submission data (ie user answers) can be exported via an API.
Requirements
Survey makes use of many new features introduced in MediaWiki 1.17, and therefore requires this version or later. It even makes use of MW 1.18 features, with fallback code for MW 1.17 It also makes use of 5.3 features, these being late static binding and anonymous functions, so it won’t work with PHP 5.2.x and earlier.
Some screenshots
Special:Surveys: the main administration interface for Surveys
Special:EditSurvey: administration interface to create and edit surveys
Example of a survey as it appears to users
Special:SurveyStats: summarizes the submissions for a single survey
Downloads
Some background
I developed the Survey extension as WikiWorks consultant for the IEEE, with some help from Yaron Koren.
What’s next?
There are many many features that can be added to this extension to make it even more awesome. I’d like to get some initial feedback on version 0.1, so the usability issues and bugs that might be there can be ironed out. Please place any feedback you might have on the discussion page. This initial release contains all the features my client needed, so if you want to have new capabilities added and can fund the work, definitely contact me
Categories: Programming, Software
As this particular project is coming to and end, I figured I’d do a quick blog post on it.
Wiki Loves Monuments (WLM) is a photo contest for European monuments, organized by Wikimedia this September. Last year some JavaScrip hacks on the regular Wikimedia Commons (the media repository for Wikipedia and other Wikimedia Foundation projects) upload interface where used for this contest. This year the new and completely awesome Upload Wizard (UW) will be used, with configuration optimized for WLM. I created a campaign-based configuration system from the UW and also added a bunch of new settings.
2 new special pages where added. One listing all campaigns, their status, and edit and delete links. This is at Special:UploadCampaigns.
The other special page handles the edit action and displays a list of all available settings that can be modified for the campaign. This is at Special:UploadCampaign/name.
A campaign can be applied to the UW by adding the “campaign” url parameter with as value the campaign name, ie ?campaign=wlm-be.
One fun thing about the architecture of the campaign system is that the setting support is very generic. I created a new settings class that pulls in the default settings, overrides these with the wikis config (ie PHP vars in LocalSettings.php), passed URL arguments and finally the upload campaign settings if a campaign is specified. I like this kind of setup, as it’s a lot nicer then dealing with over 9000 global variables, and in the meanwhile already applied some variation of it in Semantic Signup and in my new Surveys extension. And I wrote up a more general and powerful version of such setting handling in the Maps extension. Unfortunately this code uses late static bindings and thus requires PHP 5.3, making it not usable in actual code for quite a while 
Another neat thing is that the upload campaign class only specifies a lift of settings that should be configurable for upload campaigns, together with what kind of HTML form input they should be displayed. That info is then merged with the settings obtained from the settings class and put into a FormSpecialPage, which uses HTMLForm to display anything without any further hassle
Not used the Upload Wizard before and curious how it works? Go upload some nice stuff to commons then
Categories: Programming
A few days ago I released version 1.2 of the Live Translate MediaWiki extension, which is a major update bringing mainly under-the-hood improvements. I’ve worked on this for about 3 days in my free time, mainly to try out some JavaScript techniques I had not utilized yet.
These are the changes for 1.2:
- Rewrote translation control JavaScript to a jQuery plugin.
- Added caching layer utilizing memory and LocalStorage when available to speed up local translations.
- Added API module to query translation memories.
- Added i18n alias file.
- Added TMX admin user rights group.
This post is about the first two.
Some background
The Live Translate (LT) extension allows live translation of content in wiki pages. For this it uses translation services such as Google Translate or Microsoft Translator. It also allows specifying your own translations for certain words within the wiki, which will then be left alone by the (remote) translation services. Such specifications of translations are called translation memories (TM), and are typically done in a special XML-based format called Translation Memory eXchange (TMX). LT also supports a more wiki-friendly format, custom written, which is DSV-based. Translation memories in both these formats can be embedded in wiki pages designated as TM or you can point to files hosted somewhere else. What it comes down to is that there are a set of local translations, which require special handling: local translation and be ignored by the remote translation service.
On every translation, the JavaScript needs to know which are the special words that have a local translation, so translations for these can be requested, and measures can be taken to not send them to the remote translation services. This means doing a call to the wikis API to obtain these words. In case of big translation memories, this requires several calls to obtain all words, often resulting in a few seconds wait before local translations are even requested. If there are words that have local translations on the page, a single request I send to another part of the API to obtain these translations for the language currently translated to. This usually bring the total time to complete local translation to somewhere between 2 and 5 seconds, after which, in version 1.1 and earlier, remote translation is kicked of.
The idea
Translation memories do not tend to change all the time, so it’s very inefficient to request all special words for every translation, and in somewhat lesser degree to always request the translations. The obvious answer to this is local caching, and since I wanted to play around with HTML5 localStorage a bit, this is exactly what I did. I also wanted to make use of JavaScript capabilities I was not really aware of back in last December, when writing Live Translate, so took this as an opportunity to also do some JavaScript refactoring. These being primarily prototypes, closure scopes and callbacks.
The realization
I made a whole bunch of client side changes (and some server side changes to the API), but the most significant ones are the creation of a translation memory object which takes care of all caching, and the rewrite of the translation control to a jQuery plugin.
Translation memory object
In file includes/ext.lt.tm.js.
The translation memory object class is named simply “memory” and resides in the “lt” namespace. It acts as abstraction layer via which special words and translations of those special words can be accessed. It takes care of all API interaction and caching and exposes 2 simple functions, getSpecialWords and getTranslations, which are called by the translation control.
When the cache is empty, the memory will request a new hash via the API, which indicates the “version” of the translation memories on the server, and is later used for cache invalidation. It the proceeds fetching the requested special words or translations of special words and returns these via a callback passed to either getSpecialWords or getTranslations. Before this last step is done, the obtained data is cached in memory (the words and translations fields, one lines 26 and 30, respectively), and, when available, also in HTML5 localStorage. The in memory caching only yields advantages when doing multiple translations on a single page, which is rather rare, so is not that much of a win. The data stored in localStorage on the other hand, persists when navigating to other pages, even when closing the browser and re-opening it. localStorage really isn’t a cache on it’s own, but the lt.memory class uses it as one.
When the cache is not empty, a single request to the API is made to compare the earlier obtained hash and see if any changes to the TMs have been made, and thus if the cache should be invalidated. If changes have been made, the stored data is discarded and pretty much the same as when the cache was empty happens. If no changes have been made, locally stored data is used where possible. In case of the list of special words, no requests will have to be made at all, since all such words are already known. For the translations of these words it’s a little trickier, since the needed data here varies from page to page, and also depends on both the source and destination language. The lt.memory class checks which of the needed data if available, and in case there is a remainder of non-known translations, requests these. The newly obtained translations are then of course also added to the cache.
jQuery plugin
In file jquery.liveTranslate.js.
This plugin contains a lot of already existing code from Live Translate 1.1, but is structured a lot better. It takes care of creating all the HTML needed for the control (while in 1.1, the HTML was provided, and only events where bound to it) in it’s setup function (line 147). The click event handler for the translation button calls the obatinAndInsetSpecialWords which uses the getSpecialWords function of the lt.memory class to obtain the words with local translations, and then inserts them, meaning that occurrences of these words are wrapped into notranslate spans, which then enables finding all words which should be translated locally, and makes them be ignored by the remote translation services. The click handler function passed doTranslations as completion callback to obatinAndInsetSpecialWords, which starts both local and remote translation in parallel. Local translation is done, as you can uncountably guess, by calling getTranslations function.
The results
Once the cache is warm (the user made a translation before) and valid (the TMs have not changed), local translation is practically instant (~0.4 seconds in my tests). Since remote translation now starts as soon as the special words are known, this can take as little as ~0.2 seconds, a huge difference compares to the earlier up to 5 seconds and possibly longer. All assuming you are using a modern browser of course
Not to forget, the code is a lot better structures now. It should be a lot easier to track the order of execution, and it’s now possible (JavaScript wise) to place multiple translation controls onto a single page (which has little practical value, but indicates a better design).
And, maybe most importantly for me, I now have a much better grasp of the earlier mentioned prototypes, callbacks and closure scopes. Perhaps most of the time I spend on this version was figuring out how to properly use these and debugging out misconceptions I had about how they worked
Live Translate
- Documentation
- Downloads
- SVN trunk (viewvc, svnroot)
Categories: Programming
Today I released the first version of a brand new MediaWiki extension titled Semantic Watchlist. It extends Semantic MediaWiki by adding the capability to watch/follow sets of properties for groups of pages (that can be specified with categories and namespaces). You can view changes to these properties via Special:SemanticWatchlist, which works similar to the regular MediaWiki watchlist. And you can even request to be notified via email when a change is made!
Feature overview:
- A watchlist page listing changes to properties watched by the user.
- Per-user optional email notification per edit that changes properties.
- Integration with user preferences to allow users to specify which watchlist groups they want to follow, and if they want to receive emails on changes.
- Special:WatchListConditions as administration interface for watchlist groups.
- API module to query property changes grouped by edit for a single user.
- API modules to add, modify and delete the watchlist groups.
Let’s have a look at the different parts of the interface:
The watchlist
Each user can view changes to properties they watch on Special:SemanticWatchlist, which looks and works similar to the regular watchlist. Items that have not been viewed yet on the watchlist will be indicated as ‘NEW’.
Watchlist preferences
Each user can manage which watchlist groups they follow via their user preferences. They can also choose if they want to receive email notifications or not. These preferences can be found on Special:Preferences, which is linked at the right top of the page in most skins for logged in users.
Watchlist groups
The watchlist groups can be managed via the Special:WatchlistConditions page by people that have the ‘semanticwatchgroups’ right, by default only administrators. Each group has a name, which allows users to easily recognize the groups in their preferences, and a single category, namespace or concept it covers. Only changes to properties on pages in this category, namespace or concept will be shown to users watching this group. Each group also has a list of properties, which further restricts what property changes should be shown to the user.
Email notifications
When you choose to receive email notifications for changes to semantic properties covered by watchlist groups you watch, they will appear both in your watchlist and your inbox. The below screenshot is a simple example of a notification email.
Extending Semantic Watchlist
Semantic Watchlist is in part a workflow extension, which makes it important for other extensions and tools to interact with it. This is possible via the hooks and API modules Semantic Watchlist provides.
API modules:
- addswlgroup: API module to add semantic watchlist groups.
- deleteswlgroup: API module to delete semantic watchlist groups.
- editswlgroup: API module to modify semantic watchlist groups.
- semanticwatchlist: Returns a list of modified properties per page for a persons semantic watchlist.
Hooks:
- SWLBeforeEmailNotify: $group, $user, $changeSet, $describeChanges, &$title, &$emailText
- SWLBeforeEditInsert: &$this
- SWLAfterEditInsert: &$this
- SWLBeforeChangeSetInsert: &$this, &$groupsToAssociate, &$editId
- SWLAfterChangeSetInsert: &$this, $groupsToAssociate, $editId
Further plans
I think this extension opens up a lot of new possibilities for the SMW platform in the area of workflow. Due to it’s API modules and hooks, it’s very easy for other extensions to build on top of the watchlist functionality, so I’m curious as to what will happen there.
This first release comes with core functionality, but at places lacks polish. And since it’s the first release of a somewhat complex extension, I do expect issues to show up in use cases not tested for yet. Both these factors make it likely that a second release will be made relatively soonish.
Since I think this extension is such a great addition to the already awesome SMW environment, I’m going to give a talk about it at the upcoming SMWCon. That’s bound to yield some interesting feedback
Requirements
- MediaWiki 1.17 or above
- Semantic MediaWiki 1.6 or above
- PHP 5.2 or above
Download
- Semantic Watchlist 0.1 download [zip, 7z]
- List of release downloads
- SVN tag checkout: http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticWatchlist/REL_0_1/
- SVN trunk checkout: http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/SemanticWatchlist/
Some background
I developed the Semantic Watchlist extension as WikiWorks consultant for the IEEE, with some help from Yaron Koren.
Categories: Programming, Software
It’s been a little over half a year since the last mayor release of the Maps and Semantic Maps extensions, but now 1.0, featuring many new features and internal improvements, is here! This is the most significant release since version 0.1, which quite coincidentally, was released exactly 2 years ago today
Let’s have a look at all the new awesomeness:
Google Maps v3 support
Maps has had some very basic support for Google Maps v3 since version 0.5.3, back when the v3 API was still in beta. Right now the v3 API is out of beta, and the v2 one has been deprecated a few months back, so it was definitely time to further implement support for v2. This new version of Google Maps brings many improvements, focusing mostly on performance (loading speed of the maps, esp on mobile devices), but also several other cool things such as streetview support and easy adding of layers such as traffic. Since the v2 API has been deprecated it makes only sense that the default mapping service in Maps has changed from Google Maps v2 to Google Maps v3. It’s not really a new feature, but awesome nonetheless: you don’t need an API key for Google Maps v3! So when setting up Maps on a new wiki (using Google Maps), you won’t have to bother with any API key configuration any more, it’ll just work Support for the former is still there, so if you want to retain the exact same functionality as you have with v0.7 of Maps (or earlier), you can still get those maps using service=googlemaps2 (or format=googlemaps2 in SMW queries). One feature has been removed from the v2 implementation, which is the overlays control. This has to do with internal changes and performance optimizations discussed later on in this blog post. Be sure to check out the Google Maps v3 Maps documentation to discover all the cool features it supports.
Improved form inputs
Since it’s first release Semantic Maps has had form inputs using Google Maps v2, Yahoo! Maps and OpenLayers. These inputs allow for entering geographical coordinates in Semantic Forms forms via a nice GUI with a map and an option to geocode an address. Nothing much has changed to these inputs since that initial release, until now. All input, including a new Google Maps v3 one, now have a “set location” button next to the coordinates box, which sets the map to that location, as was already the case with the geocoding button next to the address field. Some minor layout improvements have also been made, and hitting enter in the coordinates or geocoding fields will result in what you’d expect, rather then submitting the form. The OpenLayers form input now also supports geocoding, making use of the new GeoNames API; more on that later.
Use of the MediaWiki resource loader
MediaWiki 1.17 introduces a resource loader for JavaScript and CSS, which both Maps and Semantic Maps now make use of for all their JS and CSS resources. The resource loader does several neat things, the most important thing being delaying loading (and execution) of resources until after page load as well as combining and minifying them, which is very important for performance (page load time). This means that when you have over 9000 maps on your page, it’ll actually load before protons decay away, initially showing only gray boxes where the maps should be, and then one-by-one loading the maps into them. The resource loader does several other cool things, such as automatic right-to-left conversion of CSS and neat conversion of i18n messages from PHP to JS. Making use of the RL when available but at the same time retaining compatibility with pre-RL MediaWiki turned out to be a bit difficult and it would greatly complicate the code, so I decided to simply not do this. This obviously means you will need MediaWiki 1.17 in order to use (Semantic) Maps 0.8.x and later. Can’t use 1.17 yet for some reason? Don’t panic! I’ll continue to support the 0.7.x for a while longer, fixing bugs as they are found. Don’t expect any new features there though.
Support for the new GeoNames API
Maps has a new geocoding service: the new GeoNames API. It already had support for GeoNames, but it seems this service is now only offered as legacy support. (I’m not completely sure about this, if someone better familiar with the service knows, please poke me.) A big change with the new service is that you need a GeoNames API account to use it, which you can create here. You then need to set your accounts user name in your LocalSettings.php file. Maps still has the ‘geonames’ service as default for #geocode and OpenLayers maps. If you set the account name, the new service will be used, if not, Maps will fall back to the old one, so you can upgrade to 0.8 without geocoding suddenly failing because you don’t have a GeoNames account. For more info see the GeoNames documentation for Maps.
JavaScript overhaul
Similarly to the form inputs in Semantic Maps, all the JavaScript in both Maps and Semantic Maps has seen quite little attention since the initial releases of the extensions. Many additions have been made to add new functionality, but the structure has remained the same ever since. All of it has now been rewritten to jQuery plugins, making it a lot more orderly and easy to extend.
More internal improvements
Not only the JavaScript has seen significant improvements, but some legacy code has also been thrown out of the PHP, making it a lot less complex, more easy to track the code flow and definitely makes it easier to add new functionality. This rewrite is very much a follow up to internal improvements made in versions 0.6, 0.7 and 0.7.3, and completes getting rid of some bad old architecture in the core Maps code. Future releases will therefore most likely focus a lot more on simply adding new features
And more…
For a full list of changes, see the release announcement. These do not list the huge amount of internationalization updates and small improvements made by a lot of contributors by reporting issues and providing patches. Thanks to all!
How stable is it?
A lot of internal changes have been made, but at the same time, most of these have been made about 3 months back. Several wikis have been using alpha versions of 1.0 ever since, and a release candidate was made a few days back. Since there are no known issues at this point, I decided to release 1.0. So yes, it should be pretty stable, although you might run into minor issues with less frequently used components. If you do, please report it, so they can be addressed quickly in a 1.0.1 release.
Legacy support for 0.7.x
As an extension developer and MediaWiki consultant, I’m quite aware that a lot of people are not in positions to update their MediaWiki to 1.17 just yet, preventing them from upgrading Maps and Semantic Maps. For this reason I decided way back when starting the development of version 1.0 to continue limited support of version 0.7.x for a while. Versions 0.7.4 to 0.7.7 have been released especially for this purpose, and I’ll continue to backport important fixes. Don’t expect any new features to show up for 0.7.x though.
What’s next?
Although the current set of functionality is pretty solid, there are many other geographical features one can imagine. Features such as marker clustering, static maps, route plotting (without the use of KML), ect, have been on the wishlist practically since the inception of the Maps extension. There is nothing really standing out for me enough to go ahead and implement it in my free time. If any such feature is important to you and you can fund it’s development, definitely contact me. Of course I’ll continue to support the extension and make fixes where needed.
The Semantic MediaWiki 1.6 release will be followed by a new Semantic Bundle, which will include this new version of Maps and Semantic Maps.
Downloads
- Maps 1.0 (archive)
- Maps 1.0 (svn tag)
- Maps and Semantic Maps 1.0 (archive)
- Semantic Maps 1.0 (svn tag)
- Maps 0.7.7 (archive)
- Maps 0.7.7 (svn tag)
- Maps and Semantic Maps 0.7.7 (archive)
- Semantic Maps 0.7.7 (svn tag)
Referata, which runs the documentation wiki for the mapping extensions has upgraded to version 1.0, so you can have a look at and try out the new features yourself on the example/demo pages.
Categories: Programming, Software
I just released a new version of the Live Translate extension for MediaWiki.
Live Translate is a simple extension that allows live translation of wiki pages using Google Translate or (as of version 1.1) Microsoft Translator. It also enables you to define a “dictionary” of certain words or phrases and their translations; any word or phrase in the original text found in the “dictionary” will be translated using that dictionary, instead of using the Google/Microsoft translation service.
As you can deduce from the above description, one of the new features in version 1.1 is the ability to choose Microsoft Translator as translation service instead of Google Translate, which remains the default for now. The reason for the addition of this service is that Google has announced that it will be retiring it’s free machine translation services in a few months, which Live Translate is using. Another new feature is the ability to place comments into translation memories using the Live Translate Format. Full release notes can be found here.
Downloads
Categories: Programming, Software
It’s been a while since I posted an update on Semantic MediaWiki and extensions in general, and my work specifically. This is due to a pile of work that has been done on different components, each of which I’ll address at the point it’s released. In this blog post I’ll provide you with a short overview of what”s (been) going on in the SMW world.
Almost two years after the latest big release, SMW 1.5, 1.6 comes with many important internal changes focused on performance, stability and extensibility and several new features. Here you have an extract of the release notes as they currently are on SVN trunk:
* Full support for synchronizing RDF stores with SMW, and for answering #ask queries based on this data. The communication happens via SPARQL (1.1), and all SPARQL-capable stores should be supported.
* The Type namespace has been abolished. Builtin types now are displayed by the special page Special:Types, and there are no “custom types” any longer. By default, the Type namespace is gone and existing pages in this namespace can no longer be accessed. This can be changed by setting $smwgHistoricTypeNamespace = true in LocalSettings.php before including SMW.
* Changed the way in which units of measurement work. Type:Number now does not accept any units, and a new type “Quantity” is used for numbers with units. Units must be declared on the property page (not on the Type page as before), and only units that have a declared conversion factor are accepted.
* The declaration of Type:Record properties has changed. Instead of a list of datatypes, the declaration now requires a list of properties that are to be used for the fields of the record. The declaration is still done with the property “has fields” as before.
* Changed the way parameters in query printers are specified and handled using the Validator extension. This includes improvements to the parameter options in the Special:Ask GUI and better error reporting for ask queries.
* Added UNIX-style DSV (Delimiter-separated values) result format.
* Reworked internal data model, cleaning up and re-implementing SMWDataValue and all of its subclasses, and introducing new data item classes to handle data. The class SMWCompatibilityHelpers provides temporal help for extensions that still depend on the old format and APIs.
* Fixed PostGre SQL issues with the installation and upgrade code.
* Added API module (smwinfo) via which statistics about the semantic data can be obtained.
That’s a lot of awesomeness no?
As you can deduce from the above notes, this release is not fully backwards compatibility with SMW 1.5.x, so it’s possible you’ll need to do some migration work. The Validator extension is also introduced as an extra dependency, but it will come bundled with SMW, so you’ll only need to care about this when getting the code from SVN.
SMW 1.6 has been in testing phase for 2 weeks or so now, and most bugs have been taken care of. With some luck, the new version will be released in a week or two Do feel free to try out the new version on non-critical wikis and report any issues you might find.
I already stated that SMW 1.6 is not fully feature compatible with SMW 1.5.x, but it’s also most definitely not compatible with earlier versions code-wise for extensions. This means that quite some SMW extensions released before the development on SMW 1.6 started won’t be compatible anymore, and will require you to also update them to their latest release when upgrading SMW to 1.6 or later. The ones that are in the Semantic Bundle are all compatible already on SVN trunk, so you should be able to upgrade everything as soon as SMW 1.6 is released.
Maps and Semantic Maps 1.0
Another very significant release is the one of Maps and Semantic Maps. I’ve been working on this version for quite a while; it was branched from version 0.7.3, and it’s the most significant (and awesome) release since the creation of both extensions, hence the bump from 0.7.x to 1.0. These are the changes:
* Added full Google Maps v3 support and set it as the default mapping service.
* Added new geocoder making use of the new GeoNames API.
* Added support for the auto-documentation features for parser hooks introduced in Validator 0.4.3.
* Added resizeable parameter to all mapping services except OSM.
* Removed compatibility with pre MediaWiki 1.17.
* Removed overlays parameter for Google Maps v2.
* Usage of the Resource Loader for all scripts and stylesheets.
* Rewrote all the map JavaScript to jQuery plugins.
* Rewrote the way parameters are translated to JavaScript. Now one big PHP object is json_encoded.
* Improved KML formatter.
* Use of Google Maps geocoding service v3 instead of v2.
* Fixed geocoding service overriding based on mapping service (merged in from Maps 0.7.5).
* Fixed fatal error occurring when running maintenance/refreshLinks.php.
* Improved default width of maps (merged in from Maps 0.7.5).
* Improved map query parameter support in the Special:Ask GUI
* Rewrote the map printers to use the SMQueryHandler class.
* Added geocoding capability to the OpenLayers form input when having a GeoNames API account.
* Added ‘update map’ button to all form inputs.
This release of the mapping extensions requires MediaWiki 1.17 or later and the new SMW 1.6, or later. For people not running an MW older then 1.17 (which currently is still not released, pretty much blocking this release of Maps and SM), the 0.7.x branch still remains supported for a while. It’s currently at 0.7.6, and I plan to release 0.7.7 soonish. Do note that Semantic Maps 0.7.x is NOT compatible with SMW 1.6 or later, it needs SMW 1.5.1 – 1.5.7 alpha.
Semantic Forms 2.2
Yaron Koren has been working on the next big release of the most popular SMW extension, Semantic Forms. It brings compatibility with SMW 1.6, and adds several new features, including:
- #autoedit parser function that allows creating a link that, when clicked, automatically sets one or more fields in another page to certain values.
- “Save and continue” button
- Handling of boolean properties can now also be done using radiobuttons and dropdowns, instead of only checkboxes.
Semantic Watchlist
Semantic Watchlist is a new SMW extension I’ve developed for the IEEE as WikiWorks consultant. Semantic Watchlist enables users to watch semantic properties by adding a new watchlist page that lists changes to these properties. Users can choose to follow one or more watchlist groups, which are administrator defined, and cover a set of properties and a set of pages (category, namespace, or SMW concept). Notification of changes to watched properties is also possible via email. I think it’s totally awesome.
It’s main features are:
- A watchlist page listing changes to properties watched by the user.
- Per-user optional email notification per edit that changes properties.
- Integration with user preferences to allow users to specify which watchlist groups they want to follow, and if they want to receive emails on changes.
- Special:WatchListConditions as administration interface for watchlist groups.
- API module to query property changes grouped by edit for a single user.
- API modules to add, modify and delete the watchlist groups.
It requires MediaWiki 1.17 or later and SMW 1.6 or later, and still has to see an initial release. It’s pretty much ready for it, and can be seen as beta right now.
The Spring 2011 SMWCon was held on April 28-30, 2011 at the Raytheon BBN Technologies office in Arlington, Virginia, in the Washington, DC area, and it was a great success. You can read more about it in Yarons writeup.
The next SMWCon, SMWCon Fall 2011, will be held on September 21–23, 2011 in Berlin, Germany. Berlin – yay! It’s going to be awesome, and I’ll be attending, probably giving some talk about Maps and Semantic Maps, and possibly other extensions as well (Semantic Watchlist being a good candidate).
Like I already noted, I’ll be posting more comprehensive (and official) release announcements for each extension when they are released I’d also like to point out that this is definitely not everything that’s been going on in the SMW world. For example there are 2 Google Summer of Code students doing SMW related work, about which I might write later on, and many people are doing SMW projects that I’m simply not aware of or am not closely following.
Categories: Programming, Software
Yesterday I quickly wrote up a simple (but awesome) MediaWiki extension that allows you to make use of the Spark library in your wiki.
Spark as described on the Spark website:
The web is not only growing in sheer size, but it also grows in how much it is interconnected. Where once the Web was a set of more or less separated sites, today sites are more and more being connected. More and more data is being offered on the Web in a way that can be further processed, and more and more sites and applications are using external data. More and more mashups are created, where data from different sources is integrated and displayed with novel visualisations.
Spark is a library that enables HTML authors to create mashups more easily than ever before. Using standard Web technologies like SPARQL, RDF, HTML5, and JavaScript, Spark can query external knowledge sources (so called triple stores or SPARQL endpoints), and then visualise the results.
With Spark, website developers can create visually appealing mashups without having to write a single line of JavaScript, but merely using some markup elements describing the source of the data that is to be shown, a query to select the appropriate data, and selecting one from an expandable set of visualisations and their parameters.
Spark is developed by Denny VrandeÄić and Andreas Harth.
This MediaWiki extension, unsurprisingly titled Spark, adds a <spark> tag to MediaWiki which is equivalent to <div class=”spark”> as described in the spark library documentation. All parameters (except the class=”spark” one) can just be copied over between spark divs in web pages, and the <spark> tag in MediaWiki. It is currently at version 0.1, which is a beta release. It includes a still experimental version of the Spark library, so you should probably not use this extension on production websites just yet. The Spark people are looking for developers to help out, so if you want to play around with SPARQL a bit, like I basically did with this extension, be sure to poke them
The extension required MediaWiki 1.17 or above (as it makes use of the new Resource Loader) and PHP 5.2 or later.
Download
- The current Spark (MediaWiki extension) release (zip)
- svn co http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/Spark/REL_0_1 (tag for 0.1, including Spark lib trunk)
- svn co http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/Spark (trunk, including Spark lib trunk)
- svn co http://rdf-spark.googlecode.com/svn/trunk/src/ (Spark lib trunk)
Further possibilities
Right now you can embed mashups with SPARQL queries that get their data from some SPARQL endpoint. This opens up a whole bunch of possibilities, but is a bit silly when you are running your own Semantic MediaWiki instance and want to visualize structured data stored by it using Spark. A possible addition to the Spark MediaWiki extension therefore is having support for Spark as a so called SMW result format. For this translation from the SMW ask query language to SPARQL is needed, which is some work. I might implement this at some future point, but have several other things I want to poke at, so it won’t be soonish
Categories: Programming, Software
Previous Entries
Categories
Tag Cloud
Blog RSS
Comments RSS
Last 50 Posts
Back
Void 
Life 
Earth 
Wind 
Water 
Fire