KML/KMZ support

The Google Maps v3 service now supports 2 new KML related parameters: kml and gkml. Both accept a url pointing to a KML (or KMLZ) file. The first one uses a KML parsing library (geoxml3) included in the extension to translate the features described in the KML file into elements to place onto the map. This is very nice for people that do not want to be dependent on third party services, but sadly enough, the library is somewhat limited. It lacks support for more advanced KML features such as polygons and paths. The gkml parameter uses Googles KML service, which pulls the KML file to some Google server, and then decides if it should be send to the client (for simple and small files), or if it should be rendered server side and send as tiles to the client (for big files or files containing advanced features such as polygons).

You can now also choose if you want the map to rezoom after the KML layers have been loaded or not using the kmlrezoom parameter. KML layers will load a bit after the map, since they require extra resources to be loaded, and there is no need to let the user wait to see the normal map until those are done loading.

Google Earth support

Maps already has support for Google Earth since on of it’s earliest versions. This was quite easy to achieve as Google Maps v2 natively supported it. Now with the switch to Google Maps v3 in Maps 1.0, people asked for Google Maps support in that as well. Unfortunately Google Earth is not natively supported here. Maps now provides support for GE using the Google Maps utility library v3. The earth type can be enabled by adding “earth” to the types parameter, or setting it as the default type using “type”. Do however note that due to this not being officially supported by Google, it has some deficiencies. For one, when switching to GE, the map controls won’t be displayed any more, preventing you from switching back. Also, the GE plugin is only supported on Windows and Mac, so won’t be usable for mobile or Linux users.

A completely new thing added in 1.0.3 is the tilt parameter, which, as you can probably guess, allows you to set the initial tilt of the GE layer.

Full list of changes since 1.0:

• Added support for Google Earth in Google Maps v3.
• Added tilt parameter for Google Earth in Google Maps v3.
• Added kmlrezoom parameter for Google Maps v3 and general $egMapsRezoomForKML setting.
• Added API module for geocoding.
• Added ‘geocoding’ right.
• Added language parameter to the mapsdoc hook.
• Use of Validator 0.4.10s setMessage method instead of setDescription for better i18n.
• Minor improvement to script loading.
• Fixed Google Maps v3 JavaScript issue occurring on MediaWiki 1.17.
• Fixed internationalization of distances (bug 30467).
• Fixed zoom and types parameters for Google Maps v3.

• Added geocodecontrol parameter for the form inputs.
• Fixed handling of the semantic link caption in the geographical coordinate datavalue class.

What’s next?

For now, I have no specific plans for changes or addition to either of the mapping extensions, further then some minor script loading improvements, as I ‘m working on several other projects. However it’s likely that people will have suggestions for new features at SMWCon Fall 2011, which is next week.

Download

• Maps downloads
• Maps and Semantic Maps downloads
• Maps SVN: tags, trunk
• Semantic Maps SVN: tags, trunk

Support for RDF triplestores

Full support added for storing SMW data directly within an RDF triplestore, and for answering #ask queries based on this data. The communication happens via SPARQL (1.1), and all SPARQL-capable stores should be supported. The following settings are needed in LocalSettings.php:

$smwgDefaultStore = ‘SMWSparqlStore’;
$smwgSparqlDatabase = ‘SMWSparqlDatabase’;
// The following should be set to the URLs to reach the store:
$smwgSparqlQueryEndpoint = ‘http://localhost:8080/sparql/’;
$smwgSparqlUpdateEndpoint = ‘http://localhost:8080/update/’;
$smwgSparqlDataEndpoint = ‘http://localhost:8080/data/’; // can be empty

The specific support that SMW used to have for the RAP RDF store has been discontinued.

The Type namespace has been abolished

Builtin types now are displayed by the special page Special:Types, and there are no “custom types” any longer. By default, the Type namespace is gone and existing pages in this namespace can no longer be accessed. This can be changed by setting $smwgHistoricTypeNamespace = true in LocalSettings.php before including SMW.

Changes to units of measurement

Changed the way in which units of measurement work. Type:Number now does not accept any units, and a new type “Quantity” is used for numbers with units. Units must be declared on the property page (not on the Type page as before), and only units that have a declared conversion factor are accepted.

Type:Record changes

The declaration of Type:Record properties has changed. Instead of a list of datatypes, the declaration now requires a list of properties that are to be used for the fields of the record. The declaration is still done with the property “has fields” as before. Properties must not be used more than once in has_fields, or the order of values will be random.

Reworked internal data model

Reworked internal data model, cleaning up and re-implementing SMWDataValue and all of its subclasses, and introducing new data item classes to handle data. The class SMWCompatibilityHelpers provides temporary help for extensions that still depend on the old format and APIs.

This change is very important compatibility wise, as it removes support for some older behavior and deprecates a bunch more. Older versions of a lot of SMW extensions will not work with SMW 1.6. However, since 1.6 has been in the making for quite some time, most, if not all, of these extensions have released one or more versions that work just fine with 1.6. So if you stick with the latest releases, you should be good.

There are already 2 SMW extensions that now require SMW version 1.6 or later: Semantic Maps 1.0 and the new kid in town: Semantic Watchlist (see Semantic Watchlist release post).

Result formats make use of Validator

Semantic MediaWiki now makes use of the Validator extension to handle parameters passed to result printers. This opens the door to numerous awesome new functionality. For one, it allows for better parameter error feedback to the user. In SMW < 1.6 when you enter a text value for a numeric parameter, it’ll simple be ignored (without giving any warning). With this new approach the user will get a warning along the lines of “The value ‘foobar’ is not valid for parameter limit, it must be an integer.”. Another advantage of using Validator here is that the query argument GUI on Special:Ask can show more fitting controls and display default values. A third win, is that writing new query printers has now become easier, as you just have to specify the parameters they should accept, and then don’t have to bother with cleaning, validating and formatting them. Further things that can be done with this in the future are automatic generation of documentation per result format, which would be a huge improvement over the current documentation and smarter behavior (such as autocompletion and live validation) in interfaces where the values for these parameters can be specified.

New DSV format

I added a new result format to SMW that outputs data as UNIX-style Delimiter Separated Values. For those not familiar with the differences between DSV and CSV (which was already supported), it comes down to sane escaping of the delimiter characters. Instead of the weird and hard to parse approach taken in CSV, the familiar backlashing technique is used in DSV. You can use this format by adding “format=dsv” to any ask query.

New API module for version info

This is a very small feature I added geared towards developers and the SMW registry tool in particular; it has not relevance to regular end users. Up until now the SMW registry tool obtained info about SMW installs by scraping Special:Version, which is a HTML page intended for humans to look at, and by scraping Special:SemanticStatistics, which is also a HTML page. Now the info provided by these pages and used by the SMW registry, ie the version of SMW and the number of properties, users and pages, can be obtained via the MediaWiki API using action=smwinfo.

New smwdoc parser hook

Similar to the new API module, this new parser hook is not geared to most end users, but rather to documenters and site admins. This hook utilizes Validators auto documentation capabilities to display tables with parameter info, such as name, type, default value and description, for a specified result format. It’s in essence a result format oriented  version of Validators more general purpose describe parser hook. As soon as the SMW documentation wiki updates to 1.6, it’ll be possible to use this hook to fill part of the documentation gap there unfortunately is.

Some noteworthy fixes

• PostgreSQL support
• Output issues in the JSON result format

I’m probably forgetting others here

Downloads

• Semantic MediaWiki 1.6 release, zip archive
• Semantic MediaWiki 1.6 release, 7z archive
• Semantic MediaWiki 1.6 release, svn tag
• Semantic MediaWiki, svn trunk

Do note that as of this release, SMW requires Validator, which is included in the release packages, but obviously needs to be obtained separately when checking out with SVN.

What’s next?

Devayon Das, a Google Summer of Code student, mentored by Markus Krötzsch, is working on improving the search and browse interfaces of SMW (read all about it here). This work will probably end up in one of the upcoming 1.6.x releases. I’d also like to make some improvements to the automatic documentation generation via the smwdoc parser hook by the next release, mainly focusing on internationalization. I’m not aware of any other concrete plans to make changes to SMW itself for now. There are many things that can be done, and several projects that are being looked at, but only time will tell which of these get any traction.

In any case, if you are interested in the future of SMW, I definitely recommend attending SMWCon, the SMW event that’s held once a year in both the US and the EU. The next one will take place from September 21st to September the 23rd, in Berlin, Germany.

More frequent updates

In an effort to provide the community with more frequent updates on the status of SMW development and other news, a Twitter and an Identi.ca account have been set up. You can follow these to stay up to date on the latest SMW developments

Further info

• Semantic MediaWiki 1.6 page on the SMW wiki
• Installation and upgrade instructions
• Raw release notes

Feature overview:

• A watchlist page listing changes to properties watched by the user.
• Per-user optional email notification per edit that changes properties.
• Integration with user preferences to allow users to specify which watchlist groups they want to follow, and if they want to receive emails on changes.
• Special:WatchListConditions as administration interface for watchlist groups.
• API module to query property changes grouped by edit for a single user.
• API modules to add, modify and delete the watchlist groups.

Let’s have a look at the different parts of the interface:

The watchlist

Each user can view changes to properties they watch on Special:SemanticWatchlist, which looks and works similar to the regular watchlist. Items that have not been viewed yet on the watchlist will be indicated as ‘NEW’.

Watchlist preferences

Each user can manage which watchlist groups they follow via their user preferences. They can also choose if they want to receive email notifications or not. These preferences can be found on Special:Preferences, which is linked at the right top of the page in most skins for logged in users.

Watchlist groups

The watchlist groups can be managed via the Special:WatchlistConditions page by people that have the ‘semanticwatchgroups’ right, by default only administrators. Each group has a name, which allows users to easily recognize the groups in their preferences, and a single category, namespace or concept it covers. Only changes to properties on pages in this category, namespace or concept will be shown to users watching this group. Each group also has a list of properties, which further restricts what property changes should be shown to the user.

Email notifications

When you choose to receive email notifications for changes to semantic properties covered by watchlist groups you watch, they will appear both in your watchlist and your inbox. The below screenshot is a simple example of a notification email.

Extending Semantic Watchlist

Semantic Watchlist is in part a workflow extension, which makes it important for other extensions and tools to interact with it. This is possible via the hooks and API modules Semantic Watchlist provides.

API modules:

• addswlgroup: API module to add semantic watchlist groups.
• deleteswlgroup: API module to delete semantic watchlist groups.
• editswlgroup: API module to modify semantic watchlist groups.
• semanticwatchlist: Returns a list of modified properties per page for a persons semantic watchlist.

Hooks:

• SWLBeforeEmailNotify: $group, $user, $changeSet, $describeChanges, &$title, &$emailText
• SWLBeforeEditInsert: &$this
• SWLAfterEditInsert: &$this
• SWLBeforeChangeSetInsert: &$this, &$groupsToAssociate, &$editId
• SWLAfterChangeSetInsert: &$this, $groupsToAssociate, $editId

Further plans

I think this extension opens up a lot of new possibilities for the SMW platform in the area of workflow. Due to it’s API modules and hooks, it’s very easy for other extensions to build on top of the watchlist functionality, so I’m curious as to what will happen there.

This first release comes with core functionality, but at places lacks polish. And since it’s the first release of a somewhat complex extension, I do expect issues to show up in use cases not tested for yet. Both these factors make it likely that a second release will be made relatively soonish.

Since I think this extension is such a great addition to the already awesome SMW environment, I’m going to give a talk about it at the upcoming SMWCon. That’s bound to yield some interesting feedback

Requirements

• MediaWiki 1.17 or above
• Semantic MediaWiki 1.6 or above
• PHP 5.2 or above

Download

• Semantic Watchlist 0.1 download [zip, 7z]
• List of release downloads
• SVN tag checkout: http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticWatchlist/REL_0_1/
• SVN trunk checkout: http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/SemanticWatchlist/

Some background

I developed the Semantic Watchlist extension as WikiWorks consultant for the IEEE, with some help from Yaron Koren.

Let’s have a look at all the new awesomeness:

Google Maps v3 support

Maps has had some very basic support for Google Maps v3 since version 0.5.3, back when the v3 API was still in beta. Right now the v3 API is out of beta, and the v2 one has been deprecated a few months back, so it was definitely time to further implement support for v2. This new version of Google Maps brings many improvements, focusing mostly on performance (loading speed of the maps, esp on mobile devices), but also several other cool things such as streetview support and easy adding of layers such as traffic. Since the v2 API has been deprecated it makes only sense that the default mapping service in Maps has changed from Google Maps v2 to Google Maps v3. It’s not really a new feature, but awesome nonetheless: you don’t need an API key for Google Maps v3! So when setting up Maps on a new wiki (using Google Maps), you won’t have to bother with any API key configuration any more, it’ll just work Support for the former is still there, so if you want to retain the exact same functionality as you have with v0.7 of Maps (or earlier), you can still get those maps using service=googlemaps2 (or format=googlemaps2 in SMW queries). One feature has been removed from the v2 implementation, which is the overlays control. This has to do with internal changes and performance optimizations discussed later on in this blog post. Be sure to check out the Google Maps v3 Maps documentation to discover all the cool features it supports.

Improved form inputs

Since it’s first release Semantic Maps has had form inputs using Google Maps v2, Yahoo! Maps and OpenLayers. These inputs allow for entering geographical coordinates in Semantic Forms forms via a nice GUI with a map and an option to geocode an address. Nothing much has changed to these inputs since that initial release, until now. All input, including a new Google Maps v3 one, now have a “set location” button next to the coordinates box, which sets the map to that location, as was already the case with the geocoding button next to the address field. Some minor layout improvements have also been made, and hitting enter in the coordinates or geocoding fields will result in what you’d expect, rather then submitting the form. The OpenLayers form input now also supports geocoding, making use of the new GeoNames API; more on that later.

Use of the MediaWiki resource loader

MediaWiki 1.17 introduces a resource loader for JavaScript and CSS, which both Maps and Semantic Maps now make use of for all their JS and CSS resources. The resource loader does several neat things, the most important thing being delaying loading (and execution) of resources until after page load as well as combining and minifying them, which is very important for performance (page load time). This means that when you have over 9000 maps on your page, it’ll actually load before protons decay away, initially showing only gray boxes where the maps should be, and then one-by-one loading the maps into them. The resource loader does several other cool things, such as automatic right-to-left conversion of CSS and neat conversion of i18n messages from PHP to JS. Making use of the RL when available but at the same time retaining compatibility with pre-RL MediaWiki turned out to be a bit difficult and it would greatly complicate the code, so I decided to simply not do this. This obviously means you will need MediaWiki 1.17 in order to use (Semantic) Maps 0.8.x and later. Can’t use 1.17 yet for some reason? Don’t panic! I’ll continue to support the 0.7.x for a while longer, fixing bugs as they are found. Don’t expect any new features there though.

Support for the new GeoNames API

Maps has a new geocoding service: the new GeoNames API. It already had support for GeoNames, but it seems this service is now only offered as legacy support. (I’m not completely sure about this, if someone better familiar with the service knows, please poke me.) A big change with the new service is that you need a GeoNames API account to use it, which you can create here. You then need to set your accounts user name in your LocalSettings.php file. Maps still has the ‘geonames’ service as default for #geocode and OpenLayers maps. If you set the account name, the new service will be used, if not, Maps will fall back to the old one, so you can upgrade to 0.8 without geocoding suddenly failing because you don’t have a GeoNames account. For more info see the GeoNames documentation for Maps.

JavaScript overhaul

Similarly to the form inputs in Semantic Maps, all the JavaScript in both Maps and Semantic Maps has seen quite little attention since the initial releases of the extensions. Many additions have been made to add new functionality, but the structure has remained the same ever since. All of it has now been rewritten to jQuery plugins, making it a lot more orderly and easy to extend.

More internal improvements

Not only the JavaScript has seen significant improvements, but some legacy code has also been thrown out of the PHP, making it a lot less complex, more easy to track the code flow and definitely makes it easier to add new functionality. This rewrite is very much a follow up to internal improvements made in versions 0.6, 0.7 and 0.7.3, and completes getting rid of some bad old architecture in the core Maps code. Future releases will therefore most likely focus a lot more on simply adding new features

And more…

For a full list of changes, see the release announcement. These do not list the huge amount of internationalization updates and small improvements made by a lot of contributors by reporting issues and providing patches. Thanks to all!

How stable is it?

A lot of internal changes have been made, but at the same time, most of these have been made about 3 months back. Several wikis have been using alpha versions of 1.0 ever since, and a release candidate was made a few days back. Since there are no known issues at this point, I decided to release 1.0. So yes, it should be pretty stable, although you might run into minor issues with less frequently used components. If you do, please report it, so they can be addressed quickly in a 1.0.1 release.

Legacy support for 0.7.x

As an extension developer and MediaWiki consultant, I’m quite aware that a lot of people are not in positions to update their MediaWiki to 1.17 just yet, preventing them from upgrading Maps and Semantic Maps. For this reason I decided way back when starting the development of version 1.0 to continue limited support of version 0.7.x for a while. Versions 0.7.4 to 0.7.7 have been released especially for this purpose, and I’ll continue to backport important fixes. Don’t expect any new features to show up for 0.7.x though.

What’s next?

Although the current set of functionality is pretty solid, there are many other geographical features one can imagine. Features such as marker clustering, static maps, route plotting (without the use of KML), ect, have been on the wishlist practically since the inception of the Maps extension. There is nothing really standing out for me enough to go ahead and implement it in my free time. If any such feature is important to you and you can fund it’s development, definitely contact me. Of course I’ll continue to support the extension and make fixes where needed.

The Semantic MediaWiki 1.6 release will be followed by a new Semantic Bundle, which will include this new version of Maps and Semantic Maps.

Downloads

• Maps 1.0 (archive)
• Maps 1.0 (svn tag)
• Maps and Semantic Maps 1.0 (archive)
• Semantic Maps 1.0 (svn tag)

• Maps 0.7.7 (archive)
• Maps 0.7.7 (svn tag)
• Maps and Semantic Maps 0.7.7 (archive)
• Semantic Maps 0.7.7 (svn tag)

Referata, which runs the documentation wiki for the mapping extensions has upgraded to version 1.0, so you can have a look at and try out the new features yourself on the example/demo pages.

Semantic MediaWiki 1.6

Almost two years after the latest big release, SMW 1.5, 1.6 comes with many important internal changes focused on performance, stability and extensibility and several new features. Here you have an extract of the release notes as they currently are on SVN trunk:

* Full support for synchronizing RDF stores with SMW, and for answering #ask queries based on this data. The communication happens via SPARQL (1.1), and all SPARQL-capable stores should be supported.
* The Type namespace has been abolished. Builtin types now are displayed by the special page Special:Types, and there are no “custom types” any longer. By default, the Type namespace is gone and existing pages in this namespace can no longer be accessed. This can be changed by setting $smwgHistoricTypeNamespace = true in LocalSettings.php before including SMW.
* Changed the way in which units of measurement work. Type:Number now does not accept any units, and a new type “Quantity” is used for numbers with units. Units must be declared on the property page (not on the Type page as before), and only units that have a declared conversion factor are accepted.
* The declaration of Type:Record properties has changed. Instead of a list of datatypes, the declaration now requires a list of properties that are to be used for the fields of the record. The declaration is still done with the property “has fields” as before.
* Changed the way parameters in query printers are specified and handled using the Validator extension. This includes improvements to the parameter options in the Special:Ask GUI and better error reporting for ask queries.
* Added UNIX-style DSV (Delimiter-separated values) result format.
* Reworked internal data model, cleaning up and re-implementing SMWDataValue and all of its subclasses, and introducing new data item classes to handle data. The class SMWCompatibilityHelpers provides temporal help for extensions that still depend on the old format and APIs.
* Fixed PostGre SQL issues with the installation and upgrade code.
* Added API module (smwinfo) via which statistics about the semantic data can be obtained.

That’s a lot of awesomeness no?

As you can deduce from the above notes, this release is not fully backwards compatibility with SMW 1.5.x, so it’s possible you’ll need to do some migration work. The Validator extension is also introduced as an extra dependency, but it will come bundled with SMW, so you’ll only need to care about this when getting the code from SVN.

SMW 1.6 has been in testing phase for 2 weeks or so now, and most bugs have been taken care of. With some luck, the new version will be released in a week or two Do feel free to try out the new version on non-critical wikis and report any issues you might find.

I already stated that SMW 1.6 is not fully feature compatible with SMW 1.5.x, but it’s also most definitely not compatible with earlier versions code-wise for extensions. This means that quite some SMW extensions released before the development on SMW 1.6 started won’t be compatible anymore, and will require you to also update them to their latest release when upgrading SMW to 1.6 or later. The ones that are in the Semantic Bundle are all compatible already on SVN trunk, so you should be able to upgrade everything as soon as SMW 1.6 is released.

Maps and Semantic Maps 1.0

Another very significant release is the one of Maps and Semantic Maps. I’ve been working on this version for quite a while; it was branched from version 0.7.3, and it’s the most significant (and awesome) release since the creation of both extensions, hence the bump from 0.7.x to 1.0. These are the changes:

* Added full Google Maps v3 support and set it as the default mapping service.
* Added new geocoder making use of the new GeoNames API.
* Added support for the auto-documentation features for parser hooks introduced in Validator 0.4.3.
* Added resizeable parameter to all mapping services except OSM.
* Removed compatibility with pre MediaWiki 1.17.
* Removed overlays parameter for Google Maps v2.
* Usage of the Resource Loader for all scripts and stylesheets.
* Rewrote all the map JavaScript to jQuery plugins.
* Rewrote the way parameters are translated to JavaScript. Now one big PHP object is json_encoded.
* Improved KML formatter.
* Use of Google Maps geocoding service v3 instead of v2.
* Fixed geocoding service overriding based on mapping service (merged in from Maps 0.7.5).
* Fixed fatal error occurring when running maintenance/refreshLinks.php.
* Improved default width of maps (merged in from Maps 0.7.5).
* Improved map query parameter support in the Special:Ask GUI
* Rewrote the map printers to use the SMQueryHandler class.
* Added geocoding capability to the OpenLayers form input when having a GeoNames API account.
* Added ‘update map’ button to all form inputs.

This release of the mapping extensions requires MediaWiki 1.17 or later and the new SMW 1.6, or later. For people not running an MW older then 1.17 (which currently is still not released, pretty much blocking this release of Maps and SM), the 0.7.x branch still remains supported for a while. It’s currently at 0.7.6, and I plan to release 0.7.7 soonish. Do note that Semantic Maps 0.7.x is NOT compatible with SMW 1.6 or later, it needs SMW 1.5.1 – 1.5.7 alpha.

Semantic Forms 2.2

Yaron Koren has been working on the next big release of the most popular SMW extension, Semantic Forms. It brings compatibility with SMW 1.6, and adds several new features, including:

• #autoedit parser function that allows creating a link that, when clicked, automatically sets one or more fields in another page to certain values.
• “Save and continue” button
• Handling of boolean properties can now also be done using radiobuttons and dropdowns, instead of only checkboxes.

Semantic Watchlist

Semantic Watchlist is a new SMW extension I’ve developed for the IEEE as WikiWorks consultant. Semantic Watchlist enables users to watch semantic properties by adding a new watchlist page that lists changes to these properties. Users can choose to follow one or more watchlist groups, which are administrator defined, and cover a set of properties and a set of pages (category, namespace, or SMW concept). Notification of changes to watched properties is also possible via email. I think it’s totally awesome.

It’s main features are:

• A watchlist page listing changes to properties watched by the user.
• Per-user optional email notification per edit that changes properties.
• Integration with user preferences to allow users to specify which watchlist groups they want to follow, and if they want to receive emails on changes.
• Special:WatchListConditions as administration interface for watchlist groups.
• API module to query property changes grouped by edit for a single user.
• API modules to add, modify and delete the watchlist groups.

It requires MediaWiki 1.17 or later and SMW 1.6 or later, and still has to see an initial release. It’s pretty much ready for it, and can be seen as beta right now.

SMWCon

The Spring 2011 SMWCon was held on April 28-30, 2011 at the Raytheon BBN Technologies office in Arlington, Virginia, in the Washington, DC area, and it was a great success. You can read more about it in Yarons writeup.

The next SMWCon, SMWCon Fall 2011, will be held on September 21–23, 2011 in Berlin, Germany. Berlin – yay! It’s going to be awesome, and I’ll be attending, probably giving some talk about Maps and Semantic Maps, and possibly other extensions as well (Semantic Watchlist being a good candidate).

Like I already noted, I’ll be posting more comprehensive (and official) release announcements for each extension when they are released I’d also like to point out that this is definitely not everything that’s been going on in the SMW world. For example there are 2 Google Summer of Code students doing SMW related work, about which I might write later on, and many people are doing SMW projects that I’m simply not aware of or am not closely following.

New tagcloud format

This release introduces a new “tagcloud” format which I created for WikiWorks. It allows displaying queried values in a tag cloud based on how many times they occur in the result set. Several options allow you to specify the minimum and maximum tags sizes, how to increase (logarithmic or linear), how to order the tags and more. The format is enabled by default, so when updating SRF to the latest version, you can immediately use it.

Improved gallery format

Some significant improvements have been made to the gallery format. Previously it was only able to display images that are returned as subjects in the query result. This is useful in some cases, but forces you to have semantic properties on your image pages, which is not a good approach if you want to have this info on the page the images belong to. Now it’s possible to point to images using a page property and then do a query in which you specify this property points to the images you want to display. An option to disable automatic captions has also been added.

And more…

Several other improvements have also been made, including fixes to the jqplot format by Yaron Koren and some clean-up of the timeline and eventline formats.

Download

Since SRF was lacking a place for release downloads, I created a new Google Code project where they are put now.

• Stable releases
• Releases as part of Semantic Bundle
• SVN tags
• SVN trunk

Semantic MediaWiki 1.5.4

Yes, 1.5.4. Since my previous blog post, I have made 2 minor updates to the SMW 1.5.x branch: SMW 1.5.3 on November 4th and SMW 1.5.4 on December 1st. The focus of these releases is mainly on fixing bugs, increasing security and compatibility with MediaWiki 1.17. The only significant new feature are explicit strict and non-strict greater-then and less-then comparators, which Saruman paid me to add.

The Semantic MediaWiki documentation wiki also got a facelift, including a new logo and redesigned Style. Most of the work here was done by Markus Krötzsch and Yaron Koren. See Yarons blog post for more info on the new look.

Maps and Semantic Maps 0.7.3

After releasing Maps and Semantic Maps 0.7.1, I continued refining the new images-as-layers functionality that this release introduced. As I had accidentally broken compatibility with MediaWiki 1.15.x without noticing, I ended up doing a pretty quick 0.7.2 release, about a week later. In late November I got contracted to add KML export functionality to Semantic Maps, which I implemented in the form of a new ‘kml’ result format, and then released in version 0.7.3. Some additional options for the KML format are available when using the code which is on svn trunk, but that’s not enough of a change to justify a new release just yet.

Semantic Bundle

Several important releases of other SMW related extensions that are in the Semantic Bundle have also been made, most notable to Semantic Forms, which has progressed no less then 5 minor releases since my last post. Each of these was followed by a Semantic Bundle release, which also include updates to Semantic Internal Objects and ApprovedRevs. Several compatibility improvements have been made to Semantic Result Formats, but this extension hasn’t brought a new version yet.

Push

The last few days I’ve been working on an awesome new extension that’s meant to facilitate pushing page content from one wiki to one or more other wikis, called ‘Push’. It makes heavy use of the MediaWiki API, and takes advantage of the MW 1.17 Resource Loader when available (but is compatible with MW 1.16). You can use it on-page by clicking the ‘push’ tab, which will get you a list of targets to push to (specified in LocalSettings).

It also supports bulk operations via Special:Push, which allows you to select pages in a similar fashion to Special:Export, and then, after submitting, pushes the pages one by one in a nice ajax-y way.

I created this extension as WikiWorks consultant for Texas Instruments.

What’s next?

MediaWiki 1.17 has just branched, so hopefully it won’t take to long for it to get released.

Version 0.7 of both the Maps and Semantic Maps extensions is now available for download. This release is made after 3 beta’s and a release candidate, so should be stable.

The most notable new features in this release are tag extension support for all the Maps parser functions, more consistent error handling via Validator 0.4 and compatibility with the upcoming MediaWiki 1.17. Most changes are internal improvements, but there are also several bug fixes and minor new features.

Whats next?

There only big new feature on the roadmap for 0.7.x is the infamous images-as-layers one. This feature was supported by the now long obsolete Semantic Layers extension (example), and has been requested by dozens of people since the first releases of Maps. Next to fixing the bugs that show up, the focus will mainly be on adding new functionality. Feel free to propose new features, or help out creating them.

This release is notable for it being the first one in which I’m happy with the code-base as a whole. It took me a year to get here, but now I think the way the mapping extensions work is good and solid. This means you can now extend Maps and not be afraid the code will be incompatible in a few weeks due to changes. This also means that I’ll be focusing more on actual functionality rather then refactoring in future releases. I’ll be progressively building a little guide that explains how the extensions work from a developers perspective and how to extend them.

I might release another minor update in the 0.6.x series if any significant issues are found in 0.6.5. Further plans are finishing up a bunch of changes I’ve started to make in Validator, which I’ll probably release as 0.4 then, and to start working on Maps and Semantic Maps 0.7, which would aim at adding new features and improving existing ones. A likely new feature I’m particularly looking forward to implementing is several tag extensions that do the equivalent of the current parser functions added by Maps. The timetable for all this depends a lot on which other things I get cough up in (I’ll probably continue putting effort into the deployment stuff for my GSoC project) and what kind of funding will be available.

Downloads:

• Maps 0.6.5 [zip - 7z]
• Maps and Semantic Maps 0.6.5 [zip - 7z]

You can also view the release announcement at the documentation wiki.

I’ll start with the things that where not nice, basically the heat and the dormitories. It was 30+ °C practically the whole time, and the dormitory I stayed in, which was reserved by the event organizers for the attendees, was more like a sauna then a dormitory. It was located right next to a viaduct, so when opening the window at night, you’d get a lot of noise. On the last night the people from the event could not stay at that dormitory, and had to go somewhere else. Me and another attendee from Belgium got assigned some dormitory a few km away from the event. Saying this was a crappy one is an understatement. Furthermore, the payment was not arranged, so we had to pay for staying, while it should have been done by the event organizers as we paid for it?!! In any case, this amounted to me having less then 10 hours of sleep over 4 nights And they don’t have Club Mate in Gdansk o_O!!!

I think the event itself was great, with lots of interesting talks, awesome people and good food.

On the first day I missed the keynote since my flight arrived to late to see it, and only followed some non-technical talks, of which the most notable one was about Liquid Threads, the totally awesome extension by Andrew Garrett and now also some other people. I left early, skipping the deinner and following events, and went straight to the dormitory, to get some sleep (which failed >_>)

I gave 2 talks, one about my Google Summer of Code project, now titled Deployment, and one about Maps and Semantic Maps, both on the second day. The Deployment talk went pretty bad, as I was extremely tired. Hopefully I got the core message across of the importance of having a solid and user friendly way of deployment. Apologies to all the people I forgot to give credits to! Daniel Kinzler had some interesting security concerns which we discussed later on the day, together with Markus.

On the second day of the event there was a series of Semantic MediaWiki talks and workshops, kicked off by a presentation about the concept and what SMW has become over the past 5 years by Markus. This was followed by a workshop by Hans-Jörg Happel and Frank Dengler about Semantic Result Formats. Daniel Herzig gave a talk about AskTheWiki, and promising extension he is developing. There where two more SMW related talks focusing on the advantages of SMW and how it can be deployed. Later on the second day I gave my talk about Maps and Semantic Maps, preceded by one of Tim Alder, who presented the capabilities of the geo-related work on the toolserver. Luckily I was awake to some extend during this talk, and it went reasonably well IMO, although I had to rush through it, cause of to little time.

The second day was closed by the word premier of the movie “Truth in Numbers“, which aims at giving people an idea about the goals of the Wikimedia Foundation, and how it works. It shows arguments from both people involved or enthusiastic about the foundation, and those who think the world is going to explode if you go to Wikipedia for whatever reason. It will be interesting to see how this movie gets distributed and what effect it will have. I’m a little sad about how the creators choose to distribute it though, as it will require you to pay for it. It would be a lot more awesome if it was free with a big donate button. All the footage, which is a lot more then what’s seen in the movie itself includes, but not edited, will be released for free under a creative commons (or similar?) licence though.

The third day I found the most enjoyable as I did get some amount of sleep the night before. (In other words, I was awake enough again to program during the talks : ) The most interesting talk for me I followed here was definitely the one by Roan Kattouw about writing MediaWiki extensions, which mentioned some things I did not know about yet (such as build in MediaWiki support for Memcached o_O), as well as provided a nice overview of the things you need to keep in mind. I wish I had seen it a year earlier though, as I would have learned a lot more from it then, and not made a lot of the noted beginner mistakes. In the next session I attended several strategy sessions, which gave me a nice idea of what all the strategy fuss is about. The last session I attended was about Wikimedia credibility, including a talk by Maarten about how information (esp the incorrect) spreads to other media. The other talks in this session where similar, and although obviously none of the issues addresses are technical in nature, I found several of them rather amusing.

It was great to meet all the people involved with SEMANTIC MediaWiki in person, as well as a bunch of people I only knew from IRC and other online communication tools. Sadly enough Yaron Koren and Brion Vibber and several other people I’d like to have seen there could not be at the event.

On the way back to Belgium I tackled a lot of small design issues that have been present in Maps and Semantic Maps for months, resulting in one big refactoring commit for each extension, which I made as soon as I was back home. After that I went to sleep, and woke up 19 hours later O_o

Next years Wikimania will be in Haifa, Israel. I’m probably going, but seriously hope it won’t be so insanely warm there