BN's blog

I created a new MediaWiki extension titled “UK geocoding for Maps“, which I released yesterday.

The goal of this extension is to provide a way to display UK postcodes on Google Maps maps, by extending the Maps extension. Since obtaining geocoded postcodes for UK Postcodes via Google services is only possible using the Google AJAX search API, the standard geocoding infrastructure of Maps could not be used, and this extension should be seen as a sort of hack. Initial development for this extension was funded by Neill Mitchell for Prescient Software Ltd.

Version 0.1 is an experimental release, in which multiple issues are expected. I’m hoping to polish the whole extension a little more once I have some free time. Bug reports are welcome though.

You can get version 0.1 from SVN at

http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/UKGeocodingForMaps/

Yesterday I released Maps and Semantic Maps 0.5.4. This is a minor update which mainly brings additional stability and security to the 0.5.x branch. It contains several bug fixes, a few code improvements and some security fixes. Check out the Maps change log and Semantic Maps change log.

The only new functionality is that you can now define OpenLayers layers in your LocalSettings file, as well as layer groups. This is done by modifying the $egMapsOLAvailableLayers, $egMapsOLLayerGroups and $egMapsOLLayerDependencies arrays, which are defined in Maps_Settings.php.

Another noteworthy point is that Semantic Maps now contains all the geographical coordinate format handling that was part of Semantic MediaWiki earlier on. The code has been removed from SMW itself, and the SMW 1.5 release will be the first one without it. A nice side effect of this is that people still using Semantic Google Maps (one of the extensions Maps and Semantic Maps are based on, which is now obsolete (and really should not be used any more!)) will be forced to switch over to Maps and SM if they want to get the latest SMW.

Both extensions now use Validator 0.2.2, which is a minor update to 0.2.1, which was used in 0.5.3.

Downloads

• Maps 0.5.4 [zip, 7z, svn tag]
• Maps and Semantic Maps 0.5.4 [zip, 7z]
• Semantic Maps [svn tag]
• Validator 0.2.2 [zip, 7z, svn tag]

I’ve been working on the Wikimedia Storyboard extension for a week now, and like to provide the people who are interested in it with some more details about what the project goal is exactly, and what my current progress is.

The idea of the Wikimedia Storyboard is to develop a fundraising landing page on http://wikimediafoundation.org/ with moderated but up-to-date stories from users and donors explaining the role Wikimedia has played in their lives. The motivation of building such a landing page is that donor and supporter stories can help to convince and persuade people of the impact an organization is having. It’s essentially an extended, moderated version of the real-time contribution history.

Basically the extension can be divided into 3 separate parts:

The storyboard interface

The storyboard interface will contain several stories stacked vertically in a box that follows the “eternal load” paradigm, much like Google Reader. Each story will consist of a picture with some text next to it, and some “share” features for stuff like Facebook and Twitter. For people with a special storyreview permission there would also be a button allowing them to hide the stories from being displayed in the storyboard. This part of the UI will be implemented using the tag extension <storyboard/>. You can see a rough layout on the underneath mock-up (on the left).

Story submission interface

As the above image reveals, there will also be an interface where you can submit stories, that will be placed on the right of the storyboard itself, in a tab gadget, together with a donate page. Like the storyboard itself, this will be implemented using a tag extension: <storysubmission/>.

Story review interface

The story review interface will allow users that have the ’storyreview’ permission to review, edit, publish, and hide stories. Hidden stories could still be published later; they would simply be available through a separate queue. This interface will be implemented as a special page (Special:StoryReview), but will be transcludable, meaning it can be put into other pages.

My progress

Over the past few week, most of my dev time has gone to reviewing documentation about how to correctly create the desired features in the Storyboard extension. Things new for me include, tag extensions, special pages, database interaction, and permission management. I now also better understand some other stuff like how variables should be escaped properly, which I’ll be applying to Maps, Semantic Maps and Validator for their next release. I’ve set up the basic structure of the extension, as you can see on svn trunk, and am now working on creating the dynamic aspects of the interface using jQuery and jQuery UI.

Some stuff I’ll still have to review further at some point is how to make the stories searchable, how to best internationalize the stories and how I’ll implement the “share” features. I’m looking forward to getting more familiar with MW core though, and getting the hang of these things should not be to difficult with the awesome support I’m getting from the devs at the MW IRC.

Today I released Maps and Semantic Maps 0.5.3. Both releases mainly include bug-fixes, some of which quite important:

• Fixed JavaScript bug causing all OSM maps to fail. Pretty important yeah
• Fixed issue causing a fatal error when executing a query on Special:Ask with the “map” format.
• Fixed issue causing PHP warnings with the type and types parameters in the Yahoo! Maps form input.
• Fixed OpenLayers form input projection bug, causing the the coordinates to be wrongly interpreted. The amazing thing about this bug is that it has been present since somewhere between 0.3 and 0.3.3, without anyone reporting it. It was caused by an error in coordinate conversion between regular and spherical Mercator projections.
• Fixed marker display for the OpenLayers form inputs.

I also discovered an oversight in the hook system of Maps, which is kind of a dormant bug, that could have caused errors for certain implementations of extensions to Maps. The oversight was that a mapping service does not necessary support a sub feature, like the parser function display_point, of a feature, like the parser functions. A check is now in place, together with a defaulting mechanism that also allows you to set the default per sub feature. I’m planning on changing this to a more recursive algorithm in the future though.

A last change is the adding of initial Google Maps v3 support. It’s only implemented for the display_map parser function, and has a rather limited set of customisation parameters at the moment, not to forget a whole load of issues. My conclusion from trying to implement this is that the GMaps 3 API is definitely not in it’s final state yet, since it’s lacking vital methods needed for basic functionality. The support for GMaps v3 is therefore sort of an alpha release, and won’t be put on the documentation page until it’s more stable and complete.

Downloads

• Maps and Semantic Maps 0.5.3 (zip)
• Maps and Semantic Maps 0.5.3 (7z)
• Maps 0.5.3 SVN tag
• Semantic Maps 0.5.3 SVN tag

As you can see, I changed the location of the releases from my forums to 2 brand new Google Code projects [ Maps, Semantic Maps ].

This has been a big item on the to-do list for new Maps features for a few months now, but I finally managed to make some time for it, and just checked in the last code into the MediaWiki SVN repo

Let’s start at the beginning: what is static map support? Supporting static maps means that you are able to display maps as plain images instead of via a JavaScript (or other non-html language, for example flash) mapping API. This is very important since it makes viewing the maps possible for people who are browsing the web with a browser that does not support JavaScript (or this other language), or simply have it disabled for some reason. This actually made this feature a requirement for getting Maps onto Wikipedia and other Wikimedia Foundation wiki’s. Another reason to use such a feature is that it can significantly reduce the load on the servers from which the map tiles are fetched, since the plain images can be cached.

So how does it work? Maps now accepts a new parameter called ’static’ that accepts the values ‘yes’ and ‘no’. When set to no (or not provided, assuming the default is unchanged, so on ‘no’), you’ll get the same map as you always did, with no changes whatsoever. When the values is ‘yes’, you’ll get a plain .png image fetched from a WF server. If you have JavaScript enabled, and click on the image, it’ll be replaced with the dynamic map.

Maps 0.5 will support this for the display_map parser function when using the OSM service. It’s possible I’ll also add support for display_point(s). Note that my work is just a link in a big chain here. Most effort to make this functionality possible is done by the guys doing the mapping server side work. Also, a lot of my code Maps is using for displaying static maps is heavily based on code from the SlippyMap extension by Ævar Arnfjörð Bjarmason and others.

Since there are no demo’s available anywhere yet, here you have some screenshots:

A static map:

The dynamic map you get after clicking the static map:

And yes, the zoom level is off, this is a bug that still needs to be fixed

It’s been almost 2 months since Maps version 0.3.4 was released. Although I did waste a lot of my time at school, I managed to get quite some important work done in that period. Version 0.4 mainly addressed new functionality, and some thorough refactoring for Maps, and only introduced 2 bug fixes for Semantic Maps. So lets have a look at everything that changed

New features

• I added a display_map parser function, that’s obviously meant to display maps. The code handling this parser function is optimized to display maps, and does not hold into account any possible markers, ect. The main reason for adding this function is that it’ll be used for the Wikimedia Foundation implementation of Maps.
• Smart geocoding support. Maps is now able to determine if a value is a set of coordinates, or an address. This enables automatic geocoding of addresses, making the specification of the value type unneeded, and allows you to mix coordinates and addresses in location lists. This functionality is implemented for display_map’s default parameter, the one of display_point(s), and display_point(s) centre property.
• OpenStreetMap service. This service uses OpenLayers to display OpenStreetMap tiles. It’s optimized for OSM, and does not allow you to display other tile sources with it. This is another of the requirements for WF usage that’s now been completed. This service is only implemented in Maps ATM, and will be added to Semantic Maps in version 0.5.
• Support for DM (Decimal Minutes) and DD (Decimal Degrees) coordinate notations. Someone requested these to be added. Several other variants, such as GPS coordinates, might be put into Maps in the future.
• Added a setting specifying the minimum and maximum width and height of maps. When a value falls out of this range, it’ll be changed to the nearest allowed value. This is to prevent post-stamp or wallpaper maps, and is inspired by the similar functionality in the SlippyMap extension.
• Parsing of marker-specific title and label values. I actually marked the lack of this feature a bug a while ago, which I changed at the point where I remembered I simply did not add such a thing yet, and only map-wide title and label value’s got parsed.

Refactoring (“under the hood” changes)

• A new hook system for the parser functions, allowing the adding or removing of additional parser function support. Each mapping service will now have one class for handling on specific parser function. Again, the main reason behind this change is that it’s required for the WF usage of Maps, where at the start, only display_map will be used. Another great advantage is that you can now very easily add new parser functions without having to mess up the core extension code. An example of such a function is display_route, which might be added at some point to maps.
• Change the geocoding functionality into a true feature hook element, so it can be easily removed. This is not easy to do, since the geocoding functionality is currently used at various points by the parser functions. So before I can start making it a feature hook element, the parser function code will have to be restructured.
• Create service hook for the geocoding feature – separate from the mapping services hook. This will allow adding and removing support for a specific geocoding service.
• Since Maps is now able to distinguish coordinates from addresses, the display_address and display_addresses parser functions have been removed. For backward compatibility, calls to them will be routed to display_point(s). This will of course eventually be removed.
• Removed a redundant path variable. One of the messy remains of the initial Maps release, which even caused problems on some installations.

Bug fixes

• Fixed a mayor bug in the initialization method causing hook code to get executed at a probably wrong moment. This bug can be the cause of some weird problems that surfaced since 0.3.3.
• Fixed a rather annoying issue with size of popups in Google Maps. They did not stretch far enough vertically for large contents. I’m happy I got this fixed now, since I’ve been aware of it for quite a while now, but simply could not find a solution. Yaron pointed me to a helpful blog where a certain post set me on the right track. The solution was ridiculously simple, and one line (in the JS code) long.
• Fixed some problems with the display of the mapping result formats on the Special:Ask page of Semantic MediaWiki. I’ve done this by adding an aliasing system for result formats to SMW itself. So to be able to benefit from this, you need to have that version of SMW, or later. Maps has a build in check to see if the aliasing functionality of SMW is present, and if not, uses the old, but a lot less efficient, solution.

For the change log listing, see Maps version history and Semantic Maps version history.

The documentation has been completely updated for both extensions. I only have to change the demo’s on my demo wiki to reflect the new syntaxis of the parser functions. After that I can get to work on the to-do’s for version 0.5, which will include the long awaited static map support

Did I mention Semantic Maps now supports over 50 languages? :p

Downloads

• Maps 0.4 (zip, 7z)
• Semantic Maps 0.4 (zip, 7z)
• Maps and Semantic Maps 0.4 (zip, 7z)
• Maps 0.4 SVN tag
• Semantic Maps 0.4 SVN tag

Yesterday evening, I released the 0.3 versions of both Maps and Semantic Maps. The documentation has been given a big overhaul, and now contains up to date info about the latest release in a more easy to comprehend format.

This are my blog posts about the changes in this new version:

• Structural changes for Maps and SM 0.3
• New features in Maps and SM 0.3
• Final changes for Maps and SM 0.3

A list of all changes can be found on the version history pages of Maps and Semantic Maps.

This is the last version I’m creating as part of my Google Summer of Code project, since it ends next Monday. It’s great that so much work has been done, and the extension turned out so well. But at the same time I’m a little sad GSoC is about to end. I had a wonderful time writing Maps and Semantic Maps, and learned a lot of new things, including how open source organizations work, some application architectures and of-course some PHP things I didn’t know yet. Anyway, I’m planning to apply for GSoC 2010 as soon as that’s possible, although I’m not sure for what project and which open source organization.

I’m planning to still get some work done for 0.4, which has currently only custom layer functionality for OpenLayers scheduled. This is a very important to-do though, since it’s in high demand.

Maps and Semantic Maps are almost ready for release now, with all but some small issues resolved. As a follow up to my 2 previous posts about both the structural changes and new feature in maps, this one will address the things changed or added since then.

Configurable map controls

Controls on both Yahoo! Maps and Google Maps map can now be configured by the user with the controls parameter. Yahoo! Maps maps already have this option for a limited set of controls since version 0.2, but the amount of available controls has now been expanded to what the Yahoo! Maps API offers. For Google Maps the change is significantly larger, since a lot of new controls can now be added. These included an overview map, a scale line, a drop down menu for map types, an automated reverse geocoding location determiner and more. All new types and their names will be added to the documentation after the release of 0.3.

Separate meta data for each point

I already described this new feature in one of my previous posts. It has now been completed, and has been extended with an icon parameter, which can be used to display a custom marker. The custom marker functionality is now also available in Semantic Maps, and is meant to be used in Semantic Compound queries, to for example display hamburgers as icons for shops of type ‘fastfood’, and t-shirts for shops of type ‘clothing’.

User friendly error reporting

Until now, no thought was given to what happened when the user provided an address that could not be geocoded. This actually pretty serious shortcoming has now been fixed. When an user provides one address, and it can’t be geocoded, an error message explaining that it couldn’t be geocoded is displayed instead of a map. The same thing will happen when the user provides multiple addresses that can’t be geocoded. When only a few of multiple addresses can’t be geocoded, the map will be displayed with the available coordinates, together with an error message saying some of the results could not be displayed and a list of these addresses. Similarly, for Semantic Maps, when a query returns no results, nothing will be displayed, instead of an empty map.

New OpenLayers layers

A whole list of OpenLayers base layers have been added. These include the satellite, street and hybrid views for Yahoo! Maps and Bing Maps, but also finally the OpenStreetMap layers. The problems I had with the different map projections between OSM and the other services has finally been resolved.

Like promised in my previous post about Maps and Semantic Maps 0.3, I’ll give you an overview of the most important new features in this new release.

Multi location parser functions

Two completely new parser functions have been added that allow the displaying of multiple points on a map. To avoid confusion, this is a feature in Maps, and has nothing to do with the semantic coordinate aggregation of Semantic Maps, which obviously already has multi coordinate capabilities. The new parser functions are display_points and display_addresses, acting as multi coordinate variants of display_point and display_address, respectively. This feature adds endless new usage options for Maps, from marking the locations you’ve been on holiday to, to a list of restaurants in a city. It will also be extremely useful to use together with the upcoming custom base layers and overlays feature for OpenLayers. An example of such usage is displaying markers with some pop-up contents on an anatomy chart. Note that a form input will likely be added to Semantic Maps to simplify such a task, by making the need to manually find and enter all coordinates obsolete.

Separate title and label for each point

This feature could actually be viewed as part of the multi location parser functions, but I like to keep them separate. Since 0.2, Maps allows you to optionally display a title and label together with the marker representing the provided coordinates or address. Logically, this should be extended to encompass multiple locations when you can put more then one on a single map. This feature has not been implemented yet, cause of some uncertainty about the correct wiki syntax. The current idea is to use something like #display_points:points=55.7557860, 37.6176330~title~label; 1,1~title; 12,34. Such a syntax’s would not allow any ‘;’ or ‘~’ to be displayed into the pop-up.

Configurable map types

In Maps 0.2, an improvement causing the ‘physical’ map type, for Google Maps, to be displayed in the map type control when this map type was set as default was made. This caused me to wonder why the whole control was not made configurable, so that users could specify the map types they want, and the order in which they want them. This is exactly what I’ve done in Maps 0.3, for both Google Maps and Yahoo! Maps. The user can now set the map types present in the map type control with the types parameter. When not set, types will be set to the (new) setting holding the default types for the relevant service. The parameter type does still precisely the same, with the addition that the provided value will be added to types when it’s not present in it yet.

While adding this new feature, I also had a close look at the available map types in the latest (sub)version of the Google Maps v2 API. It turned out to be really easy to add support for moon, Mars and sky maps (all 2D). The underneath screenshot shows a map with all the available map types for Google Maps in Maps 0.3. It also demonstrates the new multi location functionality.

The two interesting parameters in the wiki code that is responsible for this map are:

• addresses=Moscow, Russia; New York; London – This is the replacement of address in display_address.
• types=normal,satellite,hybrid,physical,earth,moon,mars,mars-elevation,mars-infrared,sky – The new types parameter that allows users to choose the available map types.

This change now makes both Google Maps and Yahoo! Maps usage more similar to the one of OpenLayers, with the difference that with OpenLayers, types is called layers, and type is called baselayer (althoguh the baselayer is not activated cause of some problems with it).

More to come

Several more new features will be added, and I still have some refactoring to-do’s on my list to tackle before the 0.3 release. I’ll post about those issues as soon as I have more news about them. The new release is coming closer and closer – I estimate it’ll be there in less then a week

Here you have my latest committed changes for both Maps and Semantic Maps.

Today I finished refactoring the parser function handling classes (in Maps) for each service and also fixed the php/js logic issue. After that I took care of a variety of to-do’s, including the centre parameter, support for all OpenLayers controls, the display of title’s and labels for markers and the adding of the Google Maps ‘physical’ map type to the map type selector control when it’s set. Here you have a nice screenshot demonstrating the last 2 features:

{{
#display_address:
service=googlemaps
|address=Moscow, Russia
|width=800
|Height=500
|type=physical
|title=Moscow (Russia)
|label=Moscow is the capital and the largest city of Russia. It is also the largest metropolitan area in Europe.
}}

There are only 2 to-do’s remaining before I can release version 0.2 of both extensions: multi-geocoder support and OSM base layers for OpenLayers maps. Both are proving to be quite a challenge though. I was planning to create a central system for the geocoders, both the ones used in parser functions, and in form inputs, but this is proving to be extremely difficult to achieve. I therefore will probably just go for separate systems (unless I find an easy way to do the original plans).