Jeroens blog

I’ve been working on Maps and Semantic Maps 0.6, the next big, awesome, release of both MediaWiki extensions, for over a month now. I also released an early alpha on the 3th of April. All planned new features have been implemented, and known bugs from 0.5.x have been fixed. So you might wonder why 0.6 is still not out.

In response to the possibility of someone doing work on Maps and Semantic Maps during the upcoming Google Summer of Code, I took a critical look at the current structure, holding into account all the things I learned over the last two month while working on Storyboard. I came to the conclusion that a bunch of things ought to be handled in a cleaner fashion, and started to refactor this code. The most difficult part here is changing how the display_map and display_point(s) parser functions handle their mapping service parameter and validate the provided location(s). I’ve been wanting to change this since 0.4, but didn’t since it’s rather tricky to do. I decided to finally get this done now, and have done most of the work. To complete these changes, I’ll have to make some rather complex modifications to Validator, which can take a while to complete. That’s the last thing that needs to be done before the 0.6 release though

I estimate this should be done in approximately 2 weeks, maybe sooner. After that I’m planning to release at least one RC, to ensure stability and complete awesomeness, cause really a lot has been changed. I figure about three quarters of the codebase has been changed in one way or another!

After this release a lot of cool new features can be added, which I might do before the GSoC coding period starts. There are also a few people who wanted to do a mapping project for GSoC at the WMF, but did not get accepted, which are planning to do some effort here after all, which is totally awesome

Like last year, I’ve been accepted for GSoC 2010 – yay!

The Google Summer of Code (GSoC) is an annual program, first held from May to August 2005, in which Google awards stipends to hundreds of students who successfully complete a requested free software / open-source coding project during the summer.

My project is creating an awesome extension management platform for MediaWiki, facilitating the installation, updating, removal and configuration of extensions. I’ll get mentored by Brion Vibber, which is probably the best known MediaWiki developer there is. The underneath paragraphs are out-takes of my actual proposal.

Project summary

The goal of this project is to create an administration panel from where wiki administrators can update, install and remove extensions. A second goal would be to allow management of the installed extensions.

A panel where wiki administrators can install, update and remove extensions would have huge benefits. First of all, people would not have to manually download an extension and put an includes in LocalSettings, neither would they need to worry about compatibility and dependencies. Hitting an update button also takes considerably less time then doing the whole download routine again, and will cause people to run more up to date extensions. Another important advantage is that people will get extensions recommended, and can easily browse them. This way people will find extensions that do something they wanted but did not know about, and in general have extensions that better suit their needs. A third advantage is that extension developers won’t need to do extreme efforts to let people know there is a new version (and probably still only reach part of the relevant public). This is inspired on the way Word-press does things.

The second goal of this project is to add setting management for individual extensions. Currently extension settings are managed via LocalSettings. The aim here is to completely remove the need of editing any file directly by storing the configuration the the MediaWiki database, and creating a GUI to modify these settings. This would involve creating API modules so extensions can add and update their own settings. Work on this will only be started after the first goal is completed, and is seen as an “if time permits to-do”.

A third, also optional, goal would be to create a management interface for the wiki’s configuration itself. This is very similar to the second goal, and should be kept in mind while creating the management for extension settings. I do not expect to complete this to-do during GSoC, but want to provide the foundations for it, so this can be completed after the project itself is finished.

Deliverables

Note: the beneath list is a guideline only.

Things the administration panel should be capable of:

Required deliverables

• Automatic checking for extension updates (and updates to MW itself)

• Download and install option for extension updates

• Download and install option for extensions that have not yet been installed

• Removal of installed extensions

• Dependency and compatibility checking (both for installation, removal and updating)

• Extension browsing
  • Recommendation of similar extensions
  • Showing a list of most used and most recommended extensions
  • Finding extensions by category or keywords

• Enabling and disabling of installed extensions (so without removing them)

If time permits

• Management of settings specific to each extension.

• Management of MediaWiki configuration.

Project schedule

I’m for a loose schedule, since I believe this is the most efficient. I have no doubt that the to-do list will change a lot during the project, items will be changed, moved, multiple new ones will be added, and some might be removed. A fixes schedule would take away flexibility and stand in the way of efficiency. One of the main reasons to have a schedule is to ensure the student does not take the project to lightly, end ends up making insufficient progress. I like to believe I have clearly demonstrated that I will put considerable effort in such projects, even without any schedule, during last yeas GSoC, and with all the commits I’ve made since then.

This list contains some loose planning without any dates:

• Discuss the best way to structure the platform with mentor, and other relevant people.

• Investigate how similar functionality works with other software, and which aspects of these implementations can be used.

• Get the requited knowledge of MW to be able to create the platform in an efficient and modular fashion.

• Create the panel with the most basic features, then release and document it.

• Add the other features, possibly spread over several releases.

• (Start on the extension setting management functionality.)

• (Start on the MediaWiki configuration management functionality.)

• (Add other awesome things to the platform.)

• Take over the worlds and make everyone use this platform.

My current planning

I have to finish up the work I’m currently doing for the Wikimedia Foundation before I can fully start on this, and also like to release Maps and Semantic Maps 0.6 before then. This will probably be in two to tree weeks from now.

Also see the Wikimedia tech blog post which links to the other accepted projects.

Yay! I finally got my Google Summer of Code T-shirt!

With my T-shirt, I also got my GSoC 2009 certificate, which marks the end of any involvement with GSoC for me (see original post). That is, until next year. When I reflect back to what I learned and achieved during GSoC, I’m more then happy with it. I’m planning to do another (awesome) Wikimedia Foundation project next year, if I manage to get accepted again. This (awesome) project will be more focused on changes to MW’s core, and so provide me with an (awesome) opportunity to learn more about the inner working of MW, which I’m currently still unfamiliar with. I’m not going to make my exact plans public yet, cause it’s possible the things I want to improve will already be taken care off, and I don’t want everyone to go rip-off my idea I’ve already put my name on the (awesome) participants list for MW, and was FIRST (awesome!) since I created the (awesome) page.

Me and two fellow Google Summer of Code students, one from the Netherlands, and one from Belgium, have decided to hold a meeting in Brussels. This will be a great opportunity to discuss each others work, and future plans.

There are over 10 other Dutch speaking GSoC students we haven’t heard of, they are of course also welcome. I’d also like to invite people who are interested in GSoC. This includes people who are simply curious and want to know what it is, but also those who are considering to participate as either student or mentor in a future GSoC. Note that non Ducth people are also welcome! Feel free to send me an email, or one on the GSoC student list, if you are interested, or simply comment on this blog post.

The meeting will take place September 29th, at 19:30, in Hacker Space Brussels (HSB). HSB is located at PRINSES ELISABETHLAAN 46, 1030 Schaarbeek, Brussels.

Today I finally announced the new releases of my extensions on a few wiki mailing lists. I also gave the Semantic Maps documentation another overhaul, and created a manual on how to extend maps.

This manual targets developers who want to add support for a mapping service to one of the features Maps and Semantic Maps support: parser functions, query printers and form inputs. I still have to create the manual for the later two features though. Currently, the subjects of adding a new mapping service, adding parser function support and creating new geocoder classes are covered.

I hope to get the Semantic Maps part done by tomorrow, as a nice conclusion to my GSoC project.

Yesterday evening, I released the 0.3 versions of both Maps and Semantic Maps. The documentation has been given a big overhaul, and now contains up to date info about the latest release in a more easy to comprehend format.

This are my blog posts about the changes in this new version:

• Structural changes for Maps and SM 0.3
• New features in Maps and SM 0.3
• Final changes for Maps and SM 0.3

A list of all changes can be found on the version history pages of Maps and Semantic Maps.

This is the last version I’m creating as part of my Google Summer of Code project, since it ends next Monday. It’s great that so much work has been done, and the extension turned out so well. But at the same time I’m a little sad GSoC is about to end. I had a wonderful time writing Maps and Semantic Maps, and learned a lot of new things, including how open source organizations work, some application architectures and of-course some PHP things I didn’t know yet. Anyway, I’m planning to apply for GSoC 2010 as soon as that’s possible, although I’m not sure for what project and which open source organization.

I’m planning to still get some work done for 0.4, which has currently only custom layer functionality for OpenLayers scheduled. This is a very important to-do though, since it’s in high demand.

Maps and Semantic Maps are almost ready for release now, with all but some small issues resolved. As a follow up to my 2 previous posts about both the structural changes and new feature in maps, this one will address the things changed or added since then.

Configurable map controls

Controls on both Yahoo! Maps and Google Maps map can now be configured by the user with the controls parameter. Yahoo! Maps maps already have this option for a limited set of controls since version 0.2, but the amount of available controls has now been expanded to what the Yahoo! Maps API offers. For Google Maps the change is significantly larger, since a lot of new controls can now be added. These included an overview map, a scale line, a drop down menu for map types, an automated reverse geocoding location determiner and more. All new types and their names will be added to the documentation after the release of 0.3.

Separate meta data for each point

I already described this new feature in one of my previous posts. It has now been completed, and has been extended with an icon parameter, which can be used to display a custom marker. The custom marker functionality is now also available in Semantic Maps, and is meant to be used in Semantic Compound queries, to for example display hamburgers as icons for shops of type ‘fastfood’, and t-shirts for shops of type ‘clothing’.

User friendly error reporting

Until now, no thought was given to what happened when the user provided an address that could not be geocoded. This actually pretty serious shortcoming has now been fixed. When an user provides one address, and it can’t be geocoded, an error message explaining that it couldn’t be geocoded is displayed instead of a map. The same thing will happen when the user provides multiple addresses that can’t be geocoded. When only a few of multiple addresses can’t be geocoded, the map will be displayed with the available coordinates, together with an error message saying some of the results could not be displayed and a list of these addresses. Similarly, for Semantic Maps, when a query returns no results, nothing will be displayed, instead of an empty map.

New OpenLayers layers

A whole list of OpenLayers base layers have been added. These include the satellite, street and hybrid views for Yahoo! Maps and Bing Maps, but also finally the OpenStreetMap layers. The problems I had with the different map projections between OSM and the other services has finally been resolved.

Like promised in my previous post about Maps and Semantic Maps 0.3, I’ll give you an overview of the most important new features in this new release.

Multi location parser functions

Two completely new parser functions have been added that allow the displaying of multiple points on a map. To avoid confusion, this is a feature in Maps, and has nothing to do with the semantic coordinate aggregation of Semantic Maps, which obviously already has multi coordinate capabilities. The new parser functions are display_points and display_addresses, acting as multi coordinate variants of display_point and display_address, respectively. This feature adds endless new usage options for Maps, from marking the locations you’ve been on holiday to, to a list of restaurants in a city. It will also be extremely useful to use together with the upcoming custom base layers and overlays feature for OpenLayers. An example of such usage is displaying markers with some pop-up contents on an anatomy chart. Note that a form input will likely be added to Semantic Maps to simplify such a task, by making the need to manually find and enter all coordinates obsolete.

Separate title and label for each point

This feature could actually be viewed as part of the multi location parser functions, but I like to keep them separate. Since 0.2, Maps allows you to optionally display a title and label together with the marker representing the provided coordinates or address. Logically, this should be extended to encompass multiple locations when you can put more then one on a single map. This feature has not been implemented yet, cause of some uncertainty about the correct wiki syntax. The current idea is to use something like #display_points:points=55.7557860, 37.6176330~title~label; 1,1~title; 12,34. Such a syntax’s would not allow any ‘;’ or ‘~’ to be displayed into the pop-up.

Configurable map types

In Maps 0.2, an improvement causing the ‘physical’ map type, for Google Maps, to be displayed in the map type control when this map type was set as default was made. This caused me to wonder why the whole control was not made configurable, so that users could specify the map types they want, and the order in which they want them. This is exactly what I’ve done in Maps 0.3, for both Google Maps and Yahoo! Maps. The user can now set the map types present in the map type control with the types parameter. When not set, types will be set to the (new) setting holding the default types for the relevant service. The parameter type does still precisely the same, with the addition that the provided value will be added to types when it’s not present in it yet.

While adding this new feature, I also had a close look at the available map types in the latest (sub)version of the Google Maps v2 API. It turned out to be really easy to add support for moon, Mars and sky maps (all 2D). The underneath screenshot shows a map with all the available map types for Google Maps in Maps 0.3. It also demonstrates the new multi location functionality.

The two interesting parameters in the wiki code that is responsible for this map are:

• addresses=Moscow, Russia; New York; London – This is the replacement of address in display_address.
• types=normal,satellite,hybrid,physical,earth,moon,mars,mars-elevation,mars-infrared,sky – The new types parameter that allows users to choose the available map types.

This change now makes both Google Maps and Yahoo! Maps usage more similar to the one of OpenLayers, with the difference that with OpenLayers, types is called layers, and type is called baselayer (althoguh the baselayer is not activated cause of some problems with it).

More to come

Several more new features will be added, and I still have some refactoring to-do’s on my list to tackle before the 0.3 release. I’ll post about those issues as soon as I have more news about them. The new release is coming closer and closer – I estimate it’ll be there in less then a week

Here you have my latest committed changes for both Maps and Semantic Maps.

Just like version 0.2 of both Maps and Semantic Maps, version 0.3 will feature a variety of large structural changes, aimed at increasing the performance of the extensions, making it possible to add new features, but most of all, make it more modular, to allow people to easily extend them. Here you have a list of the mayor changes that have been completed so far, with some explanation of the advantages they bring.

• Base class that bundles common functionality between MapsBaseMap, SMMapPrinter and SMFormInput, and gets inherited by those classes. This class has been named MapsMapFeature. This is a great step towards the complete centralization of all common code, but sadly enough was not implementable the whole way: MapsMapFeature cannot be inherited by SMMapPrinter, since this class already inherits the SMW QP. Yaron suggested to use a ‘has-a’ implementation, witch means creating an instance of one of the classes you want to inherit as a class field, instead of the ‘is-a’ which is the regular inheritance. Although this is doable in theory, it will bring with it some other problems, so will have to wait till another version.

• Separation between the common and specific parameters, and their default values. Each mapping service now holds an array containing the extra parameters they add (so the allowed ones), and another array containing their default values. Note that both arrays are merged with the common arrays at runtime, and will override similarity named items. This allows to change the default value of a common property, or change the aliases.

• Yes, indeed, that’s right, aliases! Property names now have aliases. This means you can add several alternative ways to name the same parameter, for instance, you can make so that ‘auto zoom’ and ‘auto-zoom’ will do excellently the same as the main parameter ‘autozoom’. This is particularly handy for parameters such as ‘centre’ (British spelling) and ‘center’ (American spelling).

• Hook like system for mapping services. This is without a doubt the most important change since 0.1. All hard-coded references to mapping services in the core code have been removed. A service is now added by one multi dimensional array in Maps.php (note that this can also be done in the initialization file of another extension!), which holds the name of the parser functions class and it’s location, the aliases for the service name (feature added in 0.2), and their allowed specific parameters and their aliases. This architecture will allow other people to create their own mapping extension using the Maps (and Semantic Maps) ‘API’, which enables them to focus on the mapping service specific code rather then extension related code, or common mapping code. There are numerous more advantages, like the fact that their ‘extension’ will follow some general norms, which will make it easier for uses to use (and install!) and for developers to work on, and the fact that their code will also benefit from improvements made to the core code of Maps and Semantic Maps. Once 0.3 is release, I’m going to create a new section in both the Maps and Semantic Maps documentation, which will hold a list of why people should consider choosing that approach, as opposed to creating their own little mapping extension, a list of the advantages, and documentation on how to effectively do it.

The above improvements have been made in Maps, but still have to be implemented in Semantic Maps, which is now on the top of my to-do list. I’m also busy with the adding of new functionality, and got some great results so far, but I’ll post about those when more of the work for 0.3 is done.

I’ve been trying to get the hang of how SVN works, and to be able to commit to the repository on mediawiki.org for a few weeks now, and finally succeeded – YAY!

The tools I’m now using are the PuTTY applications (this nice U3 app package), TortoiseSVN and Zend Studio. Subversion allows you to do a whole variety of nice things, and I probably won’t be able to grasp how I could ever have worked without it a few months from now.

I had a lot of problems with getting all settings right, mainly cause of some tutorials that just omitted some important step, or where unclear about some stuff. Since I had never worked with SVN, PuTTY or YotroiseSVN before, I did not notice that. This tutorial helped me out though.