BN's blog

Today I released Validator version 0.2, on which I’ve been working the last 2 days. It features massive rewriting to make it more flexible, and has some added functionality. Let’s have a look at what changed.

The most important change is, without any doubt, the new list support. Validator 0.1 had a list type, which allowed you to have enumerations of values and do crude validation on them. This version now supports lists of a type, instead of seeing list as a type. So you can now create lists of strings, lists of integers, and even lists of custom types you add. This new approach also allows per-item-validation and per-item-defaulting. This means you can set an in_array criteria, which will then be enforced for every value in your list. Closely related to this new form of list support are the new list criteria, which allow you to validate lists as a whole. At the moment the only 2 build in list criteria are item_count and unique_items, but like for regular criteria, you can hook into Validator and add your own.

Another important change are the output formats. Output formats allow you to specify additional formatting that needs to be done with the parameter value, before it is retrieved from Validator. There are currently 6 build in types, which are array, list, boolean, boolstr, string and unique_items, but again you can hook into this list via Validator. The awesome thing about output formats is that it greatly reduces the mess you otherwise have with converting your parameters from user input to true data structures. You can even specify multiple output formats, which will then do their formatting one by one.

Other things brought by Validator 0.2 include some new criteria (is_boolean, has_length and regex), a new error level, Validator_ERRORS_MINIMAL, new parameter types (boolean, number and char) and support for Validator_ERRORS_WARN in ValidatorManager.

Equally important as the changes made is that the documentation has been completely updated, to give in-depth cover of how Validator works, and how you should use it.

Both Maps and Semantic Maps 0.5.1 use Validator 0.2, allowing them to throw away a lot of repetitive, dumb manipulation, code that has been their since the initial versions of those extensions

Downloads:

• Validator 0.2 (zip)
• Validator 0.2 (7z)
• Validator 0.2 SVN tag

Earlier today, versions 0.5 of Maps and Semantic Maps where released. Some mayor new features where added, and a whole bunch of things have been refactored. I also did some effort to improve the documentation by adding some screencasts and revising the developer docs. Version 0.4.2 proved to be pretty stable, since only 2 bugs have been found and fixed.

Let’s have a look at the new, awesome, functionality:

• Strict parameter validation which allows you to get specific errors or warnings when entering invalid values or parameters. This functionality is obtained by relying on the Validator extension, which saw it’s first release today. This makes Maps dependent on Validator. Every distribution of Maps does include Validator, and Maps will automatically load Validator when it’s not loaded yet, so no extra installation work is required (unless you get the code from SVN).

• Static map support makes you able to display maps as plain images instead of via a JavaScript (or other non-html language, for example flash) mapping API. This is very important since it makes viewing the maps possible for people who are browsing the web with a browser that does not support JavaScript (or this other language), or simply have it disabled for some reason.

• Added a query printer that handles the osm result format.

• Added support for GUI parameter selection of Yaron’s new version of SMW’s Special:Ask page to the query printers.

• Added smart ‘autopanzoom’ like control for Google Maps and Yahoo! Maps.

• Added internationalization to the OSM service, and an extra parameter to define per-map languages.

The list of all things that have been refactored is rather long, so I’ll only cover the most interesting things here:

• Complete rewrite of the parameter handling. To make Maps work with Validator, this was required. The result is that Maps does not have a whole mess of specific validation and defaulting functions any more, since that’s now handled by Validator.

• Moved the geographical coordinate data type handling from Semantic MediaWiki to Semantic Maps. This will make the SMW codebase smaller by moving the geographical coordinate data type out, which is logical, since if you use coordinates, you very likely also want maps.

• Added code to unload any services from the service hook that are not present in the list of allowed services. This ensures they don’t get initialized, and makes any check to see if the service is one of the allowed ones further on unneeded.

As for documentation, I created 2 screencasts, both covering a different aspect of Maps. This way people can learn how to work with Maps in a more interactive way then just reading the documentation.

• Screencast: display_map

• Screencast: display_point

The developer documentation on how you can extend Maps using it’s hooks has been completely rewritten. This was needed since the previous version was created for Maps 0.3.3, since which a lot has been changed to the hook systems of Maps.

For a complete list of changes, see the Maps change log, and Semantic Maps change log.

Downloads:

• Maps 0.5 (includes Validator 0.1) [zip, 7z]
• Semantic Maps 0.5 [zip, 7z]
• Maps and Semantic Maps 0.5 (includes Validator 0.1) [zip, 7z]
• Maps 0.5 SVN tag
• Semantic Maps 0.5 SVN tag

One of the big new features in Maps 0.5 will be strict parameter validation. This means Maps will allow you to get specific errors or warnings when entering invalid values or parameters.

The setting determining the strictness of the validation, which can be changes in your LocalSettings file, currently accepts 4 values:

• Maps_ERRORS_NONE : Maps will not show any errors, and make the best of the imput it got.
• Maps_ERRORS_WARN : Maps will make the best of the imput it got, but will show warnings for omitted coordinates.
• Maps_ERRORS_SHOW : Maps will make the best of the imput it got, but will show a list of all errors.
• Maps_ERRORS_STRICT: Maps will only show a map when there are no errors, if there are, a list of them will be shown.

The underneath example demonstrates an error list that can be generated when the validation level is on Maps_ERRORS_SHOW or Maps_ERRORS_STRICT. In case of the former, it’ll be shown below the map,while in case of the later, it’ll be shown instead of any map. The error messages are of course fully internationalized.

The validation is done via a new class dedicated to parameter validation. To be able to validate anything, you need to feed it two things: an associative array containing the raw parameter names and their values, and a somewhat more complex, nested, array containing the allowed parameter and their meta data, such as aliases and default values. The class also provides a hook for validation types, allowing you to do specific or complex validation that is not build in. The handling of the different strictness levels and generation of the actual error messages is done by another class that uses the first to validate and get the errors. Both classes do not contain any Maps specific code, so can be used to validate the parameter of any parser function. I’m planning to split this code, after it has reached a beta level, into a separate extension, that will probably be named “Validator”. This extension will be bundled with Maps, and will not any additional steps to the installation process.

Together with implementing this new feature, I did a big overhaul of the parameter handling in Maps and Semantic Maps. Instead of the two level system, containing general parameters, and service specific parameters, that was used in Maps, there now is a four level system. The first level are the general parameters, shared by everything. These include things like width, height and zoom. Feature specific parameters make up the second level, while the third one holds service specific parameters. The last level are the parameters specific to a combination of service and feature. Maps goes through these levels, starting with the upper one, and overriding it with the following. This allows more specific behaviour and is required to be able to validate the parameters in some instances.

The changes I made to Maps and Semantic Maps during these rewrites are responsible for what are probably my biggest commits to both extensions yet.

• Changes to Maps (rev 59531)
• Changes to Semantic Maps (rev 59532)

Due to the extend of changes I made, and the lack of thorough tests yet, I expect multiple issues with this code, including several severe ones, so I advice against using the latest SVN code for the moment, except for testing purposes of course. I hope to have the code refined and bug hunted in the coming week, so I can put it in a new extension and release it. During this period I’ll also start working on the other new features planned for 0.5, so you can expect more news on this soon.

It’s been almost 2 months since Maps version 0.3.4 was released. Although I did waste a lot of my time at school, I managed to get quite some important work done in that period. Version 0.4 mainly addressed new functionality, and some thorough refactoring for Maps, and only introduced 2 bug fixes for Semantic Maps. So lets have a look at everything that changed

New features

• I added a display_map parser function, that’s obviously meant to display maps. The code handling this parser function is optimized to display maps, and does not hold into account any possible markers, ect. The main reason for adding this function is that it’ll be used for the Wikimedia Foundation implementation of Maps.
• Smart geocoding support. Maps is now able to determine if a value is a set of coordinates, or an address. This enables automatic geocoding of addresses, making the specification of the value type unneeded, and allows you to mix coordinates and addresses in location lists. This functionality is implemented for display_map’s default parameter, the one of display_point(s), and display_point(s) centre property.
• OpenStreetMap service. This service uses OpenLayers to display OpenStreetMap tiles. It’s optimized for OSM, and does not allow you to display other tile sources with it. This is another of the requirements for WF usage that’s now been completed. This service is only implemented in Maps ATM, and will be added to Semantic Maps in version 0.5.
• Support for DM (Decimal Minutes) and DD (Decimal Degrees) coordinate notations. Someone requested these to be added. Several other variants, such as GPS coordinates, might be put into Maps in the future.
• Added a setting specifying the minimum and maximum width and height of maps. When a value falls out of this range, it’ll be changed to the nearest allowed value. This is to prevent post-stamp or wallpaper maps, and is inspired by the similar functionality in the SlippyMap extension.
• Parsing of marker-specific title and label values. I actually marked the lack of this feature a bug a while ago, which I changed at the point where I remembered I simply did not add such a thing yet, and only map-wide title and label value’s got parsed.

Refactoring (“under the hood” changes)

• A new hook system for the parser functions, allowing the adding or removing of additional parser function support. Each mapping service will now have one class for handling on specific parser function. Again, the main reason behind this change is that it’s required for the WF usage of Maps, where at the start, only display_map will be used. Another great advantage is that you can now very easily add new parser functions without having to mess up the core extension code. An example of such a function is display_route, which might be added at some point to maps.
• Change the geocoding functionality into a true feature hook element, so it can be easily removed. This is not easy to do, since the geocoding functionality is currently used at various points by the parser functions. So before I can start making it a feature hook element, the parser function code will have to be restructured.
• Create service hook for the geocoding feature – separate from the mapping services hook. This will allow adding and removing support for a specific geocoding service.
• Since Maps is now able to distinguish coordinates from addresses, the display_address and display_addresses parser functions have been removed. For backward compatibility, calls to them will be routed to display_point(s). This will of course eventually be removed.
• Removed a redundant path variable. One of the messy remains of the initial Maps release, which even caused problems on some installations.

Bug fixes

• Fixed a mayor bug in the initialization method causing hook code to get executed at a probably wrong moment. This bug can be the cause of some weird problems that surfaced since 0.3.3.
• Fixed a rather annoying issue with size of popups in Google Maps. They did not stretch far enough vertically for large contents. I’m happy I got this fixed now, since I’ve been aware of it for quite a while now, but simply could not find a solution. Yaron pointed me to a helpful blog where a certain post set me on the right track. The solution was ridiculously simple, and one line (in the JS code) long.
• Fixed some problems with the display of the mapping result formats on the Special:Ask page of Semantic MediaWiki. I’ve done this by adding an aliasing system for result formats to SMW itself. So to be able to benefit from this, you need to have that version of SMW, or later. Maps has a build in check to see if the aliasing functionality of SMW is present, and if not, uses the old, but a lot less efficient, solution.

For the change log listing, see Maps version history and Semantic Maps version history.

The documentation has been completely updated for both extensions. I only have to change the demo’s on my demo wiki to reflect the new syntaxis of the parser functions. After that I can get to work on the to-do’s for version 0.5, which will include the long awaited static map support

Did I mention Semantic Maps now supports over 50 languages? :p

Downloads

• Maps 0.4 (zip, 7z)
• Semantic Maps 0.4 (zip, 7z)
• Maps and Semantic Maps 0.4 (zip, 7z)
• Maps 0.4 SVN tag
• Semantic Maps 0.4 SVN tag

Since the 0.3.4 release of both Maps and Semantic Maps, I’ve been putting the little free time I have to use by working on the to-do’s for the next release. Originally this would have been 0.3.5, but I’ve added several things to it, that made me decide to make it 0.4 recently, since it involved some relatively big changes for the users. Here are a few of the most noteworthy changes that are planned for 0.4. Some of them are already completed, while still have to start on the others. For a list of changes I’ve made since 0.3.4, view revisions 57704, 57585 and 56614 (newest first).

New Features

• A new display_map parser function. This will enable you to simply display a map without any coordinates indicated on it. Since none of the marker related code will be used, this will be a more efficient and clean way of displaying maps. The main reason I’m implementing this is cause it’s a requirement for getting Maps suitable for Wikimedia Foundation usage.
• An OSM mapping service, which uses OL, but only allows OSM layers and is optimized for OSM. This is an other to-do that got prioritized by the WF usage requirements.
• Parsing of marker-specific title and label values. I actually marked the lack of this feature a bug a while ago, which I changed at the point where I remembered I simply did not add such a thing yet, and only map-wide title and label value’s got parsed.

Refactoring

• A new hook system for the parser functions, allowing the adding or removing of additional parser function support. Each mapping service will now have one class for handling on specific parser function.
• Change the geocoding functionality into a true feature hook element, so it can be easily removed. This is not easy to do, since the geocoding functionality is currently used at various points by the parser functions. So before I can start making it a feature hook element, the parser function code will have to be restructured.
• Create service hook for the geocoding feature – separate from the mapping services hook. This will allow adding and removing support for a specific geocoding service.
• Restructure parser functions: change display_point(s) and display_address(es) to display_point(s), with auto detect to see if the provided value are coordinates or addresses. Retain display_address(es) for backward compatibility, but remove from docs.

Bug Fixes

• Fix a major bug in the initialization method causing hook code to get executed at a probably wrong moment. This bug can be the cause of some weird problems that surfaced since 0.3.3.
• Fix a rather annoying issue with size of popups in Google Maps. They did not stretch far enough vertically for large contents. I’m happy I got this fixed now, since I’ve been aware of it for quite a while now, but simply could not find a solution. Yaron pointed me to a helpful blog where a certain post set me on the right track. The solution was ridiculously simple, and one line (in the JS code) long.

After the actual release, I’ll post a complete list of changes. In any case, these changes will force me to make a heap of changes to the docs, and also to the powerpoint’s of the presentations related to Maps and SM I’m giving.

Today a new version of Maps and Semantic Maps where released. This version, 0.3.4, mainly focused on the internal structure of the extensions, performance issues and bug fixes. Some small new features have been added though.

Bug fixes

• Fixed some issues with invalid parameter handling. This comes down to 2 things. The first is that before this version, an empty parameter (so || instead of |name=value|) would override the default value when it occurred after the default. So #display_point:coordinates=1,1||service=openlayers would end up showing an error that no coordinates where provided. The second one is a fix in the error message you would get when entering a coordinate parameter with an empty value, like |coordinates=|, this previously caused the error that location “” could not be displayed, and has now been replaced by an error message saying that no coordinates have been provided, as is the case.
• In the from handling code, I fixed a more severe bug, one that caused a wiki with SM installed to fail in case Semantic Forms was not installed and enabled. Someone pointed out this bug to me via a talk page. Surprisingly, no one had noticed this pretty big bug before, which demonstrates the wide use of SF, at least by the people also using SM.
• Another bug pointed out by someone on the talk page of SM is that SM only displayed the last coordinate property in a result of a query on a map. Semantic Google Maps supported this feature, but a some point, presumably at version 0.2, I accidentally removed it while refactoring the query result handling code. Anyway, it’s back now.

Refactoring

• The biggest change in this release is the introduction of a new feature hook system. This hook system allows you to add new mapping features to Maps, very much like the mapping service hook system. This has great consequences for the modularity of Maps, which has taken a big step forward. Every feature is loaded and initialized from the initialization method of Maps. This allows any extension to adds it’s own functionality to Maps, that can then use the general mapping functions of Maps, and the services hook system, and of course then be used in other extensions. This change has rendered Semantic Maps into a simple holder of 2 features: query printers and form inputs. It does not do any initialization on it’s own any more, and simply hooks into Maps. Obviously, this new hook system brought with it a whole list of extensive changes made in both maps and Semantic Maps.
• Added internationalization for the mapping service names, and to the list notation at several places.
• Moved the SM related JS present in Maps files to separate files in SM.
• Fixed tiny performance issues all over the code.
• Added fall-back mechanism to the geocoding methods, and re-introduced the old style request, for cases where cURL is disabled.

New features

The only real new functionality is that you now can configure which mapping features should be enabled, and which should not. This is of course a direct result from the new feature hook system.

You can view a complete list of changes to Maps and changes to Semantic Maps at the relevant documentation pages.

So, what’s next?

Before 0.4, which will introduce some big new features, I’m planning to release at least one update to the 0.3.x branch. This update will have geocoding as a feature in the new hook, a hook system for parser functions, a new mapping service called OSM (optimized OSM code working with OpenLayers) and a new parser function display_map, to simply display a map without any markers.

While translating some of the C# code of MyDownloader to VB.Net for the .Net download library, I’ve already come across quite some awkward code. IMHO, the underneath example could go into the code horrors section of The Code Project.

Why? It uses breaks and returns in the middle of the code, like if C# is some lame generic language without selection structures. The use of the try-catch bloks also seems odd. Also have a look at the condition for the do while loop – lol. The only thing missing are a few GoTo statements, which would have convinced me to print this out and torture fellow geeks with

I started off translating this literally, but got so annoyed by the bad coding practice I decided to attempt to rewrite it. I say ‘attempt’, cause code written like that is not easy to understand, esp for people who are used to ‘decent’ structured code. I splitted the code into 2 subs instead of one, removed all breaks, returns, the insane do while true and put only the required code in try catches.

Like promised in my previous post about Maps and Semantic Maps 0.3, I’ll give you an overview of the most important new features in this new release.

Multi location parser functions

Two completely new parser functions have been added that allow the displaying of multiple points on a map. To avoid confusion, this is a feature in Maps, and has nothing to do with the semantic coordinate aggregation of Semantic Maps, which obviously already has multi coordinate capabilities. The new parser functions are display_points and display_addresses, acting as multi coordinate variants of display_point and display_address, respectively. This feature adds endless new usage options for Maps, from marking the locations you’ve been on holiday to, to a list of restaurants in a city. It will also be extremely useful to use together with the upcoming custom base layers and overlays feature for OpenLayers. An example of such usage is displaying markers with some pop-up contents on an anatomy chart. Note that a form input will likely be added to Semantic Maps to simplify such a task, by making the need to manually find and enter all coordinates obsolete.

Separate title and label for each point

This feature could actually be viewed as part of the multi location parser functions, but I like to keep them separate. Since 0.2, Maps allows you to optionally display a title and label together with the marker representing the provided coordinates or address. Logically, this should be extended to encompass multiple locations when you can put more then one on a single map. This feature has not been implemented yet, cause of some uncertainty about the correct wiki syntax. The current idea is to use something like #display_points:points=55.7557860, 37.6176330~title~label; 1,1~title; 12,34. Such a syntax’s would not allow any ‘;’ or ‘~’ to be displayed into the pop-up.

Configurable map types

In Maps 0.2, an improvement causing the ‘physical’ map type, for Google Maps, to be displayed in the map type control when this map type was set as default was made. This caused me to wonder why the whole control was not made configurable, so that users could specify the map types they want, and the order in which they want them. This is exactly what I’ve done in Maps 0.3, for both Google Maps and Yahoo! Maps. The user can now set the map types present in the map type control with the types parameter. When not set, types will be set to the (new) setting holding the default types for the relevant service. The parameter type does still precisely the same, with the addition that the provided value will be added to types when it’s not present in it yet.

While adding this new feature, I also had a close look at the available map types in the latest (sub)version of the Google Maps v2 API. It turned out to be really easy to add support for moon, Mars and sky maps (all 2D). The underneath screenshot shows a map with all the available map types for Google Maps in Maps 0.3. It also demonstrates the new multi location functionality.

The two interesting parameters in the wiki code that is responsible for this map are:

• addresses=Moscow, Russia; New York; London – This is the replacement of address in display_address.
• types=normal,satellite,hybrid,physical,earth,moon,mars,mars-elevation,mars-infrared,sky – The new types parameter that allows users to choose the available map types.

This change now makes both Google Maps and Yahoo! Maps usage more similar to the one of OpenLayers, with the difference that with OpenLayers, types is called layers, and type is called baselayer (althoguh the baselayer is not activated cause of some problems with it).

More to come

Several more new features will be added, and I still have some refactoring to-do’s on my list to tackle before the 0.3 release. I’ll post about those issues as soon as I have more news about them. The new release is coming closer and closer – I estimate it’ll be there in less then a week

Here you have my latest committed changes for both Maps and Semantic Maps.

Just like version 0.2 of both Maps and Semantic Maps, version 0.3 will feature a variety of large structural changes, aimed at increasing the performance of the extensions, making it possible to add new features, but most of all, make it more modular, to allow people to easily extend them. Here you have a list of the mayor changes that have been completed so far, with some explanation of the advantages they bring.

• Base class that bundles common functionality between MapsBaseMap, SMMapPrinter and SMFormInput, and gets inherited by those classes. This class has been named MapsMapFeature. This is a great step towards the complete centralization of all common code, but sadly enough was not implementable the whole way: MapsMapFeature cannot be inherited by SMMapPrinter, since this class already inherits the SMW QP. Yaron suggested to use a ‘has-a’ implementation, witch means creating an instance of one of the classes you want to inherit as a class field, instead of the ‘is-a’ which is the regular inheritance. Although this is doable in theory, it will bring with it some other problems, so will have to wait till another version.

• Separation between the common and specific parameters, and their default values. Each mapping service now holds an array containing the extra parameters they add (so the allowed ones), and another array containing their default values. Note that both arrays are merged with the common arrays at runtime, and will override similarity named items. This allows to change the default value of a common property, or change the aliases.

• Yes, indeed, that’s right, aliases! Property names now have aliases. This means you can add several alternative ways to name the same parameter, for instance, you can make so that ‘auto zoom’ and ‘auto-zoom’ will do excellently the same as the main parameter ‘autozoom’. This is particularly handy for parameters such as ‘centre’ (British spelling) and ‘center’ (American spelling).

• Hook like system for mapping services. This is without a doubt the most important change since 0.1. All hard-coded references to mapping services in the core code have been removed. A service is now added by one multi dimensional array in Maps.php (note that this can also be done in the initialization file of another extension!), which holds the name of the parser functions class and it’s location, the aliases for the service name (feature added in 0.2), and their allowed specific parameters and their aliases. This architecture will allow other people to create their own mapping extension using the Maps (and Semantic Maps) ‘API’, which enables them to focus on the mapping service specific code rather then extension related code, or common mapping code. There are numerous more advantages, like the fact that their ‘extension’ will follow some general norms, which will make it easier for uses to use (and install!) and for developers to work on, and the fact that their code will also benefit from improvements made to the core code of Maps and Semantic Maps. Once 0.3 is release, I’m going to create a new section in both the Maps and Semantic Maps documentation, which will hold a list of why people should consider choosing that approach, as opposed to creating their own little mapping extension, a list of the advantages, and documentation on how to effectively do it.

The above improvements have been made in Maps, but still have to be implemented in Semantic Maps, which is now on the top of my to-do list. I’m also busy with the adding of new functionality, and got some great results so far, but I’ll post about those when more of the work for 0.3 is done.