Maps and Semantic Maps 0.6.5 are now available for download. This release contains mainly internal changes to improve code modularity and fix some security concerns. Several bugs have been fixed as well, and a new hook has been added to Semantic Maps. This hook will get you the map format as default one for queries where you only ask for coordinates when using SMW 1.5.2 or above. For a full list of changes since 0.6.4 see changes to Maps and changes to SM. Everyone running 0.6.2 or older is advised to upgrade as soon as possible. 

This release is notable for it being the first one in which I’m happy with the code-base as a whole. It took me a year to get here, but now I think the way the mapping extensions work is good and solid. This means you can now extend Maps and not be afraid the code will be incompatible in a few weeks due to changes. This also means that I’ll be focusing more on actual functionality rather then refactoring in future releases. I’ll be progressively building a little guide that explains how the extensions work from a developers perspective and how to extend them.

I might release another minor update in the 0.6.x series if any significant issues are found in 0.6.5. Further plans are finishing up a bunch of changes I’ve started to make in Validator, which I’ll probably release as 0.4 then, and to start working on Maps and Semantic Maps 0.7, which would aim at adding new features and improving existing ones. A likely new feature I’m particularly looking forward to implementing is several tag extensions that do the equivalent of the current parser functions added by Maps. The timetable for all this depends a lot on which other things I get cough up in (I’ll probably continue putting effort into the deployment stuff for my GSoC project) and what kind of funding will be available.

Downloads:

• Maps 0.6.5 [zip - 7z]
• Maps and Semantic Maps 0.6.5 [zip - 7z]

You can also view the release announcement at the documentation wiki.

                          

I got a new diagram!!!1!11!! It’s based on my previous one, but slightly more elaborate, and a lot less messy, as I now used Dia to create it

Legend:

• Striped lines: Existing components where code will be copied from, or based upon.
• Full lines: Components of the complete deployment model.
• Thick full lines: Core components (of the deployment model) that I definitely want to have completed during GSoC.

Since my previous post about my Google Summer of Code project I have been poking at the new MediaWiki installer to see what’s there already, how it is there, and how I can integrate it with the above deployment model. I’ve made a bunch of style and documentation improvements while going over the code, and renamed some things to make more sense. And I had Tim Starling clean up a bad svn commit I made

So what I’m doing now is splitting the current ‘Installer’ class, which is part of the new installer, into 2: Installer and CoreInstaller. Installer will hold general installer functionality and be part of the whole deployment model, while CoreInstaller will hold installer functionality specific to core, and will be part of the new installer. After that I can create an initial version of CoreInstallers counterpart: ExtensionInstaller.

                          

Today my MediaWiki.org user page is one year old – I created the first version on July 20, 2009. With my SVN account also approaching it’s first birthday, I can now say I’m doing MediaWiki development for a year. A lot has happened in this year.

I created the Maps and Semantic Maps extensions, and have continues releasing big and small updates the whole year long. At the end of 2009 I created the Validator extension to facilitate parameter handling in Maps and Semantic Maps. In early 2010 I was contracted by the Wikimedia Foundation to create the Storyboard extension, and by the Karlsruhe Institute of Technology to do work on Semantic MediaWiki. In May 2010 I started working on my Google Summer of Code 2010 project to create an extension management platform for MediaWiki. In between all these things I made various contributions to other extensions, including Semantic Forms, Semantic Internal Objects, Page Object Model, Semantic Compound Queries, Semantic Result Formats and Approved Revisions.

Next to all the code I created and released, I also attended several events and gave a number of presentations. These events include SMWCamp 2009 in Karlsruhe, the Berlin developers workshop in April and Wikimania 2010 in Gdansk.

I’m currently all time MediaWiki comitter #18, with 1080 commits. Looking forward to all the awesome stuff I can do in the coming year

                          

I figured having some unit tests for Maps, the MediaWiki extension to work with geographical data and display it by embedding dynamic maps into your articles, would be beneficial to it’s quality. It’s pretty hard to try cover all possible use cases with manual tests, and consumes a lot of time in any case. I therefore decided to try create some tests for the coordinate parser and formatter class, as it’s arguably the core feature of Maps.

I started off by trying to install plain PHPUnit, which is the most commonly used unit testing framework for PHP. This took me a while, as you are supposed to install it using PEAR (PHP Extension and Application Repository), a repository tool for PHP applications, and has never used this before. After two hours or so of messing around, I got both installed Then I went on investigating how I could best integrate this into my work-flow, and discovered that PHPUnit comes bundled with Zend Studio, seamlessly integrated, working completely out of the box o_O.

I then wrote a test case for the coordinate parsing and formatting class of Maps. I had a hard time getting it to work, as I needed to include MW itself, as the class uses MW functions. After some non-constructive discussion with several fellow MW devs I found a way to get it to work by including the maintenance script entry point, and tricking MW into thinking the call was made from a CLI. I now have a test case for the coordinate class, with tests for most of it’s functionality. Some more test data, and maybe some extra tests would be nice. A tricky thing in the case of this class is founding errors, which are hard to take into account, especially if you only want to allow them to a certain degree.

This particular test case is already paying off, as it made me find 3 subtle errors in coordinate parsing or formatting, that did not show up in my manual tests, as I was not covering the test data causing the issues.

I’m now planning to maybe write test cases for the distance parser to, which should be rather easy to do. I probably won’t create any others for Maps, as it’s rather time consuming, and I have a lot of other things to do right now. When I create new classes that are suited for unit tests in the future, I’ll definitely write tests for them as I build them up though, as it’ll not cost a lot more time then doing manual tests, and will ensure the classes are really solid.

                          

About a year back I decided I had to change my development environment from working on a remote server with a simple text editor to something more solid, if I wanted to do serious PHP development.

I went for Eclipse as IDE, which is without a doubt the most awesome IDE out there, as it’s completely open source, robust, and has a lot of extensions that make it usable for an awful lot of languages. For PHP there is the open source PHP Development Tools Project, and the commercial Eclipse based IDE build on top of this, Zend Studio. At that point having my development environment as mobile as possible was rather important, as I often occupied machines other then my laptop or my own desktop. Therefore I put both Eclipse and my server on an usb.

Now, a year later, a lot has changed, and I finally updated my tools accordingly. First of all, I’m now also using Linux, and plan to leave Windows into the dust bin soon, so prefer tools that work on both operating systems. Secondly, I now only develop on my laptop and own desktop machine, so portability is not as important any more. Also, I’m now doing a lot more serious PHP development as I was doing a year ago, and would benefit a lot from more decent debugging, testing and profiling tools.

What I did was throwing out my mobile web server and install Zend Studio, together with Zend Server Community Edition (which is free). They integrate in such a way that you can do code tracing, work with breakpoints, profile code, ect, all out of the box. If you are developing PHP applications like me and in search for a good tool, I can definitely recommend this. Zend Studio isn’t free, but it’s worth the price. Not going into a complete list of awesome stuff PDT and Zend Studio include, but this blog post, although a little dates, does a good job at it.

It’ll take me a while to integrate these features into my work-flow, as I’m not used to having them available, but I expect this to start paying off rather soon then late Also waiting on a new release of Zend Studio build on Eclipse 3.6 (Helios).

(O yeah, can’t write a post about Eclipse without saying: NetBeans fails )

                          

A lot has happened in my Google Summer of Code project since my last blog post about it, so here is another update. I did give a short presentation about it at Wikimania 2010, but that did not go very well unfortunately. You can get the slides though.

I changed the goals of my project again, from building an extension management platform build onto the Deployment Framework and Configure to creating a more general Deployment base on which the new MediaWiki installer and extension management can run. The reason for this change is that it makes a lot more sense from a technical perspective. The work done by the MediaWiki installer, and the still-to-be-created realized extension management is very similar after all. Because of this change I renamed the project from Extension Management Platform to Deployment.

Goals

So let’s have a look at what my idea of the end product is, from an users perspective. There would be 4 new special pages with the extension management functionality, largely based on what you can do in WordPress:

• Special:Install: On this page administrators can browse and search through extensions that are in the connected repository. This can be very basic to start with, but should eventually include filtering on categories and keywords, popularity, rating, ect.
• Special:Update: Checks for updates for both core and extensions, and shows update options for individual components, or the whole deal.
• Special:Extensions: A page listing all installed extensions, with options to uninstall, disable and upgrade them, as well as links to documentation, ect. Once MediaWiki has a configuration database, links to configure the extensions can also be included here.
• Special:Dashboard: A dashboard for administrators containing update information and fancy stuff like statistics.

The new MediaWiki installer would also have support for installation extension on initial run, and be able to do a complete core upgrade, instead of just a database update as is now the case.

Technical

This is a rough draft of how I see the structure of the end product, and where the code is coming from:

I’m in search of a quick and easy diagram tool to make a cleaner and more elaborate version

Stuff we already have

The underneath items have been completed, or require only a little work.

• Web interface for the core installer.
• Database abstraction for the installer.
• CLI interface for the core installer? (in progress)
• Core installer class with database install and upgrade capabilities.
• Filesystem abstraction (I ported this from WP (see code here), mostly done, no testing done yet though).

Stuff that’s still needed

• Everything related to detecting updates, fetching packages and instructions, ect. The DF has some nice stuff that can be used here, so does WP. It’s be nice to also have an abstraction layer here, so multiple mechanisms can be used here. An extension repository also needs to be set up, preferably on mediawiki.org.
• Filesystem support for the installer, so it can be used to upgrade MW by clicking a button and then just fetching the new release and doing all the work. This can be achieved by creating the generic installer class and making the core installer inherit from it.
• Extension installer class and the special pages that provide an interface to it.
• Extension support for the core installer: installation and upgrade. This can be done by re-using the code of the special pages.
• CLI support for extension management

Work done

This is a list of the work I have so far done during this project, oldest first.

• I researched all involved components and created the above draft.
• I ported the WordPress filesystem abstraction classes (base class, FTP and direct one).
• I got core commit access (yay!), so I now can make changes to the new installer.

Roadmap

A rather loose planning of what I’m planning to do next:

• Finish porting SSH2 filesystem abstraction class.
• Figure out how to make the whole system secure.
• Create installer class, adapt to core installer to work with this, and also create the extension installer class.
• Take care of the fetching stuff.
• Create the interfaces.

I’ll update the deployment wiki page as I make progress, but probably won’t make a lot of blog posts about it, as I want to focus on the work itself. Suggestions are welcome on the discussion page.

                          

Maps and Semantic Maps 0.6.4 are now available for download. This release contains several new features, amongst which basic KML support for Google Maps, a new OSM service implementation and re-added service links support to the Geographical Coordinates data-type in Semantic Maps. Everyone running 0.6.2 or older is advised to upgrade as soon as possible.

Lets have a closer look at some of the changes:

• The Google Maps (v2) service now has a KML parameter, which allows you to specify one or more KML files to use. Files are separated with comma’s. The base for this feature was submitted by Andy Chow, who created it for Transit Unlimited wiki. A neat example of a map there using KML is this one. Thanks for the patch Andy!
• There again is an OpenStreetMap service, although this one is different from the one that got removed in 0.6. The implementation is done by embedding (in an iframe) a page from the WMF toolserver, which renders a map with OSM tiles and Wikipedia overlay. Example. It’s basically the same what’s done on the German Wikipedia. Credits here go to Tim Alder and all other people who are making the awesome toolserver stuff work. The reason the creation of the map is not done on the server running the MediaWiki installation itself is that it’s rather difficult to fetch the Wikipedia layer from the toolsever. Hopefully a solution is found for this, as that would be a lot cooler for non WMF usage.
• I re-added service links support for the Geographical Coordinates data-type in Semantic Maps after someone requested this. It got removed in 0.6, as the old implementation was not working anymore, and I was unsure at that point how to create a new one. You can find the updated documentation on this here, although the examples are not working yet (semantic-mediawiki.org is using an alpha of 0.6). I will migrate the examples to the mapping documentation wiki at some later date.
• OpenLayers i18n (translation of the controls) should now be working. A style fix has also been made which caused OpenLayers to render badly in Chrome.
• Multiple small bugfixes have been made in both extensions, and a few in Validator. I guess this is now be the most stable release to date yet, so be sure to upgrade!

For a full list of changes since 0.6.3 see changes to Maps and changes to SM.

Downloads

• Maps 0.6.4: zip, 7z
• Maps and Semantic Maps 0.6.4: zip, 7z

See also

• The release notes on the documentation wiki

Andy Chow

                          

Maps and Semantic Maps 0.6.3 are now available for download. This release is one aimed to improve stability and addresses a number of bugs that where present in 0.6.x. Some notable fixed issues are a failure of parsing any coordinates containing a degree symbol that surfaced in 0.6.2, and the failing of non-Google Maps form inputs. Some internal rewriting has also been done, which was needed to address some of these bugs, and will enable future awesomeness increase. The included OpenLayers library has also been upgraded to 2.9.1. (changes to Maps, changes to SM). People running any 0.6.x are advised to upgrade. Especially people using Semantic Maps in conjunction with Semantic Forms, as both 0.6 and 0.6.1 contain undesired behaviour in the forms.

A new release of Semantic Bundle, which will include these versions of the mapping extensions, is likely to soon be released as well

Downloads

• Maps 0.6.3 (7z)
• Maps and Semantic Maps 0.6.3 (7z)

See also

• The release notes on the documentation wiki

                          

After doing a pile of research on how to best create an extension management platform for MediaWiki as my Google Summer of Code project, I realized that a lot of the work that I wanted to do was already done in some form or another, and decided to somewhat augment my goals. Since I can build upon Configure, and the Deployment Framework of Ontoprise, it should now be possible to also take care of the MediaWiki and extension configuration that I put as optional in my original proposal. To have some transparency here, and not to cause misconceptions, I created an awesome new roadmap.

I did an attempt to get some feedback by posting my roadmap on wikitech-l, but apparently everyone is either happy with it, or more likely, Parkinson’s Law of Triviality is in play here. So feedback and suggestions are definitely welcome, you can post them on the discussion page.

As Configure is doing an awesome job already without making significant changes to any existing code, I decided to start off with the Deployment Framework. I’m currently in the process of figuring out how it works exactly, so I’m able to extend it’s features, and build a GUI in the form of several MediaWiki special pages on top of it. This will have some immediate pay-off as Ontoprise will be able to use this improved version directly.

Both my presentations for Wikimania 2010 have been accepted, of which one is about my project, which will give me a change to explain to people what I’m doing and why it’s so awesome

                          

Maps and Semantic Maps 0.6.2 are now available for download.

This release contains improvements to parameter handling, case insensitivity has been re-added, and using default parameters together with named ones will work more intuitive. In Maps the handling of distances has also been considerably improved: there is a new #distance parser function, additional distance related parameters in #geodistance, and new settings that allow you to customize the default distance behaviour. In Semantic Maps a rather important bugfix has been made which solves a problem with editing articles with forms that contain maps that arose in 0.6.1. (changes to Maps, changes to SM). People running 0.6 and 0.6.1 are advised to upgrade. Especially people using Semantic Maps in conjunction with Semantic Forms, as both 0.6 and 0.6.1 contain undesired behaviour in the forms.

Read this announcement on the documentation wiki