Michal Čihař - Archive for Jan. 1, 2016

Nostalgy

Sometimes you don't realize how does the time go until something reminds it to you. For me it when I've received some stuff from Marc, long term admin of the phpMyAdmin project. He had decided to leave some time ago and yesterday I've just received stuff he had collected at home including several awards.

phpMyAdmin awards

Looking at the awards is sometimes funny. For example in 2008 we got "Most Likely to Be the Next $1B Acquisition" :-).

Anyway this all reminded me that I've been around phpMyAdmin for almost 15 years now (my first contribution seems to be from 14th May 2001) and that's quite some time.

Weekly phpMyAdmin contributions 2016-W09

The last week was mostly spent on bugfixing and cleanup after security releases. Hopefully the amount of security reports will go down now.

Most of the bug fixes were in the SQL parser which influences quite a lot of parts of phpMyAdmin. It is responsible for splitting queries on import, generating queries for export or linting the queries as users type them.

Additionally Debian packages were also updated, for both unstable and testing and for Ubuntu PPA.

Handled issues:

Weekly phpMyAdmin contributions 2016-W08

As you could see tonight, last week was again full of security work. Mostly again XSS issues, but nothing really easily exploitable.

On the refactoring side, the most time was spent on bringing php-gettext to better shape. In the it ended up as being separate library which can be used by others as well - motranslator.

Another major task was to identify some easier tasks for prospective GSoC students and help them with implementing the changes. This worked quite well, but we will most likely loose those students as we were not selected this year to participate.

List of handled issues:

Introducing motranslator

Since several years ago we've been using php-gettext in phpMyAdmin, but it's about time to change it. The change will not be that big, we're just moving to own fork of that library :-).

You probably ask why to fork? You will probably guess it easily, but to name some reasons:

  • The php-gettext library is not maintained anymore
  • It doesn't work with recent PHP version (phpMyAdmin has patched version)
  • It's not possible to install it using Composer
  • There was place for performance improvements in the library

So I've taken existing php-gettext codebase and turned that into motranslator and now 0.1 release of it is out. The recommended way to install it is from Packagist and it has no additional dependencies.

What changes you can expect? First of all it supports all current PHP versions. It also performs way better - in my tests loading of mo file is 4-5 times faster and memory consumption went down about 10 percent. You can additionally use object API instead of traditional function based.

On the other side some features we don't need were removed - there is no support for using native Gettext, it doesn't do any encoding conversion (assuming that UTF-8 is on both sides these days) and it doesn't support delayed loading of messages. The last change means that it's not suitable for applications with huge MO files.

Any feedback is welcome, the code is still fresh and probably needs some polishing.

Weekly phpMyAdmin contributions 2016-W07

As the flow of incoming bugs for upcoming 4.6.0 has slowed down a bit it was more time for code cleanups and related tasks. But it's also time where potential Google Summer of Code students come to our organization and want to get involved.

On the cleanup side the biggest was change to remove embedded PHP libraries which are available on Packagist from our Git and use Composer to manage the dependencies. This change will happen in 4.7.0, so it's still some time ahead, but it's already in our master branch. There still some third party libraries which we use and can not be installed using Composer, so we keep these for now.

Besides the usual bug fixing stuff, I've noticed that we lack issues which can be easily understood and fixed by potential Google Summer of Code. We require them to get involved before the program starts, so that we can see they are capable of useful contributions and also to see how they behave if asked for patch improvements. To fix this deficit we're prepared few small cleanup or refactoring tasks, where the students can show their skills.

All handled issues:

Weekly phpMyAdmin contributions 2016-W06

As we're getting closer to release of new major release (4.6.0), the focus moves to the bug fixing.

Most of the fixed issues came from our error reporting server, which collects error reports from the installations. Vast majority if bugs were affecting older releases as well, so these were fixed for upcoming 4.5.5 as well.

Besides bugfixing there was also some cleanups in the master branch - shared code for processing sprites and covering it by tests and removed caching of MySQL server information as that didn't bring any speed improvements and lead to cache consistency issues.

All handled issues:

Weblate supports Python 3

During the last days I've completed quite long time challenge - make Weblate compatible with Python 3. It's probably still not in shape you would use it on production server, but it runs fine and the testsuite completes without errors.

As usual, the major obstacle are dependencies. The translate-toolkit doesn't support Python 3 in any released version, so you need latest Git snapshot. Another problematic bit is Mercurial, which will not support Python 3 in near future. But in case you need it, you can still install Python 2 version and use it from Weblate (it uses command line interface).

The porting was mostly dealing with encoding mess - in Python 2 you really didn't have to distinguish between binary and text data, while Python 3 quite enforces that. This turned out to be especially tricky with CSV files where csv module expects text streams in Python 3, while translate-toolkit CSV parser expects binary ones.

All this has one downside as well - the test matrix got bigger again, what makes the testsuite run longer with 5 workers available.

While doing these changes, I've also decided to drop support for Django 1.7, this will allow us to get rid of lot of deprecation warnings in Django 1.9 and prepare codebase for Django 1.10.

Weekly phpMyAdmin contributions 2016-W05

Last week was really focused on code cleanups. The biggest change was removal of PmaAbsoluteUri configuration directive, which has caused quite some pain in past and is not really needed these days (when browsers support relative paths in the Location HTTP header).

This lead to cleanup in other parts as well - support for dead Mozilla Prism is gone, used HTTPS for OpenStreetMap tiles (the map layer now works on HTTPS as well), removed ForceSSL configuration directive as this is something what really needs to be handled at web server level. To improve test coverage, several tests no longer require runkit as the header() call is wrapped within Response class and can be overridden for testing without using runkit.

The list of handled issues is not that impressive this week:

Bug squashing in Gammu

I've not really spent much time on Gammu in past months and it was about time to do some basic housekeeping.

It's not that there would be too much of new development, I rather wanted to go through the issue tracker, properly tag issues, close questions without response and resolve the ones which are simple to fix. This lead to few code and documentation improvements.

Overall the list of closed issues is quite huge:

Do you want more development to happen on Gammu? You can support it by money.

Gammu 1.37.0

Today, Gammu 1.37.0 has been released. As usual it collects bug fixes. This time there is another important change as well - improved error reporting from SMSD.

This means that when SMSD fails to connect to the database, you should get a bit more detailed error than "Unknown error".

Full list of changes:

  • Improved compatibility with ZTE MF190.
  • Improved compatibility with Huawei E1750.
  • Improved compatibility with Huawei E1752.
  • Increased detail of reported errors from SMSD.

Would you like to see more features in Gammu? You an support further Gammu development at Bountysource salt or by direct donation.