Michal Čihař - Archive for 2/2016

Since several years ago we've been using php-gettext in phpMyAdmin, but it's about time to change it. The change will not be that big, we're just moving to own fork of that library :-).

You probably ask why to fork? You will probably guess it easily, but to name some reasons:

• The php-gettext library is not maintained anymore
• It doesn't work with recent PHP version (phpMyAdmin has patched version)
• It's not possible to install it using Composer
• There was place for performance improvements in the library

So I've taken existing php-gettext codebase and turned that into motranslator and now 0.1 release of it is out. The recommended way to install it is from Packagist and it has no additional dependencies.

What changes you can expect? First of all it supports all current PHP versions. It also performs way better - in my tests loading of mo file is 4-5 times faster and memory consumption went down about 10 percent. You can additionally use object API instead of traditional function based.

On the other side some features we don't need were removed - there is no support for using native Gettext, it doesn't do any encoding conversion (assuming that UTF-8 is on both sides these days) and it doesn't support delayed loading of messages. The last change means that it's not suitable for applications with huge MO files.

Any feedback is welcome, the code is still fresh and probably needs some polishing.

As the flow of incoming bugs for upcoming 4.6.0 has slowed down a bit it was more time for code cleanups and related tasks. But it's also time where potential Google Summer of Code students come to our organization and want to get involved.

On the cleanup side the biggest was change to remove embedded PHP libraries which are available on Packagist from our Git and use Composer to manage the dependencies. This change will happen in 4.7.0, so it's still some time ahead, but it's already in our master branch. There still some third party libraries which we use and can not be installed using Composer, so we keep these for now.

Besides the usual bug fixing stuff, I've noticed that we lack issues which can be easily understood and fixed by potential Google Summer of Code. We require them to get involved before the program starts, so that we can see they are capable of useful contributions and also to see how they behave if asked for patch improvements. To fix this deficit we're prepared few small cleanup or refactoring tasks, where the students can show their skills.

All handled issues:

• #11980 Fix #11979 DECLARE not accepted as valid SQL
• #11609 Errors when Analyzer fails
• #11974 debug event in phpmyadmin for day and month (error)
• #11975 unrecognized keyword left in where clause
• #11973 Tracker won't track some queries executed from SQL tab
• #11907 Update query shows twice
• #11850 CSV import to new table broken
• #11958 Bug Fix #11907
• #11470 Impossible to cancel "Drop files here" overlay
• #6146 Syslog authentication requests / brute-force protection
• #6261 All-Languages version via composer/packagist.org
• #11947 Notice in sql.lib
• #11669 Can't edit rows if table name has one upper case char (OS X)
• #11968 Syntax-errored ALTER TABLE is exported
• #11967 Warning on loading corrupted XML
• #11959 Uncaught ReferenceError: isStorageSupported is not defined
• #22 Make INI settings configurable via environment variables

As we're getting closer to release of new major release (4.6.0), the focus moves to the bug fixing.

Most of the fixed issues came from our error reporting server, which collects error reports from the installations. Vast majority if bugs were affecting older releases as well, so these were fixed for upcoming 4.5.5 as well.

Besides bugfixing there was also some cleanups in the master branch - shared code for processing sprites and covering it by tests and removed caching of MySQL server information as that didn't bring any speed improvements and lead to cache consistency issues.

All handled issues:

• #11955 Undefined variable: format
• #11953 Warning on realpath failure
• #11952 Warning on invalid file upload
• #11951 Warning open_basedir restriction in effect
• #11950 Undefined index: is_ajax_request
• #11948 Javascript error on longer location hash
• #11946 Notice when file upload fails
• #11945 Help me please with error code
• #11938 Document some known issues
• #11944 Error with disabled php_uname
• #11940 Warning on checking docs
• #11941 Warning when error reporting server is unreachable
• #11939 Not submitting errors without curl
• #11928 [doc] Installation with Composer - add --no-dev option
• #11933 Attempt to change table structure results in exception "file not found" in HHVM
• #11930 Can't change AUTO_INCREMENT
• #11931 Fix Can't change AUTO_INCREMENT, Issue #11930
• #11880 Issue#11872 Resolved
• #11872 Missing parameter error doesn't render HTML
• #11868 v 4.5.3.1 When editing field with primary key: "table does not contain a unique column" (OS X)
• #11887 Discrepancy between the calculated result and the displayed result of a query.
• #11905 Row count displayed by PMA does not match real result
• #11914 Undefined constant PHPSECLIB_INC_DIR with autoloader
• #11927 Remove MySQL version caching
• #11924 Database Server Information on start page wrong
• #11897 Remove direct support for openssl in cookie auth [WIP]
• #11929 No table data after inline edit
• #11916 Cookie auth keep asking for the password
• #18 Use port 80 instead of port 8080
• #19 Switch to port 80 (instead of 8080)

During the last days I've completed quite long time challenge - make Weblate compatible with Python 3. It's probably still not in shape you would use it on production server, but it runs fine and the testsuite completes without errors.

As usual, the major obstacle are dependencies. The translate-toolkit doesn't support Python 3 in any released version, so you need latest Git snapshot. Another problematic bit is Mercurial, which will not support Python 3 in near future. But in case you need it, you can still install Python 2 version and use it from Weblate (it uses command line interface).

The porting was mostly dealing with encoding mess - in Python 2 you really didn't have to distinguish between binary and text data, while Python 3 quite enforces that. This turned out to be especially tricky with CSV files where csv module expects text streams in Python 3, while translate-toolkit CSV parser expects binary ones.

All this has one downside as well - the test matrix got bigger again, what makes the testsuite run longer with 5 workers available.

While doing these changes, I've also decided to drop support for Django 1.7, this will allow us to get rid of lot of deprecation warnings in Django 1.9 and prepare codebase for Django 1.10.

Last week was really focused on code cleanups. The biggest change was removal of PmaAbsoluteUri configuration directive, which has caused quite some pain in past and is not really needed these days (when browsers support relative paths in the Location HTTP header).

This lead to cleanup in other parts as well - support for dead Mozilla Prism is gone, used HTTPS for OpenStreetMap tiles (the map layer now works on HTTPS as well), removed ForceSSL configuration directive as this is something what really needs to be handled at web server level. To improve test coverage, several tests no longer require runkit as the header() call is wrapped within Response class and can be overridden for testing without using runkit.

The list of handled issues is not that impressive this week:

• #11913 Fix #11864 Move Noselect method to makegrid.js
• #11412 Wrong redirect from SECURE connection to INSECURE
• #17 Increase "max_input_vars" value

I've not really spent much time on Gammu in past months and it was about time to do some basic housekeeping.

It's not that there would be too much of new development, I rather wanted to go through the issue tracker, properly tag issues, close questions without response and resolve the ones which are simple to fix. This lead to few code and documentation improvements.

Overall the list of closed issues is quite huge:

• #11 how do I connect my "nokia 500" with wammu?
• #14 Nokia 6303 Classic Unknown response
• #54 wammu tries an inappropriate ioctl on usb port on fedora 20
• #61 error sending sms
• #63 Smsd with mysql does not set delivery report
• #169 No delivery report with ZTE MF190
• #78 Bug when gammu-smsd start if [sql] table not named as default
• #93 SMS get status fail if modem busy
• #97 Need help with Siemens MC35i
• #130 GSM_SetIncomingUSSDCallback is never called with libgammu 1.36.5
• #151 ZTE MF190 identify issue
• #155 gammu-smsd kills modem
• #163 smsd exits on HUP signal
• #162 Sending messages out of order
• #157 Database Driver Issues
• #185 Properly report wrongly encoded strings in database
• #179 getussd: used wrong encoding
• #183 Huawei E169/E620/E800 initialization error
• #178 'gammu getussd' gives blank reply
• #181 crash wammu
• #180 crash wammu
• #13 Alcatel Onetouch 2005D unhandled exception
• #15 Problem while transmitting SMS to Nokia 3720c
• #17 Manual wrongly states: The "LogFile" entry in the [gammu] section is ignored
• #14 Problem with samsung GT-B2710 on Fedora 22
• #21 Unhandled exception appeared.

Do you want more development to happen on Gammu? You can support it by money.

Today, Gammu 1.37.0 has been released. As usual it collects bug fixes. This time there is another important change as well - improved error reporting from SMSD.

This means that when SMSD fails to connect to the database, you should get a bit more detailed error than "Unknown error".

Full list of changes:

• Improved compatibility with ZTE MF190.
• Improved compatibility with Huawei E1750.
• Improved compatibility with Huawei E1752.
• Increased detail of reported errors from SMSD.

Would you like to see more features in Gammu? You an support further Gammu development at Bountysource salt or by direct donation.

I had some pile of hosting requests in queue since half of January and my recent talk on FOSDEM had some impact on requests for hosting translations as well, so it's about time to process them.

New kids on the block are:

• DSC Auto Rename
• Openbmap / radiocells.org
• Lexica
• Transistor

Second round includes:

• OpenBikeSharing
• Binary Tick Trade App

Unfortunately I had to reject some projects as well mostly due to lack of file format support. This is still the same topic - when translating project, please stick with some standard format. Preferably what is usual on your platform.

If you like this service, you can support it on Bountysource salt or Gratipay. There is also option for hosting translations of commercial products.

As I've already mentioned in separate blog post we mostly had some security issues fun in past weeks, but besides that some other work has been done as well.

I've still focused on code cleanups and identified several pieces of code which are no longer needed (given our required PHP version). Another issue related to security updates was to set testing of 4.0 branch using PHP 5.2 as this is what we've messed up in the security release (what is quite bad as this is only branch supporting PHP 5.2).

In addition to this, I've updated phpMyAdmin packages in both Debian and Ubuntu PPA.

All handled issues:

• #11902 Fix issue 11834
• #11900 Fix #11896 Remove hard dependency on phpseclib
• #11899 Fixes for #11892 and #11896 for 4.5 branch
• #11895 4.5.4 not in STABLE branch.
• #11894 Fix #11892 Error with PMA 4.4.15.3
• #11893 Fix #11891 Error with PMA 4.0.10.13 with PHP 5.2
• #11889 Update documents with Isaac as the new release manager
• #11888 Update release verification docs for Isaac
• #11886 CI failure notifications to mailing list
• #11883 HTTPS Redirect Loop when using ForceSSL on CentOS7 with Apache
• #11882 Add note about passing HTTP auth headers when running as FastCGI

I've recently bought this beast and of course want to run it on Linux. Vendor does provide Linux drivers which even come with source, so that looked quite okay in the beginning.

However it turned out not to be that easy. First attempt was to install the 64-bit drivers and all I got from the printer is almost blank page with:

**** Unable to open the initial device, quitting.

Okay, that's not much helpful. Meanwhile I did install i386 system where it worked just fine. I started to smell some problems and looked at the source. It turned out to be almost complete, there was just single i386 binary, which is obviously needed by the driver.

Once realizing this, it was quite easy to make it work on 64 bit system as well:

dpkg --add-architecture i386
apt install libxml2:i386 libstdc++6:i386

Not that I'd be happy to run binary blob on my system, but at least printing now works.

Scanning from the device is easy - all you need to configure access to email and Samba and it works pretty without problems.