Michal Čihař - Archive for Jan. 1, 2016

wlc 0.3, a command line utility for Weblate, has been just released. This is probably first release which is worth using so it's probably also worth of bigger announcement.

It is built on API introduced in Weblate 2.6 and still being in development. Several commands from wlc will not work properly if executed against Weblate 2.6, first fully supported version will be 2.7 (current git is okay as well, it is now running on both demo and hosting servers).

How to use it? First you will probably want to store the credentials, so that your requests are authenticated (you can do unauthenticated requests as well, but obviously only read only and on public objects), so lets create ~/.config/weblate:

[weblate]
url = https://hosted.weblate.org/api/

[keys]
https://hosted.weblate.org/api/ = APIKEY

Now you can do basic commands:

$ wlc show weblate/master/cs
...
last_author: Michal Čihař
last_change: 2016-05-13T15:59:25
revision: 62f038bb0bfe360494fb8dee30fd9d34133a8663
share_url: https://hosted.weblate.org/engage/weblate/cs/
total: 1361
total_words: 6144
translate_url: https://hosted.weblate.org/translate/weblate/master/cs/
translated: 1361
translated_percent: 100.0
translated_words: 6144
url: https://hosted.weblate.org/api/translations/weblate/master/cs/
web_url: https://hosted.weblate.org/projects/weblate/master/cs/

You can find more examples in wlc documentation.

After week of vacation I got back to work. There was lot of bug screening this week, it seems that people are finally migrating to 4.6 and discovering new problems there. Fortunately 4.6.1 has been released fixing most annoying issues.

Several issues were quite tricky to debug issue happening solely on Windows servers and looked quite tricky from beginning (breaking utf-8 chars). It turned out to be caused by preg_replace calls on the content, which could break utf-8 chars in some cases.

Handled issues:

• #12246 phpmyadmin with Mysql 5.7 - Empty value for 'port' specified
• #12252 Login failure Ubuntu 16.04 PHP7. Follow up to #12227
• #11705 Error code: 200 when loading table's structure (Windows 10)
• #12227 login page failure to load, php7.0
• #12249 Cannot login using PMA 4.6.x with Microsoft Edge or IE 11
• #12118 Chinese language currently do not work in version 4.6.0
• #12245 mbstring extension is missing
• #12239 Export tab isempty
• #12244 Too long URL with js/get_scripts.php
• #12240 Error with adding new fields to the table
• #12234 Designer Export Schema 414
• #11553 SQL Error upon login
• #12225 Make documentation accessible by HTTPS
• #12230 Error message "pma__tracking was not locked with LOCK TABLES"

Today it's fifteen years from my first contribution to free software. I've changed several jobs since that time, all of them involved quite a lot of free software and now I'm fully working on free software.

The first contribution happened to be on phpMyAdmin and did consist of Czech translation:

Subject: Updated Czech translation of phpMyAdmin
From: Michal Cihar <cihar@email.cz>
To: swix@users.sourceforge.net
Date: Mon, 14 May 2001 11:23:36 +0200
X-Mailer: KMail [version 1.2]

Hi

I've updated (translated few added messages) Czech translation of phpMyAdmin. 
I send it to you in two encodings, because I thing that in distribution 
should be included version in ISO-8859-2 which is more standard than Windows 
1250.

Regards
    Michal Cihar

Many other contributions came afterwards, several projects died on the way, but it has been a great ride so far. To see some of these you can look at my software page which contains both current and past projects and also includes later opensourced tools I've created earlier (mostly for Windows).

These days you can find me being active on phpMyAdmin, Gammu, python-gammu and Wammu, Debian and Weblate.

After getting complains from apt and users, I've finally decided to upgrade signing key on my Debian repository to something more decent that DSA. If you are using that repository, you will now have to fetch new key to make it work again.

The old DSA key was there really because my laziness as I didn't want users to reimport the key, but I think it's really good that apt started to complain about it (it doesn't complain about DSA itself, but rather on using SHA1 signatures, which is most you can get out of DSA key).

Anyway the new key ID is DCE7B04E7C6E3CD9 and fingerprint is 4732 8C5E CD1A 3840 0419 1F24 DCE7 B04E 7C6E 3CD9. It's signed by my GPG key, so you can verify it this way. Of course instruction on my Debian repository page have been updated as well.

Last week was quite split into many smaller tasks - working on our libraries (both SQL parser and motranslator got new releases with bug fixes), fixing bugs for upcoming 4.6.1 and working on documentation.

From the libraries side, probably most visible is release of motranslator 1.0, just to claim it's now stable enough. Let's see if somebody else will pick it up as well or it will stay only for our use.

Most time was however spent on our documentation. We've agreed to move wiki from our server to GitHub wiki and reduce content available on the wiki. So far it's really mixture of user documentation, notes and developer documentation. The final shape should be that wiki will contain only developer documentation and all end user documentation will go to our documentation. So far I've gone through about half of user docs pages, deleted duplicated ones and moved content to our documentation. It is most visible on the user guide which now contains way more information and hopefully it will get more complete in near future.

Handled issues:

• #12208 Get fatal javascript error version 4.6
• #12223 ALTER TABLE statements are not saved in history
• #11966 Rewrite Tracker to use SQL parser
• #12039 Refactored Tracker to use SqlParser, Issue #11966
• #12171 Issue with request for Edit view using web interface
• #12144 SQL parser shows errors when using MySQL function name as table alias
• #12205 Unrecognized keyword FULL and OUTER
• #12217 ReferenceError: pred_password is not defined
• #12215 Error: unrecognized expression
• #12212 Call to undefined function __() in libraries/sanitizing.lib.php on line 135
• #12214 fata error
• #12123 Error during upgrade from 4.5.5.1 to 4.6.0
• #29 Lock Table problem with phpmyadmin

Going back to faster release cycle, Weblate 2.6 has been just released. There is improved support for Python 3 or brand new HTTP REST API.

Full list of changes for 2.6:

• Fixed validation of subprojects with language filter.
• Improved support for XLIFF files.
• Fixed machine translation for non English sources.
• Added REST API.
• Django 1.10 compatibility.
• Added categories to whiteboard messages.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

After two months since it's announcement I think it's good time to release 1.0 version of motranslator. This release doesn't bring any major changes, it's more to indicate that the library is stable :-).

The motranslator is a translation library used in current phpMyAdmin master (upcoming 4.7.0) with focus on speed and memory usage. It uses Gettext MO files to load the translations. It also comes with testsuite (100% coverage) and basic documentation.

Recommended way to install it is using composer from Packagist repository:

composer require phpmyadmin/motranslator

The Debian package will be available probably at point phpMyAdmin 4.7.0 will be out, but if you see need to have it earlier, just let me know.

Last week was again focused on bug fixing due to increased amount of received bug reports on 4.6.0 release. Fortunately most of the annoying bugs are already fixed in git and will be soon released as 4.6.1.

Another bigger task which was started last week was wiki migration. So far we've been using own wiki running MediaWiki and we're migrating it to GitHub wiki. The wiki on GitHub is way simpler, but it seems as better choice for us. During the migration all user documentation will be merged into our documentation, so that it's all in one place and wiki will be targeted on developers.

Handled issues:

• #12146 Issue connecting to MySQL using SSL
• #12203 fatal on insert/modifying password filed as password function
• #12175 Exporting as .zip removes export timestamp, PHP version, and server version
• #12204 Report what is actually wrong if disk is full
• #12206 /setup script couldn't load previous config.inc.php
• #12199 Class 'PMA\libraries\navigation\nodes\Node' not found
• #12196 $cfg['ThemePath'] doesn't accept absolute paths
• #12200 ndbcluster engine missing relation view menu
• #12202 Win10 - EDGE - v4.6.0 - Call to undefined function PMA\libraries\PMA_langSet()
• #12201 PhpMyAdmin dies with gettext error, then unable to login
• #12197 Error when INSERT statement containing backslash
• #12190 GZip .....unavailable on this system ?
• #12194 Not able to Add Multiple Hosts in phpMyAdmin
• #12193 Do not hide errors from database connection attempts
• #12181 Call to undefined function __() in /pma/libraries/sanitizing.lib.php on line 135 (with signon and direct link)
• #12172 Error 1064
• #12186 Export and Import page goes blank. v4.6.0
• #12185 Some errors have been detected on the server
• #23 I cannot login to PMA - Error #2002 - php_network_getaddresses: getaddrinfo failed
• #27 Add Kubernetes support
• #28 Add ability to connect via MySQL socket instead of TCP

While reviewing comments on phpMyAdmin wiki (which we're shrinking down to developer documentation and moving end user documentation to proper documentation) I've noticed that people complained there on broken links in our documentation. Indeed there was quite some of them as this is something nobody really checks. It seems like obvious task to automate.

It seemed to me as obvious as somebody had to do it already. Unfortunately I have not found much, but at least there was Using Jenkins to parse sphinx warnings. This helps with the build warnings, but unfortunately I found no integration for the linkcheck builder. Fortunately it's quite easy with the Jenkins Warnings plugin to write custom parsers and to parse linkcheck output as well.

The Sphinx output parser based on above link can be configured like:

Regular Expression:

^(.*):(\d+): \((.*)\) (.*)

Mapping Script:

import hudson.plugins.warnings.parser.Warning

String fileName = matcher.group(1)
String lineNumber = matcher.group(2)
String category = matcher.group(3)
String message = matcher.group(4)

return new Warning(fileName, Integer.parseInt(lineNumber), "sphinx", category, message);

Example Log Message:

Percona-Server-1.0.2-3.rst:67: (WARNING/2) Inline literal start-string without end-string.

The Sphinx linkcheck output is quite similar:

Regular Expression:

^(.*):(\d+): \[([^\]]*)\] (.*)

Mapping Script:

import hudson.plugins.warnings.parser.Warning

String fileName = matcher.group(1)
String lineNumber = matcher.group(2)
String category = matcher.group(3)
String message = matcher.group(4)

return new Warning(fileName, Integer.parseInt(lineNumber), "sphinx-linkcheck", category, message);

Example Log Message:

faq.rst:793: [broken] http://www.hardened-php.net/: <urlopen error [Errno -3] Temporary failure in name resolution>

All you need to do now is to enable these in your Jenkins project, let the Sphinx parse output and the Sphinx linkcheck one file generated by linkcheck (usually _build/linkcheck/output.txt). The result can be found on the phpMyAdmin CI server.

After weeks of bugfixing my focus has again shifted to refactoring and code cleanups.

One big area was charsets and collations, which were cached in the session data so far. This had bad effect of making the session data quite huge leading to performance loss on every page, while the cached information is needed only on few pages. I've removed this caching, cleaned up the code and everything seems to be behave faster, even the pages which used cached content in the past.

Second area was handling of file uploads. Historically we had two copies of code doing almost the same thing. I've tried to merge them and use File class for all the operations. However this code was built to handle lot of corner cases, so I'm a bit afraid of breaking some special setups.

Handled issues:

• #12177 /localhost/phpmyadmin showing PHP code instead of login page
• #12178 Export creates invalid SQL with comments
• #12176 export creates invalid sql