Michal Čihař - Archive for 3/2016

Last week we've finally released phpMyAdmin 4.6.0. That also means that 4.5.5 is out of maintenance now (however 4.4.15 and 4.0.10 are still supported if you're stuck with older PHP or MySQL versions for some reason).

The biggest focus was spent on handling incoming issues and checking if they are still present in 4.6.0 what in most cases proven not to be true.

Handled issues:

• #12121 PMA_Response class call not migrated
• #12126 Faq maint
• #12127 Fixup for missing glossary terms
• #12128 Update location of tcpdf library in FAQ
• #12118 Chinese language currently do not work in version 4.6.0
• #11604 Export with join table fails
• #12111 Ex- and Import definitly broken!
• #12071 Parser error on invalid ordering of LIMIT and FROM
• #12112 previous record

For quite some time I maintain set of scripts to convert some freely available dictionaries to StarDict format. I think it's time to make it easily available to others as well, so it has seen first release on PyPI after the code is almost 10 years old.

The project originally started for English - Czech dictionary, but over the time support for more sources have been introduced and you can generate StarDict dictionaries from many other sources. So we now have German - Czech dictionary as well. There is also support for dicts.info which provides many bilingual dictionaries, unfortunately it's license doesn't allow to distribute them.

Last week was a bit less working for me as I had few days off. The 4.6.0 is about to be released in the time as this blog post will be published, so the focus was obviously on fixing bugs.

Most of annoying issues I've fixed were caused by our SQL Parser which we need to generate some export statements and which didn't properly understand some more complex syntax. Most of the export issues are now fixed, but there are still situations where it doesn't parse the query right, usually this is true for nested SELECT statements or some more complex constructs (CASE). Anyway it's being improved and works better with every release.

Handled issues:

• #12109 Linter doesn't recognize CREATE TABLE new_tbl SELECT * FROM orig_tbl; syntax
• #12026 Queries with UNION may get corrupted when statements are enclosed in round brackets
• #12074 Invalid export
• #12068 Its impossible to export a database
• #12089 Multiple server doesn't work?
• #12098 HTTP auth fails if token parameter is present but wrong

Last week was quite equally split between refactoring and bug fixing. As we're getting closer to 4.6.0 release more people give feedback on it and some corner case issues are being discovered.

However the biggest challenge was strange bug that was reported long time ago - it affected only Italian translation on Windows server. After lot of debugging, I've realized that trying to encode Cardinalità</td> to json (using json_encode) causes this problem. Obviously there have to be more conditions met as this string only doesn't cause it, however removing it or placing x between à and < fixes the problem. In the end I've workarounded it in our code and hopefully I will get to create proper bugreport for php so that it can be fixed upstream as well.

The refactoring was mostly focused on encoding conversion part which is now all embedded in the Encoding class. It was quite simplified and should also speed up pages a bit as it does the initialization only when needed not on every phpMyAdmin's page.

Handled issues:

• #12075 Info: Blank Login Page and URL-Query-Sorting
• #12084 1064 error when importing a wordpress sql-file
• #11705 Error code: 200 when loading table's structure (Windows 10)
• #12081 Space Usage alternating rows wrong when overhead is present
• #12083 mb_ereg_replace function was removed
• #12040 Update common.css.php
• #12065 INT column with default value and comment exported faulty
• #12077 phpmyadmin export invalid - Missing ENGINE/CHARSET/COLLATE
• #12076 phpmyadmin export invalid - COMMENT AS
• #11805 issue with mysql 5.5 sql_mode=TRADITIONAL with PMA 4.5.3.1
• #12072 database copy and export creating bad SQL
• #3 Typo in readme

After almost six months of development Weblate 2.5 has been released. It brings lot of improvements and it's quite hard to point few ones. The most important ones include support for Python 3, reports generators, placeables highlighting, extended keyboard shortcuts, configurable dashboard or group based ACLs.

Full list of changes for 2.5:

• Fixed automatic translation for project owners.
• Improved performance of commit and push operations.
• New management command to add suggestions from command line.
• Added support for merging comments on file upload.
• Added support for some GNU extensions to C printf format.
• Documentation improvements.
• Added support for generating translator credits.
• Added support for generating contributor stats.
• Site wide search can search only in one language.
• Improve quality checks for Armenian.
• Support for starting translation components without existing translations.
• Support for adding new transations in Qt TS.
• Improved support for translating PHP files.
• Performance improvements for quality checks.
• Fixed sitewide search for failing checks.
• Added option to specify source language.
• Improved support for XLIFF files.
• Extended list of options for import_project.
• Improved targeting for whiteboard messages.
• Support for automatic translation across projects.
• Optimized fulltext search index.
• Added management command for auto translation.
• Added placeables highlighting.
• Added keyboard shortcuts for placeables, checks and machine translations.
• Improved translation locking.
• Added quality check for AngularJS interpolation.
• Added extensive group based ACLs.
• Clarified terminology on strings needing review (formerly fuzzy).
• Clarified terminology on strings needing action and not translated strings.
• Support for Python 3.
• Dropped support for Django 1.7.
• Dropped dependency on msginit for creating new Gettext po files.
• Added configurable dashboard views.
• Improved notifications on parse erorrs.
• Added option to import components with duplicate name to import_project.
• Improved support for translating PHP files
• Added XLIFF export for dictionary.
• Added XLIFF and Gettext PO export for all translations.
• Documentation improvements.
• Added support for configurable automatic group assignments.
• Improved adding of new translations.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Sometimes you don't realize how does the time go until something reminds it to you. For me it when I've received some stuff from Marc, long term admin of the phpMyAdmin project. He had decided to leave some time ago and yesterday I've just received stuff he had collected at home including several awards.

Looking at the awards is sometimes funny. For example in 2008 we got "Most Likely to Be the Next $1B Acquisition" :-).

Anyway this all reminded me that I've been around phpMyAdmin for almost 15 years now (my first contribution seems to be from 14th May 2001) and that's quite some time.

The last week was mostly spent on bugfixing and cleanup after security releases. Hopefully the amount of security reports will go down now.

Most of the bug fixes were in the SQL parser which influences quite a lot of parts of phpMyAdmin. It is responsible for splitting queries on import, generating queries for export or linting the queries as users type them.

Additionally Debian packages were also updated, for both unstable and testing and for Ubuntu PPA.

Handled issues:

• #12067 Adding "JSON" option to dropdown
• #12047 Filtering databases on databases listing
• #12052 Filtering databases on databases listing, Issue #12047
• #12063 duplicate a table occur an error,for uft-8?
• #12064 Cross-site scripting (XSS) vulnerability in phpMyAdmin Version 4.5.4.1
• #11776 SQL Linter Problems
• #12025 Import reports false SQL error with MariaDB
• #12045 unrecognized keyword left in where clause #11975 (REMAINS UNSOLVED)
• #12041 Missing indexes and constratins in export
• #12028 "ALL" keyword not recognized
• #12054 MySQLDump .sql import in v4.5.5 fails becuase of escaped characters
• #12053 upload-release error
• #12055 Parse eror with 4.6.0-rc1 and master
• #12056 Invalid data stored in $_SESSION[' PMA_token '] if openssl_random_pseudo_bytes() fails
• #12015 create-release errors
• #12048 SQL parser doesn't honor vendor config
• #12024 Better icon for table hiding
• #12032 Icons added and code changed to display icons #12024
• #12037 Fix parse git data without gz support (bug 12030)
• #12030 Do not try to parse git data without gz support
• #12044 Fix example in test/README.rst
• #10 1.1 XSS in Static analysis of SQL query [PMASA-2016-10]
• #11 1.2 XSS in "Edit inline" of SQL query [PMASA-2016-11]
• #16 1.6 XSS Via HOST header [PMASA-2016-11]
• #17 1.7 XSS in file_echo.php by mime sniffing text/plain (only in old IE <= 8 & old Safari on windows) [PMASA-2016-11]
• #19 2. insecure CURL SSL Settings [PMASA-2016-13]
• #37 Fix DROP VIEW statement is not constructed properly by the parser, Issue #36
• #39 Fails to parser CREATE TABLE
• #36 DROP VIEW statement is not constructed properly by the parser
• #38 Recognize ALL when used with WHERE clause

As you could see tonight, last week was again full of security work. Mostly again XSS issues, but nothing really easily exploitable.

On the refactoring side, the most time was spent on bringing php-gettext to better shape. In the it ended up as being separate library which can be used by others as well - motranslator.

Another major task was to identify some easier tasks for prospective GSoC students and help them with implementing the changes. This worked quite well, but we will most likely loose those students as we were not selected this year to participate.

List of handled issues:

• #12029 4.5.5 ExportSql.class.php incorrect call to SqlParser\Context
• #12018 Replace builtin php-gettext with external library
• #12023 Replace embedded php-gettext library with motranslator
• #12021 Cookieless Applicatiion Support - PhpMyAdmin
• #11988 Refactor javascript escaping
• #11989 Refactor text sanitization
• #11990 Refactor url parmeters generating
• #11996 Issue#11989, #11988, #11990
• #11978 Resolve #11643 : Maintain filter states for db/table navigation filters
• #11963 Fixes #11865 partially
• #11986 Removal of Swekey
• #11962 Switch dependency handling to composer
• #11976 Move dependencies handling to composer
• #11994 Refactored url parmeters generating, Issue #11990
• #11995 remove_msie9 (fix_11987)
• #11987 Remove code for older MSIE < 9
• #11981 Remove Swekey authentication support