Michal Čihař - Blog Archives for phpMyAdmin

Last week was quite equally split between refactoring and bug fixing. As we're getting closer to 4.6.0 release more people give feedback on it and some corner case issues are being discovered.

However the biggest challenge was strange bug that was reported long time ago - it affected only Italian translation on Windows server. After lot of debugging, I've realized that trying to encode Cardinalità</td> to json (using json_encode) causes this problem. Obviously there have to be more conditions met as this string only doesn't cause it, however removing it or placing x between à and < fixes the problem. In the end I've workarounded it in our code and hopefully I will get to create proper bugreport for php so that it can be fixed upstream as well.

The refactoring was mostly focused on encoding conversion part which is now all embedded in the Encoding class. It was quite simplified and should also speed up pages a bit as it does the initialization only when needed not on every phpMyAdmin's page.

Handled issues:

• #12075 Info: Blank Login Page and URL-Query-Sorting
• #12084 1064 error when importing a wordpress sql-file
• #11705 Error code: 200 when loading table's structure (Windows 10)
• #12081 Space Usage alternating rows wrong when overhead is present
• #12083 mb_ereg_replace function was removed
• #12040 Update common.css.php
• #12065 INT column with default value and comment exported faulty
• #12077 phpmyadmin export invalid - Missing ENGINE/CHARSET/COLLATE
• #12076 phpmyadmin export invalid - COMMENT AS
• #11805 issue with mysql 5.5 sql_mode=TRADITIONAL with PMA 4.5.3.1
• #12072 database copy and export creating bad SQL
• #3 Typo in readme

Sometimes you don't realize how does the time go until something reminds it to you. For me it when I've received some stuff from Marc, long term admin of the phpMyAdmin project. He had decided to leave some time ago and yesterday I've just received stuff he had collected at home including several awards.

Looking at the awards is sometimes funny. For example in 2008 we got "Most Likely to Be the Next $1B Acquisition" :-).

Anyway this all reminded me that I've been around phpMyAdmin for almost 15 years now (my first contribution seems to be from 14th May 2001) and that's quite some time.

The last week was mostly spent on bugfixing and cleanup after security releases. Hopefully the amount of security reports will go down now.

Most of the bug fixes were in the SQL parser which influences quite a lot of parts of phpMyAdmin. It is responsible for splitting queries on import, generating queries for export or linting the queries as users type them.

Additionally Debian packages were also updated, for both unstable and testing and for Ubuntu PPA.

Handled issues:

• #12067 Adding "JSON" option to dropdown
• #12047 Filtering databases on databases listing
• #12052 Filtering databases on databases listing, Issue #12047
• #12063 duplicate a table occur an error,for uft-8?
• #12064 Cross-site scripting (XSS) vulnerability in phpMyAdmin Version 4.5.4.1
• #11776 SQL Linter Problems
• #12025 Import reports false SQL error with MariaDB
• #12045 unrecognized keyword left in where clause #11975 (REMAINS UNSOLVED)
• #12041 Missing indexes and constratins in export
• #12028 "ALL" keyword not recognized
• #12054 MySQLDump .sql import in v4.5.5 fails becuase of escaped characters
• #12053 upload-release error
• #12055 Parse eror with 4.6.0-rc1 and master
• #12056 Invalid data stored in $_SESSION[' PMA_token '] if openssl_random_pseudo_bytes() fails
• #12015 create-release errors
• #12048 SQL parser doesn't honor vendor config
• #12024 Better icon for table hiding
• #12032 Icons added and code changed to display icons #12024
• #12037 Fix parse git data without gz support (bug 12030)
• #12030 Do not try to parse git data without gz support
• #12044 Fix example in test/README.rst
• #10 1.1 XSS in Static analysis of SQL query [PMASA-2016-10]
• #11 1.2 XSS in "Edit inline" of SQL query [PMASA-2016-11]
• #16 1.6 XSS Via HOST header [PMASA-2016-11]
• #17 1.7 XSS in file_echo.php by mime sniffing text/plain (only in old IE <= 8 & old Safari on windows) [PMASA-2016-11]
• #19 2. insecure CURL SSL Settings [PMASA-2016-13]
• #37 Fix DROP VIEW statement is not constructed properly by the parser, Issue #36
• #39 Fails to parser CREATE TABLE
• #36 DROP VIEW statement is not constructed properly by the parser
• #38 Recognize ALL when used with WHERE clause

As you could see tonight, last week was again full of security work. Mostly again XSS issues, but nothing really easily exploitable.

On the refactoring side, the most time was spent on bringing php-gettext to better shape. In the it ended up as being separate library which can be used by others as well - motranslator.

Another major task was to identify some easier tasks for prospective GSoC students and help them with implementing the changes. This worked quite well, but we will most likely loose those students as we were not selected this year to participate.

List of handled issues:

• #12029 4.5.5 ExportSql.class.php incorrect call to SqlParser\Context
• #12018 Replace builtin php-gettext with external library
• #12023 Replace embedded php-gettext library with motranslator
• #12021 Cookieless Applicatiion Support - PhpMyAdmin
• #11988 Refactor javascript escaping
• #11989 Refactor text sanitization
• #11990 Refactor url parmeters generating
• #11996 Issue#11989, #11988, #11990
• #11978 Resolve #11643 : Maintain filter states for db/table navigation filters
• #11963 Fixes #11865 partially
• #11986 Removal of Swekey
• #11962 Switch dependency handling to composer
• #11976 Move dependencies handling to composer
• #11994 Refactored url parmeters generating, Issue #11990
• #11995 remove_msie9 (fix_11987)
• #11987 Remove code for older MSIE < 9
• #11981 Remove Swekey authentication support

Since several years ago we've been using php-gettext in phpMyAdmin, but it's about time to change it. The change will not be that big, we're just moving to own fork of that library :-).

You probably ask why to fork? You will probably guess it easily, but to name some reasons:

• The php-gettext library is not maintained anymore
• It doesn't work with recent PHP version (phpMyAdmin has patched version)
• It's not possible to install it using Composer
• There was place for performance improvements in the library

So I've taken existing php-gettext codebase and turned that into motranslator and now 0.1 release of it is out. The recommended way to install it is from Packagist and it has no additional dependencies.

What changes you can expect? First of all it supports all current PHP versions. It also performs way better - in my tests loading of mo file is 4-5 times faster and memory consumption went down about 10 percent. You can additionally use object API instead of traditional function based.

On the other side some features we don't need were removed - there is no support for using native Gettext, it doesn't do any encoding conversion (assuming that UTF-8 is on both sides these days) and it doesn't support delayed loading of messages. The last change means that it's not suitable for applications with huge MO files.

Any feedback is welcome, the code is still fresh and probably needs some polishing.

As the flow of incoming bugs for upcoming 4.6.0 has slowed down a bit it was more time for code cleanups and related tasks. But it's also time where potential Google Summer of Code students come to our organization and want to get involved.

On the cleanup side the biggest was change to remove embedded PHP libraries which are available on Packagist from our Git and use Composer to manage the dependencies. This change will happen in 4.7.0, so it's still some time ahead, but it's already in our master branch. There still some third party libraries which we use and can not be installed using Composer, so we keep these for now.

Besides the usual bug fixing stuff, I've noticed that we lack issues which can be easily understood and fixed by potential Google Summer of Code. We require them to get involved before the program starts, so that we can see they are capable of useful contributions and also to see how they behave if asked for patch improvements. To fix this deficit we're prepared few small cleanup or refactoring tasks, where the students can show their skills.

All handled issues:

• #11980 Fix #11979 DECLARE not accepted as valid SQL
• #11609 Errors when Analyzer fails
• #11974 debug event in phpmyadmin for day and month (error)
• #11975 unrecognized keyword left in where clause
• #11973 Tracker won't track some queries executed from SQL tab
• #11907 Update query shows twice
• #11850 CSV import to new table broken
• #11958 Bug Fix #11907
• #11470 Impossible to cancel "Drop files here" overlay
• #6146 Syslog authentication requests / brute-force protection
• #6261 All-Languages version via composer/packagist.org
• #11947 Notice in sql.lib
• #11669 Can't edit rows if table name has one upper case char (OS X)
• #11968 Syntax-errored ALTER TABLE is exported
• #11967 Warning on loading corrupted XML
• #11959 Uncaught ReferenceError: isStorageSupported is not defined
• #22 Make INI settings configurable via environment variables

As we're getting closer to release of new major release (4.6.0), the focus moves to the bug fixing.

Most of the fixed issues came from our error reporting server, which collects error reports from the installations. Vast majority if bugs were affecting older releases as well, so these were fixed for upcoming 4.5.5 as well.

Besides bugfixing there was also some cleanups in the master branch - shared code for processing sprites and covering it by tests and removed caching of MySQL server information as that didn't bring any speed improvements and lead to cache consistency issues.

All handled issues:

• #11955 Undefined variable: format
• #11953 Warning on realpath failure
• #11952 Warning on invalid file upload
• #11951 Warning open_basedir restriction in effect
• #11950 Undefined index: is_ajax_request
• #11948 Javascript error on longer location hash
• #11946 Notice when file upload fails
• #11945 Help me please with error code
• #11938 Document some known issues
• #11944 Error with disabled php_uname
• #11940 Warning on checking docs
• #11941 Warning when error reporting server is unreachable
• #11939 Not submitting errors without curl
• #11928 [doc] Installation with Composer - add --no-dev option
• #11933 Attempt to change table structure results in exception "file not found" in HHVM
• #11930 Can't change AUTO_INCREMENT
• #11931 Fix Can't change AUTO_INCREMENT, Issue #11930
• #11880 Issue#11872 Resolved
• #11872 Missing parameter error doesn't render HTML
• #11868 v 4.5.3.1 When editing field with primary key: "table does not contain a unique column" (OS X)
• #11887 Discrepancy between the calculated result and the displayed result of a query.
• #11905 Row count displayed by PMA does not match real result
• #11914 Undefined constant PHPSECLIB_INC_DIR with autoloader
• #11927 Remove MySQL version caching
• #11924 Database Server Information on start page wrong
• #11897 Remove direct support for openssl in cookie auth [WIP]
• #11929 No table data after inline edit
• #11916 Cookie auth keep asking for the password
• #18 Use port 80 instead of port 8080
• #19 Switch to port 80 (instead of 8080)

Last week was really focused on code cleanups. The biggest change was removal of PmaAbsoluteUri configuration directive, which has caused quite some pain in past and is not really needed these days (when browsers support relative paths in the Location HTTP header).

This lead to cleanup in other parts as well - support for dead Mozilla Prism is gone, used HTTPS for OpenStreetMap tiles (the map layer now works on HTTPS as well), removed ForceSSL configuration directive as this is something what really needs to be handled at web server level. To improve test coverage, several tests no longer require runkit as the header() call is wrapped within Response class and can be overridden for testing without using runkit.

The list of handled issues is not that impressive this week:

• #11913 Fix #11864 Move Noselect method to makegrid.js
• #11412 Wrong redirect from SECURE connection to INSECURE
• #17 Increase "max_input_vars" value

As I've already mentioned in separate blog post we mostly had some security issues fun in past weeks, but besides that some other work has been done as well.

I've still focused on code cleanups and identified several pieces of code which are no longer needed (given our required PHP version). Another issue related to security updates was to set testing of 4.0 branch using PHP 5.2 as this is what we've messed up in the security release (what is quite bad as this is only branch supporting PHP 5.2).

In addition to this, I've updated phpMyAdmin packages in both Debian and Ubuntu PPA.

All handled issues:

• #11902 Fix issue 11834
• #11900 Fix #11896 Remove hard dependency on phpseclib
• #11899 Fixes for #11892 and #11896 for 4.5 branch
• #11895 4.5.4 not in STABLE branch.
• #11894 Fix #11892 Error with PMA 4.4.15.3
• #11893 Fix #11891 Error with PMA 4.0.10.13 with PHP 5.2
• #11889 Update documents with Isaac as the new release manager
• #11888 Update release verification docs for Isaac
• #11886 CI failure notifications to mailing list
• #11883 HTTPS Redirect Loop when using ForceSSL on CentOS7 with Apache
• #11882 Add note about passing HTTP auth headers when running as FastCGI

As you can now see on phpMyAdmin's security page, we've managed to spend 9 security announcements on todays release. Hopefully it won't continue that bad in rest of the year.

Anyway receiving such extensive report was really challenging for us - correctly tracking and fixing all reported issues, discovering which versions are affected. This proven to be quite difficult given that most of the affected code has been refactored meanwhile. But I'm quite happy we've managed to fix ll issues on three supported branches in two weeks.

Another challenge (especially for Isaac) was that this all came with change of our release manager, so forgive us some minor problems with the releases (especially not updated changelogs), we will do it better next time!

PS: Updated packages are on their way to Debian and phpMyAdmin PPA.

PS2: It seems we've messed few more things, so expect quick followup releases for older versions.