Michal Čihař - Archive for 9/2016

Last week was again mostly spent on reviewing pull requests and screening issues. This little housecleaning work is sometimes surprisingly time consuming :-).

Besides that I've again reviewed potential security weaknesses in our process reported by Emanuel Bronshtein. This lead to various hardenings in our Docker container, Debian packages or our website. There are still places to improve, but we're getting better with every commit.

Additionally there was release for motranslator and SQL parser, both of these are now properly listed on GitHub releases page.

Handled issues:

• #12578 Fix error in adcaf7c
• #12580 Assign LIMIT clause only to syntactically correct queries
• #12574 SELECT * FROM INFORMATION_SCHEMA.SCHEMATA error
• #12568 Remove token from links and icon links in Navigation bar
• #12556 Fix #12300 : Uncheck the Add CREATE PROCEDURE / FUNCTION etc. checkbox for a selective export
• #12544 Fix #12530: Show enabled links for Edit and Execute after proper priv checks
• #12555 Allow vertical scrolling to read longer text values in grid editing
• #12557 Don't use PMA_DROP_IMPORT on Table insert if it has Blob field
• #12565 German Translation under Replication is wrong
• #54 allow to describe host with description
• #52 Is it necessary to run it as root?
• #53 Start PHP-FPM as nobody
• #31 Project's birthday is September 9
• #86 incorrect "Latest release" in sql-parser
• #83 XSS in the highlighter
• #84 Escape sequence injection in Formatting SQL query (cli fromat)
• #85 Fix parsing of user@host without backquotes

wlc 0.6, a command line utility for Weblate, has been just released. There have been some minor fixes, but the most important news is that Windows and OS X are now supported platforms as well.

Full list of changes:

• Fixed error when invoked without command.
• Tested on Windows and OS X (in addition to Linux).

wlc is built on API introduced in Weblate 2.6 and still being in development. Several commands from wlc will not work properly if executed against Weblate 2.6, first fully supported version is 2.7 (it is now running on both demo and hosting servers). You can usage examples in the wlc documentation.

Last week was heavily focused on reviewing incoming code, mostly on our SQL parser. Thanks to several contributions we have made it even better.

The SQL parser s releases now include list of changes, so you can easily see what has been changed. While touching the SQL parser code, I've added some missing bits in testsuite code coverage and we're now really close to 100%.

Another useful thing for our library users is API documentation which is now available at https://develdocs.phpmyadmin.net/. It covers all libraries we've recently released (motranslator, sql-parser and shapefile).

Handled issues:

• #12560 Avoid encoding release username in tarballs
• #12547 prevent direct calls of include files
• #12553 Fix #12320 : Copying a user does not copy usergroup
• #12548 Fix Export and Copy of Tables with Generated/Virtual columns
• #12397 Review mb_strlen uses
• #12510 Add https://keybase.io/phpmyadmin_sec to security page
• #12444 Comment command does not work correctly
• #12282 whitespaces before and after the column names
• #12512 BBCode not transfered to HTML
• #12550 idle requests every 2 seconds
• #11628 INSERT ... ON DUPLICATE KEY
• #28 Add DebConf16 team photo and short description to Team page
• #42 Example usage and wiki
• #81 Please provide a Changelog
• #82 fix default options in Formater constructor
• #80 Add support for PHP 5.3 [WIP]
• #77 Fix parsing of REPLACE INTO ... Statements
• #79 Stop SelectStatement parsing and revert back in case of ON DUPLICATE KEY …
• #76 Fix parsing of INSERT...SELECT and INSERT...SET syntax
• #70 Fix #59: Non-reserved keywords should be allowed as a field name
• #75 Add correct parsing of SET CHARACTER SET, CHARSET , NAMES statements
• #71 Fix #55 : Add support for spatial datatypes

Last week was a bit calmer on my phpMyAdmin contributions as I've spent more time on other free software projects (namely Gammu).

Anyway there was still some amount of reviewing pull requests and bug screening and I've added PHP 7.1 to the testsuite on Travis while fixing some bugs this has revealed (thanks Deven for helping in this as well). Most notable these were bugs in our testsuite feeding invalid mock data in some situations. Without type warnings in upcoming PHP 7.1 these would be further unnoticed.

Handled issues:

• #12302 Use number_format parameters instead of Util::localizeNumber
• #12537 Using number_format parameters instead of Util::localizeNumber
• #12507 Documentation Update after PMASA-2016-54 fix
• #12528 Documentation Update after PMASA-2016-54 fix #12507
• #12470 Store 'tab_' + tab_id into a variable
• #12541 Fixes #12470 Stores tab_id hash into a variable instead of location.hash
• #12540 Fixes #12470 Stores tab_id hash into a variable instead of location.hash
• #12532 Fix #12531 : Properly flag queries as DROP DATABASE
• #12539 Fix numeric errors in PHP7.1
• #51 [enhancement] Version only php-fpm, without nginx and supervisord
• #68 Fix #49: Add Support for 'CREATE TABLE table_copy LIKE table;'
• #67 Fix #53: Allow parsing of COMMENT operation in ALTER statement
• #69 Fix spaces after expression in PartitionDefinition's build
• #66 Fix #10: Add a check if we find a new start to new statement before delimiter

In last days, the Weblate Docker container got several improvements to make it better fit for production setup. All changes are heavily based on pull request by Fred Cox.

What all does this change bring?

• It now brings complete environment for Weblate including web server.
• The preferred database engine is now PostgreSQL.
• There is also memcached instance for better performance.
• Much more things can be configured using environment variables (machine translation or social auth).

I think this makes Weblate deployment for you much simpler if you already use Docker. I think this can can be further extended to other services like Kubernetes, so patches are welcome :-).

Last week had still major focus on our Docker container, which is now better than ever :-). There also was a motranslator release to push some of the bug fixes to users.

Other than that there was usual amount of bug screening and fixing which you can see from list of issues below.

Handled issues:

• #12405 Create PHP code in Insert Tab
• #12522 Fixes #12509 Removes unused function PMA_addJSVar
• #12509 Dead code
• #12524 Check for token mismatches only if it's a POST request
• #12520 phpmyadmin bot comment has links redirecting to a 404
• #12521 no functions possible in 4.0.x at all?
• #12303 Wrong table status size reported
• #12517 An error in Linter.php has been fixed.
• #12513 Invalid IFRAME tag
• #12516 Missing intval
• #12515 Add entries to .gitignore
• #12380 One code to handle HTTP requests
• #12465 Fixes #12380 Refactored http requests into one function "httpRequest" in Util.php
• #12502 Proper type support for mysqli_real_connect() parameters
• #32 Frequently losing session
• #49 PMA_ARBITRARY seems to be ignored in the latest image build (but its ok with the phpmyadmin/phpmyadmin:4.6.4-1 image)
• #20 Image does not shutdown gracefully
• #26 Change STOPSIGNAL: SIGINT
• #48 Use supervisor + nginx + php-fpm
• #63 fixup of example in documentation

Last version of Enca has been released several months and now it's time for new release. There are various bug fixes which have been committed to the Git repository meanwhile.

If you don't know Enca, it is an Extremely Naive Charset Analyser. It detects character set and encoding of text files and can also convert them to other encodings using either a built-in converter or external libraries and tools like libiconv, librecode, or cstocs.

Full list of changes for 1.19 release:

• fix possible memory leak
• make utf-8 detection work even on one character

Still enca is in maintenance mode only and I have no intentions to write new features. However there is no limitation to other contributors, join the project at GitHub :-).

You can download from https://cihar.com/software/enca/.