Michal Čihař - Blog Archives for English

Testing Sphinx documentation with Jenkins

While reviewing comments on phpMyAdmin wiki (which we're shrinking down to developer documentation and moving end user documentation to proper documentation) I've noticed that people complained there on broken links in our documentation. Indeed there was quite some of them as this is something nobody really checks. It seems like obvious task to automate.

It seemed to me as obvious as somebody had to do it already. Unfortunately I have not found much, but at least there was Using Jenkins to parse sphinx warnings. This helps with the build warnings, but unfortunately I found no integration for the linkcheck builder. Fortunately it's quite easy with the Jenkins Warnings plugin to write custom parsers and to parse linkcheck output as well.

The Sphinx output parser based on above link can be configured like:

Regular Expression:

^(.*):(\d+): \((.*)\) (.*)

Mapping Script:

import hudson.plugins.warnings.parser.Warning

String fileName = matcher.group(1)
String lineNumber = matcher.group(2)
String category = matcher.group(3)
String message = matcher.group(4)

return new Warning(fileName, Integer.parseInt(lineNumber), "sphinx", category, message);

Example Log Message:

Percona-Server-1.0.2-3.rst:67: (WARNING/2) Inline literal start-string without end-string.

The Sphinx linkcheck output is quite similar:

Regular Expression:

^(.*):(\d+): \[([^\]]*)\] (.*)

Mapping Script:

import hudson.plugins.warnings.parser.Warning

String fileName = matcher.group(1)
String lineNumber = matcher.group(2)
String category = matcher.group(3)
String message = matcher.group(4)

return new Warning(fileName, Integer.parseInt(lineNumber), "sphinx-linkcheck", category, message);

Example Log Message:

faq.rst:793: [broken] http://www.hardened-php.net/: <urlopen error [Errno -3] Temporary failure in name resolution>

All you need to do now is to enable these in your Jenkins project, let the Sphinx parse output and the Sphinx linkcheck one file generated by linkcheck (usually _build/linkcheck/output.txt). The result can be found on the phpMyAdmin CI server.

Weekly phpMyAdmin contributions 2016-W15

After weeks of bugfixing my focus has again shifted to refactoring and code cleanups.

One big area was charsets and collations, which were cached in the session data so far. This had bad effect of making the session data quite huge leading to performance loss on every page, while the cached information is needed only on few pages. I've removed this caching, cleaned up the code and everything seems to be behave faster, even the pages which used cached content in the past.

Second area was handling of file uploads. Historically we had two copies of code doing almost the same thing. I've tried to merge them and use File class for all the operations. However this code was built to handle lot of corner cases, so I'm a bit afraid of breaking some special setups.

Handled issues:

Gammu 1.37.2

Yesterday, Gammu 1.37.1 and Gammu 1.37.2 have been released. As usual it collects bug fixes and minor improvements.

Full list of changes in 1.37.1:

  • Properly report errors in HEX encoded strings from SMSD SQL backends.
  • Configurable SMSD table names.
  • Improved support for Huawei E303.
  • Improved support for Vodafone K4511.
  • Improved support for Telit M2M modules.

Full list of changes in 1.37.2:

  • Fixed compilation of SMSD.

Would you like to see more features in Gammu? You an support further Gammu development at Bountysource salt or by direct donation.

Weekly phpMyAdmin contributions 2016-W14

Last week was focused on our SQL parser as several issues have been found there. If was for me also first release of SQL parser to packagist repository.

Besides code changes, there was quite some time spent on reviewing incoming issues. Most of they have been already fixed and users will get fixes in the 4.6.1 release.

Handled issues:

API for Weblate

The long requested feature for Weblate is here - it now has API which you can use to control it. It's scope is currently limited, but it will expand in future releases. The API is currently available in Git, deployed on both Demo server and Hosted Weblate, our hosted solution. It will be also part of 2.6 release, which should be released by end of April.

Anyway the API now provides basic operations up to translation level - you can not edit individual translations, handle suggestions or other advanced features. However what you can do includes:

  • Manipulating with the underlaying VCS repository (commit, push, pull, reset)
  • Download or upload translation files
  • Get information about translations, components, projects and languages
  • Lock or unlock components for translation

As you can see this really covers the basic operations which were most frequently requested, more will come in future. You can see more detailed the Weblate API documentation.

Ukolovnik 1.5

There were quite some translation and code fixes in the repository, so I guess it's time to release it as well. So here comes Ukolovnik 1.5, released almost four years since last release.

Full list of changes:

  • Translations update.
  • Changed licence to GPL 3 or later.
  • Various code cleanup.
  • Development moved to GitHub.

PS: I don't plan any further development of this tool, this is just to bring all (few) users to current code in git.

Weekly phpMyAdmin contributions 2016-W12

Last week we've finally released phpMyAdmin 4.6.0. That also means that 4.5.5 is out of maintenance now (however 4.4.15 and 4.0.10 are still supported if you're stuck with older PHP or MySQL versions for some reason).

The biggest focus was spent on handling incoming issues and checking if they are still present in 4.6.0 what in most cases proven not to be true.

Handled issues:

stardicter 0.9

For quite some time I maintain set of scripts to convert some freely available dictionaries to StarDict format. I think it's time to make it easily available to others as well, so it has seen first release on PyPI after the code is almost 10 years old.

The project originally started for English - Czech dictionary, but over the time support for more sources have been introduced and you can generate StarDict dictionaries from many other sources. So we now have German - Czech dictionary as well. There is also support for dicts.info which provides many bilingual dictionaries, unfortunately it's license doesn't allow to distribute them.

Weekly phpMyAdmin contributions 2016-W11

Last week was a bit less working for me as I had few days off. The 4.6.0 is about to be released in the time as this blog post will be published, so the focus was obviously on fixing bugs.

Most of annoying issues I've fixed were caused by our SQL Parser which we need to generate some export statements and which didn't properly understand some more complex syntax. Most of the export issues are now fixed, but there are still situations where it doesn't parse the query right, usually this is true for nested SELECT statements or some more complex constructs (CASE). Anyway it's being improved and works better with every release.

Handled issues: