Michal Čihař - Blog Archives for Debian

Continuous integration on multiple platforms

Over the weekend I've played with continuous integration for Gammu to make it run on more platforms. I had to remember many things from the Windows world on the way and the solution is not yet complete, but the basic build is working, the only problematic part are external dependencies.

First of all we already have Linux builds on Travis CI. These cover compilation with both GCC and Clang compilers, hopefully covering most of the possible problems.

Recently I've added OS X builds on Travis CI, what was pretty much painless and worked out of the box.

The next major architecture to support is Windows. Once I've discovered AppVeyor I thought it might be the way to go. The have free plans for open-source projects (though it has only one parallel build compared to four provided by Travis CI).

As our build system is cross platform based on CMake, it should work pretty much out of the box, right? Well almost, tweaking the basics took some time (unfortunately there is no CMake support on AppVeyor, so you have to script it a bit).

The most painful things on the way:

  • finding our correct way to invoke build and testsuite
  • our code was broken on Windows, making the testsuite to fail
  • how to work with power shell (no, I'm not going to like it)
  • how to download and install executable to PATH
  • test output integration with AppVeyor - done using XSLT transformation and uploading test results manually
  • 32-bit / 64-bit mess, CMake happily finds 32-bit libs during the 64-bit build and vice versa, what makes the build fail later when linking - fixed by trying if code can be built with given library
  • 64-bit code crashes in dummy driver, causing testsuite failures (this has to be something Windows specific as the code works fine on 64-bit Linux) - this seems to be caused by too big allocations on stack, moving them to heap will fix this

You can check our current appveyor.yml in case you're going to try something similar. Current build results are on AppVeyor.

As a nice side effect, we now have up to date Windows binaries for Gammu.

Wammu 0.42

Yesterday, I've released Wammu 0.42. There are no major updates, more likely it's usual localization and minor bugfixes release.

As usual up to date packages are now available in Debian sid, Gammu PPA for Ubuntu or openSUSE buildservice for various RPM based distros.

Want to support further Wammu development? Check our donation options or support Gammu team on BountySource Salt.

Weekly phpMyAdmin contributions 2016-W32

Tonight phpMyAdmin 4.0.10.17, 4.4.15.8, and 4.6.4 were released and you can probably see that there are quite some security issues fixed. Most of them are not really exploitable unless your PHP and webserver are poorly configured, but still it's good idea to upgrade.

If you are running Debian unstable, use our phpMyAdmin PPA for Ubuntu or use phpMyAdmin Docker image upgrading should be as simple as pulling new version.

Besides fixing security issues, we're generally hardening our infrastructure. I'm really grateful that Emanuel Bronshtein (@e3amn2l) is doing great review of all of our code and helps us in this area. This will really make our code and infrastructure much better.

Handled issues:

Revoking old PGP key

It has been already six years since I've moved to using RSA4096 PGP key. For various reasons, the old DSA key was still kept valid till today. This is no longer true and it has been revoked now.

The revoked key is DC3552E836E75604 and new one is 9C27B31342B7511D. In case you've signed the old one and not the new one (quite unlikely if you did not sign it more than six years ago), there has been migration document, where you can verify my new key being signed by the old one.

Gammu 1.37.4

It has been almost three months since last Gammu release and it's time to push fixes out to users. This time the amount of fixes is quite small, covering Huawei devices and text mode for sending SMS.

Full list of changes in 1.37.4:

  • Improved support for Huawei E3131.
  • Fixed SMS support for MULTIBAND 900E.
  • Fixed SMS created in text mode.

Would you like to see more features in Gammu? You an support further Gammu development at Bountysource salt or by direct donation.

New projects on Hosted Weblate

For almost two months I found very little time to process requests to host free software on Hosted Weblate. Today the queue has been emptied, what means that you can find many new translations there.

To make it short, here is list of new projects:

PS: If you didn't receive reply for your hosting request today, it was probably lost, so don't hesitate to ask again.

Weblate 2.7

Slightly later than on monthly schedule but Weblate 2.7 is out today. This release brings improvements to the API and is first to officially support wlc a command line client for Weblate.

Full list of changes for 2.7:

  • Removed Google web translate machine translation.
  • Improved commit message when adding translation.
  • Fixed Google Translate API for Hebrew language.
  • Compatibility with Mercurial 3.8.
  • Added import_json management command.
  • Correct ordering of listed traslations.
  • Show full suggestion text, not only a diff.
  • Extend API (detailed repository status, statistics, ...).
  • Testsuite no longer requires network access to test repositories.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

wlc 0.4

wlc 0.4, a command line utility for Weblate, has been just released. This release doesn't bring much changes, but still worth announcing.

The most important change is that development repository has been moved under WeblateOrg organization at GitHub, you can now find it at https://github.com/WeblateOrg/wlc. Another important news is that Debian package is currently waiting in NEW queue and will hopefully soon hit unstable.

wlc is built on API introduced in Weblate 2.6 and still being in development. Several commands from wlc will not work properly if executed against Weblate 2.6, first fully supported version will be 2.7 (current git is okay as well, it is now running on both demo and hosting servers). You can usage examples in the wlc documentation.

uTidylib 0.3

Several years ago I've complained about uTidylib not being maintained upstream. Since that time I've occasionally pushed some fixes to my GitHub repository with uTidylib code, but without any clear intentions to take it over.

Time has gone and there was still no progress and I started to consider becoming upstream maintainer as well. I quickly got approval from Cory Dodt, who was the original author of this code, unfortunately he is not owner of the PyPI entry and the claim request seems to have no response (if you know how to get in touch with "cntrlr" or how to take over PyPI module please let me know).

Anyway the amount of patches in my repository is big enough to warrant new release. Additionally Debian bug report about supporting new HTML tidy library came in and that made me push towards releasing 0.3 version of the uTidylib.

As you might guess, the amount of changes against original uTidylib is quite huge, to name the most important ones:

Anyway as I can not update PyPI entry, the downloads are currently available only on my website: https://cihar.com/software/utidylib/

PHP shapefile library

Since quite a long time phpMyAdmin had embedded the bfShapeFiles library for import of geospatial data. Over the time we had to apply fixes to it to stay compatible with newer PHP versions, but there was really no development. Unfortunately, as it seems to be only usable PHP library which can read and write ESRI shapefiles.

With recent switch of phpMyAdmin to dependency handling using Composer I wondered if we should get rid of the last embedded PHP library, which was this one - bfShapeFiles. As I couldn't find alive library which would work well for us, I resisted that for quite long, until pull request to improve it came in. At that point I've realized that it's probably better to separate it and start to improve it outside our codebase.

That's when phpmyadmin/shapefile was started. The code is based on bfShapeFiles, applies all fixes which were used in phpMyAdmin and adds improvements from the pull request. On top of that it has brand new testsuite (the coverage is still much lower than I'd like to have) and while writing the tests several parsing issues have been discovered and fixed. Anyway you can now get the source from GitHub or install using Composer from Packagist.

PS: While fixing parser bugs I've looked at other parsers as well to see how they handle some situations unclear in the specs and I had to fix Python pyshp on the way as well :-).