Michal Čihař - Blog Archives for SUSE

What's coming in Weblate 2.12

Weblate should be released by end of February, so it's now pretty much clear what will be there. So let's look at some of the upcoming features.

There were many improvements in search related features. They got performance improvements (this is especially noticeable on site wide search). Additionally you can search for strings within translation project. On related topic, search and replace is now available for component or project wide operations, what can help you in case of massive renaming in your translations.

We have worked on improving machine translations as well, this time we've added support for Yandex. In case you know some machine translation service which we do not yet support, please submit that to our issue tracker.

Biggest improvement so far comes for visual context feature - it allows you to upload screenshots which are later shown to translators to give them better idea where and in which context the translation is used. So far you had to manually upload screenshot for every source string, what was far from being easy to use. With Weblate 2.12 (and this is already available on Hosted Weblate right now) the screenshots management got way better.

There is now separate interface to manage screenshots (see screenshots for Weblate as an example), you can assign every screenshot to multiple source strings, however you can also let Weblate automatically recognize texts on the screenshots using OCR and suggest strings to assign. This can save you quite a lot of effort, especially with screenshots with lot of strings. This feature is still in early phase, so the suggestions are not always 100% matching, but we're working to improve it further.

There will be some more features as well, you can look at our 2.12 milestone at GitHub to follow the process.

Weblate 2.11

Exactly on the schedule, Weblate 2.11 is out today. This release brings extended stats available to users and various other improvements and bug fixes.

Full list of changes:

  • Include language detailed information on language page.
  • Mercurial backend improvements.
  • Added option to specify translation component priority.
  • More consistent usage of Group ACL even with less used permissions.
  • Added WL_BRANCH variable to hook scripts.
  • Improved developer documentation.
  • Better compatibility with various Git versions in Git exporter addon.
  • Included per project and component stats.
  • Added language code mapping for better support of Microsoft Translate API.
  • Moved fulltext cleanup to background job to make translation removal faster.
  • Fixed displaying of plural source for languages with single plural form.
  • Improved error handling in import_project.
  • Various performance improvements.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

See you at FOSDEM

This has become sort of tradition for me, but still worth mentioning - I'm going to to FOSDEM this weekend.

This time I have no talk (I somehow failed to submit anything in time), but still I'm there to meet people and listen to some talks. As I've agreed to help Software Freedom Conservancy on stand (in the H building), it's quite likely that you will find me there. You will also have unique chance to grab phpMyAdmin stickers at this stand.

Also if you spot somebody wearing Weblate t-shirt, then it's most likely me ;-).

stardicter 0.11

Stardicter 0.11, the set of scripts to convert some freely available dictionaries to StarDict format, has been released today. There are mostly minor changes and it's time to push them out in official release. The most important being fixed sorting of ascii dictionaries, what did break searching in some programs.

Full list of changes:

  • Improved deaccent filter.
  • Fixed sorting of ASCII dictionaries.

As usual, you can install from pip, download source or download generated dictionaries from my website.

Weblate 2.10.1

This is first security bugfix release for Weblate. This has to come at some point, fortunately the issue is not really severe. But Weblate got it's first CVE ID today, so it's time to address it in a bugfix release.

Full list of changes:

  • Do not leak account existence on password reset form (CVE-2017-5537).

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far!

Seven tools that help us develop Weblate

Weblate probably would not exist (or at least would be much harder to manage) without several services that help us to develop, improve and fix bugs in our code base.

Over the time the development world has become very relying on cloud services. As every change this has both sides - you don't have to run the service, but you also don't have control on the service. Personally I'd prefer to use more free software services, on the other side I really love this comfort and I'm lazy to setup things which I can get for free.

The list was written down mostly for showing up how we work and the services are not listed in any particular order. All of the services provide free offerings for free software projects or for limited usage.

GitHub

I guess there is not much to say here, it has become standard place to develop software - it has Git repositories, issue tracker, pull requests and several other features.

Travis CI

Running tests on every commit is something what will make you feel confident that you didn't break anything. Of course you still need to write the tests, but having them run automatically is really great help. Especially great for automatically checking pull requests.

AppVeyor

Continuous integration on Windows - it's still widely used platform with it's quirks, so it's really good idea to test there as well. With AppVeyor you can do that and it works pretty nicely.

Codecov

When running tests it's good to know how much of your code is covered by them. Codecov is one of the best interfaces I've seen for this. They are also able to merge coverage reports from multiple builds and platforms (for example for wlc we have combined coverage for Linux, OSX and Windows coming from Travis CI and AppVeyor builds).

SauceLabs

Unit testing is good, but the frontend testing in browser is also important. We run Selenium tests in several browsers in SauceLabs to verify that we haven't screwed up something from the user interface.

Read the Docs

Documentation is necessary for every project and having it built automatically is nice bonus.

Landscape

Doing code analysis is a way to avoid some problems which are not spot during testing. These can be code paths not covered by test or simply coding style issues. There are several such services, but Landscape is my favorite one right now.

wlc 0.7

wlc 0.7, a command line utility for Weblate, has been just released. There are several new commands like translation file download or statistics fetching.

Full list of changes:

  • Added reset operation.
  • Added statistrics for project.
  • Added changes listing.
  • Added file downloads.

wlc is built on API introduced in Weblate 2.6 and still being in development, you need Weblate 2.10 for some feature (already available on our hosting offering). You can find usage examples in the wlc documentation.

Weblate 2.10

Quite on the schedule, Weblate 2.10 is out today. This release brings Git exporter module, improves support for machine translation services and adds various CSV exports and API interfaces.

Full list of changes:

  • Added quality check to check whether plurals are translated differently.
  • Fixed GitHub hooks for repositories with authentication.
  • Added optional Git exporter module.
  • Support for Microsoft Cognitive Services Translator API.
  • Simplified project and component user interface.
  • Added automatic fix to remove control chars.
  • Added per language overview to project.
  • Added support for CSV export.
  • Added CSV download for stats.
  • Added matrix view for quick overview of all translations
  • Added basic API for changes and units.
  • Added support for Apertium APy server for machine translations.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

New location for Weblate

Today, Weblate got new home. The difference is not that big - it has been moved from my personal GitHub account to WeblateOrg organization.

The main motivation is to have all Weblate related repositories in one location (all others including wlc, Docker or website are already there). The move will also allow to better manage the project in future as having it in separate repositories provides less management options on GitHub than using organization.

In case you have cloned the git repository, please update

git remote set-url origin https://github.com/WeblateOrg/weblate.git

Of course all issue tracker locations have changed as well (I believe the redirect on GitHub will stay as long as I won't fork the repository, so expect it to work at least month). See GitHub documentation on repository moving.

I'm sorry for all the troubles, but I think this is really necessary move.

phpMyAdmin security issues

You might wonder why there is so high number of phpMyAdmin security announcements this year. This situations has two main reasons and I will comment a bit on those.

First of all we've got quite a lot of attention of people doing security reviews this year. It has all started with Mozilla SOS Fund funded audit. It has discovered few minor issues which were fixed in the 4.6.2 release. However this was really just the beginning of the story and the announcement has attracted quite some attention to us. In upcoming weeks the security@phpmyadmin.net mailbox was full of reports and we really struggled to handle such amount. Handling that amount actually lead to creating more formalized approach to handling them as we clearly were no longer able to deal with them based on email only. Anyway most work here was done by Emanuel Bronshtein, who is really looking at every piece of our code and giving useful tips to harden our code base and infrastructure.

Second thing which got changed is that we release security announcements for security hardening even when there might not be any practical attack possible. Typical example here might be PMASA-2016-61, where using hash_equals is definitely safer, but even if the timing attack would be doable here, the practical result of figuring out admin configured allow/deny rules is usually not critical. Many of the issues also cover quite rare setups (or server misconfigurations, which we've silently fixed in past) like PMASA-2016-54 being possibly caused by server executing shell scripts shipped together with phpMyAdmin.

Overall phpMyAdmin indeed got safer this year. I don't think that there was any bug that would be really critical, on the other side we've made quite a lot of hardenings and we use current best practices when dealing with sensitive data. On the other side, I'm pretty sure our code was not in worse shape than any similarly sized projects with 18 years of history, we just become more visible thanks to security audit and people looked deeper into our code base.

Besides security announcements this all lead to generic hardening of our code and infrastructure, what might be not that visible, but are important as well:

  • All our websites are server by https only
  • All our releases are PGP signed
  • We actively encourage users to verify the downloaded files
  • All new Git tags are PGP signed as well