Michal Čihař - Archives

Good morning DebConf

Yesterday evening I've arrived at DebConf 13. I was quite tired after traveling so I just had few beers and went to bed.

The travel went quite well besides somebody else broke wine and my suitcase was all wet from it. Fortunately almost nothing got inside so I have some clean clothes for next week :-).

As I went to bed quite early, I woke up early as well so I went to short walk to see what is nearby:

Sunrise at Lake Neuchâtel Castle Vaumarcus Castle Vaumarcus

Roadmap for Weblate 1.7

Weblate 1.6 is already two weeks out without any regressions, so now it's time to think about features for 1.7.

I definitely want to improve support for monolingual translation formats as they currently suffer by various metadata issues (eg. checks not working correctly). This requires quite big change in underlying database, but I think it's quite manageable.

When talking about checks, I want to make them configurable per project as global configuration is quite limiting for shared sites like hosted Weblate.

I will definitely need to touch registration and authentication, either by supporting latest django-registration API or to move some other solution with support for federated login. Which of these approaches will be used depends on my time, what is always the most limiting factor.

This might sound as too few features, but I'd like to make releases more often, so that should fit into this plan. However if you want to push for some feature, you still have two days to fund something on Catincan!

Weblate 1.6 in SUSE Studio

Finally, Weblate 1.6 is available as ready to use appliance in SUSE Studio.

This took quite long as openSUSE 12.3 does not work as expected inside SUSE Studio and I was too lazy to find all necessary tweaks earlier. The same problem actually prevented from Weblate 1.5 being there at all.

The biggest obstacle was MySQL setup - after enabling "Set up MySQL" in studio, you end up with MySQL database with empty mysql.users table, which obviously can not authenticate anybody. As I've found out, the only way around is manual setup.

Once this was figured out, the appliance only needed minor tweaking and is now ready to use in SUSE Gallery in all formats used for virtualization.

Weblate 1.6

Weblate 1.6 has been released today. It comes with lot of improvements, especially adds voting for suggestions, improves import performance and brings more helpful admin interface. This is also first release with crowdfunded feature, but there are still some features to fund!

Full list of changes for 1.6:

  • Nicer error handling on registration.
  • Browsing of changes.
  • Fixed sorting of machine translation suggestions.
  • Improved support for MyMemory machine translation.
  • Added support for Amagama machine translation.
  • Various optimizations on frequently used pages.
  • Highlights searched phrase in search results.
  • Support for automatic fixups while saving the message.
  • Tracking of translation history and option to revert it.
  • Added support for Google Translate API.
  • Added support for managing SSH host keys.
  • Various form validation improvements.
  • Various quality checks improvements.
  • Performance improvements for import.
  • Added support for voting on suggestions.
  • Cleanup of admin interface.

You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances will be soon available in SUSE Studio Gallery.

Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others.

If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you.

Speed improvements in Weblate

Yesterday, I've spent some time trying to improve speed of translation import in Weblate. It got annoyingly slow while adding new features and I felt there has to be some area for improvements even with all the features.

I was testing on development server with debug stuff enabled (what slows down things quite a lot already) and first import took about 30 seconds.

Quick look revealed that for linked repositories, the link lookup happens too often, so some caching could be used. This has saved about 3 seconds...

With offloaded indexing (what is anyway only useful setup for production use), every reindex request was checking whether there is already one before creating new one. In most cases there is none before, so it's simply faster to try to create one and ignore error in case it is already there. We're now 5 seconds faster.

Another big resource consumer are quality checks. They provide useful feedback to translators, but it means that quite a lot of code needs to be executed for every single string. First thing I've realized is that unit check state is recalculated everytime regardless the fact whether there was any change, what was easy to fix. The other speed issue is in consistency check - it does too complex SQL query, which usually does not match anything. Converting this to much simpler one (which will in most cases still have no results) improves performance for usual path while having minimal impact for the real checking. Another 4 seconds down on my test import.

Last thing which I changed is way how per translation statistics are updated. In past this required quite a lot of queries, but yesterday I've realized this can be much reduced. During test suite run on PostgreSQL, I saw that it's not that easy - PostgreSQL does not allow to sum boolean fields, but fortunately this can be fixed as well. This has saved about 2 seconds.

In the end the tested import took 16 seconds instead of original 30 and issued 2500 SQL queries instead of 5000 at the beginning. I hope you will enjoy regained speed of imports :-).

Weblate has suggestion voting

The first crowdfunded feature for Weblate is here - Voting for suggestions.

Once you enable this for your translation, you can vote for translation suggestions and once they reach defined number of votes, it will get accepted. You can however combine this with other Weblate features to achieve different setups, see our documentation for some other use cases.

This is also most likely last feature for Weblate 1.6, so you can expect release in next week or two in case no major problems appear.

PS: There are still some features to fund.

New tools for generating StarDict dictionaries

For quite some time, I maintain set of few scripts to generate dictionaries for StarDict. The generated dictionaries are quite popular, but the scripts become quite horrible over the time.

Now I finally found time to cleanup and refactoring, so that they share as much code as possible. As the duplication was quite massive, I was able to reduce code size to 60%, while adding new features and doing better error handling. Also now it should be pretty easy to add support for additional resources, so contributions are welcome :-).

The code can be found on GitHub.

Fund Weblate features

Weblate is now year and half old, has quite many users and almost all features I wanted it to have. Now it's time to push it further to fit also other projects.

Therefore I've decided to try crowdfunding on Catincan to see which of requested features are actually wanted most. Yes, you can vote with your money :-).

The first set of features to fund is:

This is my first experiment with crowdfunding, so I'm looking forward to it's results.

phpMyAdmin in GSoC 2013

As student application period of Google Summer of Code 2013 is over, it's time to look at proposals we got for phpMyAdmin.

First of all we got slightly less proposals than in past years. This is probably related to the fact that we still heavily focus on code cleanup this year and this is definitely not that interesting topic as adding new features. I think quality has also slightly improved and this time we received no bogus or spam proposals.

Also as usual, people tend to leave submission for very last date (though not that much as in past years):

Number of applications over time

Quite unsurprisingly the most interesting topic seemed to be interface improvements, though it is quite wide. You can see how other topics were frequent in following chart:

Number of applications for various types

Please note that the numbers are not 100% accurate as some proposals really did not fit into above categories.

Anyway we're just working on evaluation and will finalize it in upcoming days. Of course you will know the results from Google on May 27th.

Weblate 1.5

Weblate 1.5 has been released today. It comes with lot of improvements, especially in performance, reporting and support for machine translations.

Full list of changes for 1.5:

  • Please check manual for upgrade instructions.
  • Added public user pages.
  • Better naming of plural forms.
  • Added support for TBX export of glossary.
  • Added support for Bitbucket notifications.
  • Activity charts are now available for each translation, language or user.
  • Extended options of import_project admin command.
  • Compatible with Django 1.5.
  • Avatars are now shown using libravatar.
  • Added possibility to pretty print JSON export.
  • Various performance improvements.
  • Indicate failing checks or fuzzy strings in progress bars for projects or languages as well.
  • Added support for custom pre-commit hooks and commiting additional files.
  • Rewritten search for better performance and user experience.
  • New interface for machine translations.
  • Added support for monolingual po files.
  • Extend amount of cached metadata to improve speed of various searches.
  • Now shows word counts as well.

You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances will be soon available in SUSE Studio Gallery.

Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others.

If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you.