Michal Čihař - Blog Archives for Debian

Weblate 2.14.1

Weblate 2.14.1 has been released today. It is bugfix release fixing possible migration issues, search results navigation and some minor security issues.

Full list of changes:

  • Fixed possible error when paginating search results.
  • Fixed migrations from older versions in some corner cases.
  • Fixed possible CSRF on project watch and unwatch.
  • The password reset no longer authenticates user.
  • Fixed possible captcha bypass on forgotten password.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Turris, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

HackerOne experience with Weblate

Weblate has started to use HackerOne Community Edition some time ago and I think it's good to share my experience with that. Do you have open source project and want to get more attention of security community? This post will answer how it looks from perspective of pretty small project.

I've applied with Weblate to HackerOne Community Edition by end of March and it was approved early in April. Based on their recommendations I've started in invite only mode, but that really didn't bring much attention (exactly none reports), so I've decided to go public.

I've asked for making the project public just after coming from two weeks vacation, while expecting the approval to take some time where I'll settle down things which have popped up during vacation. In the end that was approved within single day, so I was immediately under fire of incoming reports:

Reports on HackerOne

I was surprised that they didn't lie - you will really get huge amount of issues just after making your project public. Most of them were quite simple and repeating (as you can see from number of duplicates), but it really provided valuable input.

Even more surprisingly there was second peak coming in when I've started to disclose resolved issues (once Weblate 2.14 has been released).

Overall the issues could be divided to few groups:

  • Server configuration such as lack of Content-Security-Policy headers. This is certainly good security practice and we really didn't follow it in all cases. The situation should be way better now.
  • Lack or rate limiting in Weblate. We really didn't try to do that and many reporters (correctly) shown that this is something what should be addressed in important entry points such as authentication. Weblate 2.14 has brought lot of features in this area.
  • Not using https where applicable. Yes, some APIs or web sites did not support https in past, but now they do and I didn't notice.
  • Several pages were vulnerable to CSRF as they were using GET while POST with CSRF protection would be more appropriate.
  • Lack of password strength validation. I've incorporated Django password validation to Weblate hopefully avoiding the weakest passwords.
  • Several issues in authentication using Python Social Auth. I've never really looked at how the authentication works there and there are some questionable decisions or bugs. Some of the bugs were already addressed in current releases, but there are still some to solve.

In the end it was really challenging week to be able to cope with the incoming reports, but I think I've managed it quite well. The HackerOne metrics states that there are 2 hours in average to respond on incoming incidents, what I think will not work in the long term :-).

Anyway thanks to this, you can now enjoy Weblate 2.14 which more secure than any release before, if you have not yet upgraded, you might consider doing that now or look into our support offering for self hosted Weblate.

The downside of this all was that the initial publishing on HackerOne made our website target of lot of automated tools and the web server was not really ready for that. I'm really sorry to all Hosted Weblate users who were affected by this. This has been also addressed now, but the infrastructure really should have been prepared before on this. To share how it looked like, here is number of requests to the nginx server:

nxing requests

I'm really glad I could make Weblate available on HackerOne as it will clearly improve it's security and security of hosted offering we have. I will certainly consider providing swag and/or bounties on further severe reports, but that won't be possible without enough funding for Weblate.

Weblate 2.14

Weblate 2.14 has been released today slightly ahead of the schedule. There are quite a lot of security improvements based on reports we got from HackerOne program, API extensions and other minor improvements.

Full list of changes:

  • Add glossary entries using AJAX.
  • The logout now uses POST to avoid CSRF.
  • The API key token reset now uses POST to avoid CSRF.
  • Weblate sets Content-Security-Policy by default.
  • The local editor URL is validated to avoid self-XSS.
  • The password is now validated against common flaws by default.
  • Notify users about imporant activity with their account such as password change.
  • The CSV exports now escape potential formulas.
  • Various minor improvements in security.
  • The authentication attempts are now rate limited.
  • Suggestion content is stored in the history.
  • Store important account activity in audit log.
  • Ask for password confirmation when removing account or adding new associations.
  • Show time when suggestion has been made.
  • There is new quality check for trailing semicolon.
  • Ensure that search links can be shared.
  • Included source string information and screenshots in the API.
  • Allow to overwrite translations through API upload.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Turris, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

New projects on Hosted Weblate

Hosted Weblate provides also free hosting for free software projects. The hosting requests queue was over one month long, so it's time to process it and include new project.

This time, the newly hosted projects include:

We now also host few new Minetest mods:

If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do them on Liberapay or Bountysource.

Weblate 2.13.1

Weblate 2.13.1 has been released quickly after 2.13. It fixes few minor issues and possible upgrade problem.

Full list of changes:

  • Fixed listing of managed projects in profile.
  • Fixed migration issue where some permissions were missing.
  • Fixed listing of current file format in translation download.
  • Return HTTP 404 when trying to access project where user lacks privileges.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Weblate 2.13

Weblate 2.13 has been released today pretty much on the schedule. The most important change being more fine grained access control and some smaller UI improvements. There are other new features and bug fixes as well.

Full list of changes:

  • Fixed quality checks on translation templates.
  • Added quality check to trigger on losing translation.
  • Add option to view pending suggestions from user.
  • Add option to automatically build component lists.
  • Default dashboard for unauthenticated users can be configured.
  • Add option to browse 25 random strings for review.
  • History now indicates string change.
  • Better error reporting when adding new translation.
  • Added per language search within project.
  • Group ACLs can now be limited to certain permissions.
  • The per project ALCs are now implemented using Group ACL.
  • Added more fine grained privileges control.
  • Various minor UI improvements.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

New free software projects on Hosted Weblate

Hosted Weblate provides also free hosting for free software projects. Finally I got to processing requests a bit faster, so there are just few new projects.

This time, the newly hosted projects include:

  • Pext - Python-based extendable tool
  • Dino - modern Jabber/XMPP Client using GTK+/Vala

If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do them on Liberapay or Bountysource.

Tests coverage from Windows builds revisited

Few months ago, I've written about getting coverage information from many platforms into one report. That approach worked, but I've always felt guilty for pushing almost thousand of files to Codecov.

This week I finally found time to revisit this and make it work better and faster. Actually just uploading these files took about 30 minutes. Together with slow tests execution (which took about 30 minutes as well), we were reaching AppVeyor build time limits and builds did timeout quite often, what is not really nice result.

I've started with rewriting the wrapper used to execute OpenCppCoverage. I've originally used Python for that, which is nice, but I thought the overhead must be noticeable. As it is not possible to execute Python script directly from CTest, it was wrapped in simple bat file, adding another overhead. Reimplementing this in C showed that there is indeed overhead, but this is not going to save more time than few minutes.

Next obvious step was to look at uploading coverage files as this is really something what should be fast and not take such enormous time. When writing the original post, I've already tried to merge coverage data using OpenCppCoverage, but that showed to be too slow to make it actually work (the testsuite didn't complete in given 60 minutes).

I was also looking at existing solutions to merge Cobertura XML files, but I've found nothing working reasonably fast. The problem is that all of these always merge two files at one step, making merging thousand files really slow job as you're constantly generating, parsing and processing the resulting xml file for 1000 times. Also these solutions are probably more generic that what I needed.

In the end these files are just simple XML and merging them should not be hard. I was able to quickly write Python script to merge them. It does not support all of the Cobertura attributes, it just merges line based coverage (as this is the only thing which OpenCppCoverage generates), but works pretty fast and reliable.

Overall the build time went down from 60 minutes to 35 minutes and I don't see much space to improve besides improving OpenCppCoverage speed, what is really out of scope for me. Actually older version (0.9.5.3) performs way faster than current one (0.9.6), which is 2-3 times slower.

Gammu 1.38.2

Yesterday Gammu 1.38.2 has been released. This is bugfix release fixing for example USSD or MMS decoding in some situations.

The Windows binaries are available as well. These are built using AppVeyor and will help bring Windows users back to latest versions.

Full list of changes and new features can be found on Gammu 1.38.2 release page.

Would you like to see more features in Gammu? You an support further Gammu development at Bountysource salt or by direct donation.

Life of free software project

During last week I've noticed several interesting posts about challenges being free software maintainer. After being active in open source for 16 years I can share much of the feelings I've read and I can also share my dealings with the things.

First of all let me link some of the other posts on the topic:

I guess everybody involved in in some popular free software project knows it - there is much more work to be done than people behind the project can handle. It really doesn't matter it those are bug reports, support requests, new features or technical debt, it's simply too much of that. If you are the only one behind the project it can feel even more pressing.

There can be several approaches how to deal with that, but you have to choose what you prefer and what is going to work for you and your project. I've used all of the below mentioned approaches on some of the projects, but I don't think there is a silver bullet.

Finding more people

Obviously if you can not cope with the work, let's find more people to do the work. Unfortunately it's not that easy. Sometimes people come by, contribute few patches, but it's not that easy to turn them into regular contributor. You should encourage them to stay and to care about the part of the project they have touched.

You can try to attract completely new contributors through programs as Google Summer of Code (GSoC) or Outreachy, but that has it's own challenges as well.

With phpMyAdmin we're participating regularly in GSoC (we've only missed last year as we were not chosen by Google that year) and it indeed helps to bring new people on the board. Many of them even stay around your project (currently 3 of 5 phpMyAdmin team members are former GSoC students). But I think this approach really works only for bigger organizations.

You can also motivate people by money. It's way which is not really much used on free software projects, partly because lack of funding (I'll get to that later) and partly because it doesn't necessarily bring long time contributors, just cash hunters. I've been using Bountysource for some of my projects (Weblate and Gammu) and so far it mostly works other way around - if somebody posts bounty on the issue, it means it's quite important for him to get that fixed, so I use that as indication for myself. On attracting new developers it never really worked well, even when I've tried to post bounties to some easy to fix issues, where newbies could learn our code base and get paid for that. These issues stayed opened for months and in the end I've fixed them myself because they annoyed me.

Don't care too much

I think this is most important aspect - you simply can never fix all the problems. Let's face it and work according to that. There can be various levels of don't caring. I find it always better to try to encourage people to fix their problem, but you can't expect big success rate in that, so you might find it not worth of the time.

What I currently do:

  • I often ignore direct emails asking for fixing something. The project has public issue tracker on purpose. Once you solve the issue there others will have chance to find it when they face similar problem. Solving things privately in mails will probably make you look at similar problems again and again.
  • I try to batch process things. It is really easier to get focused when you work on one project and do not switch contexts. This means people will have to wait until you get to their request, but it also means that you will be able to deal them much more effectively. This is why Free hosting requests for Hosted Weblate get processed once in a month.
  • I don't care about number of unread mails, notifications or whatever. Or actually I try to not get much of these at all. This is really related to above, I might to some things once in a month (or even less) and that's still okay. Maybe you're just getting notifications for things you really don't need to get notified on? Do you really need notification for new issues? Isn't it better just to look at the issue tracker once in a time than constantly feeling the pressure of not read notifications?
  • I don't have to fix every problem. When it seems like something what could be as well fixed by the reporter, I just try to give them guidance how to dig deeper into the issue. Obviously this can't work for all cases, but getting more people on board always helps.
  • I try to focus on things which can save time in future. Many issues turn out to be just some unclear things and once you figure out that, spend few more minutes to improve your documentation to cover that. It's quite likely that this will save your time in future.

If you still can't handle that, you should consider abandoning the project as well. Does it bring something to you other than frustration of not completed work? I know it can be hard decision, in the end it is your child, but sometimes it's the best think you can do.

Get paid to do the work

Are you doing your fulltime job and then work on free software on nights or weekends? It can probably work for some time, but unless you find some way to make these two match, you will lack free time to relax and spend with friends or family. There are several options to make these work together.

You can find job where doing free software will be natural part of it. This worked for me pretty well at SUSE, but I'm sure there are more companies where it will work. It can happen that the job will not cover all your free software activities, but this still helps.

You can also make your project to become your employer. This can be sometimes challenging to make volunteers and paid contractors to work on one project, but I think this can be handled. Such setup currently works currently quite well for phpMyAdmin (we will announce second contractor soon) and works quite well for me with Weblate as well.

Funding free software projects

Once your project is well funded, you can fix many problems by money. You can pay yourself to do the work, hire additional developers, get better infrastructure or travel to conferences to spread word about it. But the question is how to get to the point of being well funded.

There are several crowdfunding platforms which can help you with that (Liberapay, Bountysource salt, Gratipay or Snowdrift to mention some). You can also administer the funding yourself or using some legal entity such as Software Freedom Conservancy which handles this for phpMyAdmin.

But the most important thing is to persuade people and companies to give back. You know there are lot of companies relying on your project, but how to make them fund the project? I really don't know, I still struggle with this as I don't want to be too pushy in asking for money, but I'd really like to see them to give back.

What kind of works is giving your sponsors logo / link placement on your website. If your website is well ranked, you can expect to get quite a lot of SEO sponsors and the question is where to draw a line what you still find acceptable. Obviously the most willing to pay companies will have nothing to do with what you do and they just want to get the link. The industry you can expect is porn, gambling, binary options and various MFA sites. You will get some legitimate sponsors related to your project as well. We felt we've gone too far with phpMyAdmin last year and we've stricten the rules recently, but the outcome is still not visible on our website (as we've just limited new sponsors, but existing contracts will be honored).

Another option is to monetize your project more directly. You can offer consulting services or provide it as a service (this is what I currently do with Weblate). It really depends on the product if you can build customer base on that or not, but certainly this is not something what would work well for all projects.

Thanks for reading this and I hope it's not too chaotic, as I've moved parts there and back while writing and I'm afraid it got too long in the end.