Michal Čihař - Archive for Jan. 1, 2017

Weekly phpMyAdmin contributions 2017-W11

The Google Summer of Code season is coming and as usual that brought attention of students who want to participate with phpMyAdmin. This means bigger traffic on our mailing list, issue tracker and pull requests.

Quite a lot of time in last week was spent on reviewing incoming pull requests and giving students hints how to improve them. Quite a lot of contributions have been merged and more will be in upcoming weeks. I've managed to fix some issues on my own as well.

There was some more work on the infrastructure as well, for example all our web servers now support HTTP/2, what should help in faster load times when you're using some recent browser.

Handled issues:

Life of free software project

During last week I've noticed several interesting posts about challenges being free software maintainer. After being active in open source for 16 years I can share much of the feelings I've read and I can also share my dealings with the things.

First of all let me link some of the other posts on the topic:

I guess everybody involved in in some popular free software project knows it - there is much more work to be done than people behind the project can handle. It really doesn't matter it those are bug reports, support requests, new features or technical debt, it's simply too much of that. If you are the only one behind the project it can feel even more pressing.

There can be several approaches how to deal with that, but you have to choose what you prefer and what is going to work for you and your project. I've used all of the below mentioned approaches on some of the projects, but I don't think there is a silver bullet.

Finding more people

Obviously if you can not cope with the work, let's find more people to do the work. Unfortunately it's not that easy. Sometimes people come by, contribute few patches, but it's not that easy to turn them into regular contributor. You should encourage them to stay and to care about the part of the project they have touched.

You can try to attract completely new contributors through programs as Google Summer of Code (GSoC) or Outreachy, but that has it's own challenges as well.

With phpMyAdmin we're participating regularly in GSoC (we've only missed last year as we were not chosen by Google that year) and it indeed helps to bring new people on the board. Many of them even stay around your project (currently 3 of 5 phpMyAdmin team members are former GSoC students). But I think this approach really works only for bigger organizations.

You can also motivate people by money. It's way which is not really much used on free software projects, partly because lack of funding (I'll get to that later) and partly because it doesn't necessarily bring long time contributors, just cash hunters. I've been using Bountysource for some of my projects (Weblate and Gammu) and so far it mostly works other way around - if somebody posts bounty on the issue, it means it's quite important for him to get that fixed, so I use that as indication for myself. On attracting new developers it never really worked well, even when I've tried to post bounties to some easy to fix issues, where newbies could learn our code base and get paid for that. These issues stayed opened for months and in the end I've fixed them myself because they annoyed me.

Don't care too much

I think this is most important aspect - you simply can never fix all the problems. Let's face it and work according to that. There can be various levels of don't caring. I find it always better to try to encourage people to fix their problem, but you can't expect big success rate in that, so you might find it not worth of the time.

What I currently do:

  • I often ignore direct emails asking for fixing something. The project has public issue tracker on purpose. Once you solve the issue there others will have chance to find it when they face similar problem. Solving things privately in mails will probably make you look at similar problems again and again.
  • I try to batch process things. It is really easier to get focused when you work on one project and do not switch contexts. This means people will have to wait until you get to their request, but it also means that you will be able to deal them much more effectively. This is why Free hosting requests for Hosted Weblate get processed once in a month.
  • I don't care about number of unread mails, notifications or whatever. Or actually I try to not get much of these at all. This is really related to above, I might to some things once in a month (or even less) and that's still okay. Maybe you're just getting notifications for things you really don't need to get notified on? Do you really need notification for new issues? Isn't it better just to look at the issue tracker once in a time than constantly feeling the pressure of not read notifications?
  • I don't have to fix every problem. When it seems like something what could be as well fixed by the reporter, I just try to give them guidance how to dig deeper into the issue. Obviously this can't work for all cases, but getting more people on board always helps.
  • I try to focus on things which can save time in future. Many issues turn out to be just some unclear things and once you figure out that, spend few more minutes to improve your documentation to cover that. It's quite likely that this will save your time in future.

If you still can't handle that, you should consider abandoning the project as well. Does it bring something to you other than frustration of not completed work? I know it can be hard decision, in the end it is your child, but sometimes it's the best think you can do.

Get paid to do the work

Are you doing your fulltime job and then work on free software on nights or weekends? It can probably work for some time, but unless you find some way to make these two match, you will lack free time to relax and spend with friends or family. There are several options to make these work together.

You can find job where doing free software will be natural part of it. This worked for me pretty well at SUSE, but I'm sure there are more companies where it will work. It can happen that the job will not cover all your free software activities, but this still helps.

You can also make your project to become your employer. This can be sometimes challenging to make volunteers and paid contractors to work on one project, but I think this can be handled. Such setup currently works currently quite well for phpMyAdmin (we will announce second contractor soon) and works quite well for me with Weblate as well.

Funding free software projects

Once your project is well funded, you can fix many problems by money. You can pay yourself to do the work, hire additional developers, get better infrastructure or travel to conferences to spread word about it. But the question is how to get to the point of being well funded.

There are several crowdfunding platforms which can help you with that (Liberapay, Bountysource salt, Gratipay or Snowdrift to mention some). You can also administer the funding yourself or using some legal entity such as Software Freedom Conservancy which handles this for phpMyAdmin.

But the most important thing is to persuade people and companies to give back. You know there are lot of companies relying on your project, but how to make them fund the project? I really don't know, I still struggle with this as I don't want to be too pushy in asking for money, but I'd really like to see them to give back.

What kind of works is giving your sponsors logo / link placement on your website. If your website is well ranked, you can expect to get quite a lot of SEO sponsors and the question is where to draw a line what you still find acceptable. Obviously the most willing to pay companies will have nothing to do with what you do and they just want to get the link. The industry you can expect is porn, gambling, binary options and various MFA sites. You will get some legitimate sponsors related to your project as well. We felt we've gone too far with phpMyAdmin last year and we've stricten the rules recently, but the outcome is still not visible on our website (as we've just limited new sponsors, but existing contracts will be honored).

Another option is to monetize your project more directly. You can offer consulting services or provide it as a service (this is what I currently do with Weblate). It really depends on the product if you can build customer base on that or not, but certainly this is not something what would work well for all projects.

Thanks for reading this and I hope it's not too chaotic, as I've moved parts there and back while writing and I'm afraid it got too long in the end.

Weekly phpMyAdmin contributions 2017-W10

Last week was a bit more focused on the infrastructure than the code. Most of our servers have been upgraded to Debian Stretch and can (finally) benefit from PHP 7.

The upgrade also brought more up to date versions to the demo server, it's now running MariaDB 10.1.21 and MySQL 8.0.0. The latter one will probably show some bugs in our code base, as it has not yet been fully tested.

Code wise, the most visible change is to bundle Let's encrypt root SSL certificate with our code, so that we can verify the server even if the host SSL verification is broken (what seems to happen on some Windows based setups).

Handled issues:

Weblate users survey

Weblate is growing quite well in last months, but sometimes it's development is really driven by people who complain instead of following some roadmap with higher goals. I think it's time to change it at least a little bit. In order to get broader feedback I've sent out short survey to active project owners in Hosted Weblate week ago.

I've decided to target at smaller audience for now, though publicly open survey might follow later (but it's always harder to evaluate feedback across different user groups).

Overall feelings were really positive, most people find Weblate better than other similar services they have used. This is really something I like to hear :-).

Weblate overall experience

Weblate compared with other tools

But the most important part for me was where users want to see improvements. This somehow matches my expectation that we really should improve the user interface.

Weblate future development

We have quite a lot features, which are really hidden in the user interface. Also interface for some of the features is far from being intuitive. This all probably comes from the fact that we really don't have anybody experienced with creating user interfaces right now. It's time to find somebody who will help us. In case you are able to help or know somebody who might be interested in helping, please get in touch. Weblate is free software, but this can still be paid job.

Last part of the survey was focused on some particular features, but the outcome was not as clear as I hoped for as almost all feature group attracted about same attention (with one exception being extending the API, which was not really wanted by most of the users).

Overall I think doing some survey like this is useful and I will certainly repeat it (probably yearly or so), to see where we're moving and what our users want. Having feedback from users is important for every project and this seemed to worked quite well. Anyway if you have further feedback, don't hesitate to use our issue tracker at GitHub or contact me directly.

Weekly phpMyAdmin contributions 2017-W09

Last was equally spent on bugfixing phpMyAdmin and improving our Docker container.

From the bugfixing phpMyAdmin, I've spent great time with HHVM. It all started with bug report that phpMyAdmin is not working with HHVM 3.18. I've quickly fixed reported errors and wanted to add HHVM to our test matrix on Travis CI. This was not that easy as expected due to need to upgrade Travis image to Trusty. Once I did that, the build no longer worker reliably as MySQL server was sometimes not started. This showed to be known Travis bug, which can be workarounded by not using their container based infrastructure.

Once I had everything building and testing, I've noticed that the complete test takes incredibly long on recent HHVM versions (3 minutes with HHVM 3.3, 12 minutes with HHVM 3.12 and 7 minutes with HHVM 3.18). After digging a bit deeper (and adding listing of 10 slowest testcases) I've been able to produce testcase and report issue to HHVM. So if you're using phpMyAdmin with recent HHVM and it's slow, this might be the cause.

Last but not least, our Docker container got several performance tweaks, mostly tweaking PHP's opcache settings.

Handled issues:

Weblate 2.12

Weblate 2.12 has been released today, few days behind schedule. It brings improved screenshots management, better search and replace features or improved import. Many of the new features were already announced in previous post, where you can find more details about them.

Full list of changes:

  • Improved admin interface for groups.
  • Added support for Yandex Translate API.
  • Improved speed of sitewide search.
  • Added project and component wide search.
  • Added project and component wide search and replace.
  • Improved rendering of inconsistent translations.
  • Added support for opening source files in local editor.
  • Added support for configuring visual keyboard with special characters.
  • Improved screenshot management with OCR support for matching source strings.
  • Default commit message now includes translation information and URL.
  • Added support for Joomla translation format.
  • Improved reliability of import across file formats.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Aptoide, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Weekly phpMyAdmin contributions 2017-W08

The amount of new issues reported for 4.7.0 seems to be going down, so there was some time left to review pull requests on master branch with new features. But there was fair amount of bug fixing as well.

We've also managed to release new version of SQL parser which brings support for LOAD DATA statements and other improvements.

Handled issues:

New free software projects on Hosted Weblate

Hosted Weblate provides also free hosting for free software projects. I'm quite slow in processing the hosting requests, but when I do that, I process them in a batch and add several projects at once.

This time, the newly hosted projects include:

  • DoubleContact - offline contact manager
  • LanguageTool - style and grammar checker for 25+ languages
  • Ghini - botanic collection manager
  • Converse.js - XMPP chat client in your browser
  • Pulse - IT Infrastructure Management Software
  • Turris - high performance and open source router
  • Icinga Editor - GUI for generating Icinga configuration files

If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do them on Liberapay or Bountysource.

Weekly phpMyAdmin contributions 2017-W07

Last week was mostly spent on bugfixing what can be easily seen from list of handled issues, which got quite large this week.

There is probably nothing extraordinary in the list, just dozens of issues which appeared in the 4.7.0 beta. The final 4.7.0 should be out in a month, so it's really time to polish it. Also it's time to work on translations, so that's something where you help is welcome.

Handled issues:

What's coming in Weblate 2.12

Weblate should be released by end of February, so it's now pretty much clear what will be there. So let's look at some of the upcoming features.

There were many improvements in search related features. They got performance improvements (this is especially noticeable on site wide search). Additionally you can search for strings within translation project. On related topic, search and replace is now available for component or project wide operations, what can help you in case of massive renaming in your translations.

We have worked on improving machine translations as well, this time we've added support for Yandex. In case you know some machine translation service which we do not yet support, please submit that to our issue tracker.

Biggest improvement so far comes for visual context feature - it allows you to upload screenshots which are later shown to translators to give them better idea where and in which context the translation is used. So far you had to manually upload screenshot for every source string, what was far from being easy to use. With Weblate 2.12 (and this is already available on Hosted Weblate right now) the screenshots management got way better.

There is now separate interface to manage screenshots (see screenshots for Weblate as an example), you can assign every screenshot to multiple source strings, however you can also let Weblate automatically recognize texts on the screenshots using OCR and suggest strings to assign. This can save you quite a lot of effort, especially with screenshots with lot of strings. This feature is still in early phase, so the suggestions are not always 100% matching, but we're working to improve it further.

There will be some more features as well, you can look at our 2.12 milestone at GitHub to follow the process.