Michal Čihař - Blog Archives for English

Weekly phpMyAdmin contributions 2017-W20

Last week I was again quite active on development side bringing several improvements to master branch.

The biggest news is probably that phpMyAdmin no longer relies on eval() function. We've used it to run advisory rules on server configuration, but that is now done using Symfony ExpressionLanguage (which we anyway need due to motranslator).

When looking at things this does pull in, I've noticed that there is mbstring polyfill, which can be used instead of the one we ship (and was never completed). Thanks to this the mbstring dependency is now optional, but still recommended for performance reasons.

Another quite visible change is adding JSON metadata to our themes. Right now it covers basic things like theme compatibility and authorship, but more can be added later. This is also covered in our documentation.

Handled issues:

HackerOne experience with Weblate

Weblate has started to use HackerOne Community Edition some time ago and I think it's good to share my experience with that. Do you have open source project and want to get more attention of security community? This post will answer how it looks from perspective of pretty small project.

I've applied with Weblate to HackerOne Community Edition by end of March and it was approved early in April. Based on their recommendations I've started in invite only mode, but that really didn't bring much attention (exactly none reports), so I've decided to go public.

I've asked for making the project public just after coming from two weeks vacation, while expecting the approval to take some time where I'll settle down things which have popped up during vacation. In the end that was approved within single day, so I was immediately under fire of incoming reports:

Reports on HackerOne

I was surprised that they didn't lie - you will really get huge amount of issues just after making your project public. Most of them were quite simple and repeating (as you can see from number of duplicates), but it really provided valuable input.

Even more surprisingly there was second peak coming in when I've started to disclose resolved issues (once Weblate 2.14 has been released).

Overall the issues could be divided to few groups:

  • Server configuration such as lack of Content-Security-Policy headers. This is certainly good security practice and we really didn't follow it in all cases. The situation should be way better now.
  • Lack or rate limiting in Weblate. We really didn't try to do that and many reporters (correctly) shown that this is something what should be addressed in important entry points such as authentication. Weblate 2.14 has brought lot of features in this area.
  • Not using https where applicable. Yes, some APIs or web sites did not support https in past, but now they do and I didn't notice.
  • Several pages were vulnerable to CSRF as they were using GET while POST with CSRF protection would be more appropriate.
  • Lack of password strength validation. I've incorporated Django password validation to Weblate hopefully avoiding the weakest passwords.
  • Several issues in authentication using Python Social Auth. I've never really looked at how the authentication works there and there are some questionable decisions or bugs. Some of the bugs were already addressed in current releases, but there are still some to solve.

In the end it was really challenging week to be able to cope with the incoming reports, but I think I've managed it quite well. The HackerOne metrics states that there are 2 hours in average to respond on incoming incidents, what I think will not work in the long term :-).

Anyway thanks to this, you can now enjoy Weblate 2.14 which more secure than any release before, if you have not yet upgraded, you might consider doing that now or look into our support offering for self hosted Weblate.

The downside of this all was that the initial publishing on HackerOne made our website target of lot of automated tools and the web server was not really ready for that. I'm really sorry to all Hosted Weblate users who were affected by this. This has been also addressed now, but the infrastructure really should have been prepared before on this. To share how it looked like, here is number of requests to the nginx server:

nxing requests

I'm really glad I could make Weblate available on HackerOne as it will clearly improve it's security and security of hosted offering we have. I will certainly consider providing swag and/or bounties on further severe reports, but that won't be possible without enough funding for Weblate.

phpMyAdmin available in Packagist

For quite some time, we did provide Composer packages for phpMyAdmin, though they were available only in separate repository and not in the main Packagist repository, but now it's there!

The reason why we didn't do that was that it really doesn't integrate well with our release process - we release ready to use tarballs, while the VCS doesn't contain all things end users expect (eg. byte compiled localization files). Putting generated content to VCS didn't sound right and there is no option of using own tarballs on Packagist repo.

That's why we've ended up providing own channel with release tarballs. However this approach is not good either as that already bundles dependencies installable by composer, possibly causing problems when trying to upgrade these.

Therefore I've decided to generate separate VCS for composer packages. This way it doesn't pollute development VCS, but still Composer gets what it expects. The phpmyadmin/phpmyadmin is now using separate VCS and is updated daily using shell script. There might be some glitches during initial runs, so please report me any problems you see.

You can find more information on installing phpMyAdmin using Composer in our documentation.

Weblate 2.14

Weblate 2.14 has been released today slightly ahead of the schedule. There are quite a lot of security improvements based on reports we got from HackerOne program, API extensions and other minor improvements.

Full list of changes:

  • Add glossary entries using AJAX.
  • The logout now uses POST to avoid CSRF.
  • The API key token reset now uses POST to avoid CSRF.
  • Weblate sets Content-Security-Policy by default.
  • The local editor URL is validated to avoid self-XSS.
  • The password is now validated against common flaws by default.
  • Notify users about imporant activity with their account such as password change.
  • The CSV exports now escape potential formulas.
  • Various minor improvements in security.
  • The authentication attempts are now rate limited.
  • Suggestion content is stored in the history.
  • Store important account activity in audit log.
  • Ask for password confirmation when removing account or adding new associations.
  • Show time when suggestion has been made.
  • There is new quality check for trailing semicolon.
  • Ensure that search links can be shared.
  • Included source string information and screenshots in the API.
  • Allow to overwrite translations through API upload.

If you are upgrading from older version, please follow our upgrading instructions.

You can find more information about Weblate on https://weblate.org, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Weblate is also being used on https://hosted.weblate.org/ as official translating service for phpMyAdmin, OsmAnd, Turris, FreedomBox, Weblate itself and many other projects.

Should you be looking for hosting of translations for your project, I'm happy to host them for you or help with setting it up on your infrastructure.

Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far! The roadmap for next release is just being prepared, you can influence this by expressing support for individual issues either by comments or by providing bounty for them.

Weekly phpMyAdmin contributions 2017-W19

Last week I finally got to doing something else than bug screening and fixing.

First of all the daily snapshots were improved in order to indicate the snapshot detail on our website, so that it's clear when it has been built and from which Git commit.

I've also looked at long outstanding issue of removing eval() usage from our codebase. The last piece where it has been used for Advisor and there is now my pull request to get rid of that.

Second long annoying thing is that we really don't have theme metadata in some easy to read format. Some of the information is set by PHP code and that's not really something you want to use to just get theme name, author or compatibility (actually the last bit is not really there). I've rewritten this to use JSON and there is pull request to implement the changes.

Probably both pull requests will land into master this week.

Handled issues:

New projects on Hosted Weblate

Hosted Weblate provides also free hosting for free software projects. The hosting requests queue was over one month long, so it's time to process it and include new project.

This time, the newly hosted projects include:

We now also host few new Minetest mods:

If you want to support this effort, please donate to Weblate, especially recurring donations are welcome to make this service alive. You can do them on Liberapay or Bountysource.

Weekly phpMyAdmin contributions 2017-W18

Last week I finally got back to work after mostly two weeks of vacation, so there was quite a lot of things to do. I've merged several pull requests, gone through incoming bugs and generally did some cleanup in our issue trackers.

I've also worked on new daily snapshots of our code, which are now available for download of for use from Docker Hub.

Handled issues:

New daily snapshots for phpMyAdmin

We have stopped providing daily snapshots for phpMyAdmin pretty much at time we've moved to GitHub, which allowed to download any branch as zip file. However since introduction of Composer to manage our dependencies, additional steps were required to get working copy of phpMyAdmin out of the snapshots.

Since today the ready to use snapshots are available again. They will be updated every day and are built in exactly same way as our releases, so all you need to do is download them and start using.

These snapshots can be also used from Docker - the phpMyAdmin image now has brand new tags edge-4.7 and edge-4.8 which are updated with every snapshot and contain latest changes from development branches.

Weekly phpMyAdmin contributions 2017-W17

I got back to work last week after vacation I had. The GSoC selection process is almost done (in two days the accepted students will be announced by Google) and things got again a bit calmer, so the usual amount of bug fixing and pull requests review has been done.

One thing worth mentioning is that we have started to use Twig templates and first templates were just ported to this. It will better enforce logic separation from the templates and also it makes templates easier to read.

Handled issues:

Weekly phpMyAdmin contributions 2017-W15

Last week was a bit shorter for me due to Easter, but still quite some work has been done. My time was mostly was spent on Docker and handling pull requests.

Anyway you won't see my report next week as I'm having few days off.

Handled issues: