Nijel's weblog | blog.cihar.com

I just came to another decision point. Curently new Gammu API is quite in good shape (even most of functions are documented), but I think whether it's time to push in also some other big changes, which I would like to see, or just wait for things to settle down before starting other major changes.

For application developers (using C API, Python ones won't notice any of those changes), it would be definitely better to do all changes in one big bang. However I'm really affraid of possible impacts on stability, because extreme big changes. I'm still more and more convinced about stabilizing current code and keep those big changes for later time.

When talking about this, I should also mentions change I want to introduce :-). First is switch to wchar_t instead of own UCS-2-BE encoding. This will help a lot in possible errors on allocating memory for strings and also allow us to use libc functions for string operations. The second change is even bigger - make all storages scalable to avoid problems with hardcoded limits. But for this change it should be possible to add compatibility API functions, so that users don't have to notice anything.

I never knew, that someone would certify few lines of Python code, but Softpedia did so:

   Your product "IMAP utils 0.2" has been tested by the Softpedia labs and
   found to be completely clean of adware/spyware components.

   We are impressed with the quality of your product and encourage you to
   keep this high standards in the future.

   To assure our visitors that "IMAP utils 0.2" is clean, we have granted
   it with the "100% FREE" Softpedia award. Moreover, to let your users
   know about this certification, you may display this award on your
   website, on software boxes or inside your product.

I still wonder why simplest "product" I ever released is getting so much publicity…

Recently spam checking stopped to be enough to prevent my mailbox to grow to incredible sizes, so I had to look for some possibility to block viruses and other malware. The obvious choice went to open source anti-virus Clamav.

Setting up was pretty easy, especially when there are tons of howtos on the net (e.g. Installing and configuring Exim 4 on Debian). All I had to do was to install Clamav and configure Exim to talk to it and reject unwanted stuff.

And what are results? Quite impressive - 154860 caught viruses in last 24 hours.I wonder that lots of them were also caught by SpamAssassin, because I definitely didn't receive that much malware in a day.

I just released new version of Gammu, you can download it from download server. The biggest change is new API. It is probably not yet completed, but I want to release it right now to receive feedback from other developers. Full list of changes:

• Support for some new Nokia fields.
• Made SMSD mode case insensitive.
• Fix iCalendar export typo (Alin Năstac).
• Better compatibility with Sharp phones.
• Completely changed public API to avoid too often ABI changes.
• See PORTING file for changes you have to do.
• Dropped autoconf support, use CMake instead.
• Please note that new API is still work in progress!

goText (Java apllication for your phone to send messages onver GPRS) has just received support to send messages to Czech Vodafone network. It is using public SMS gateway, the only drawback is that you have to detect captcha used on this page. But it still is cheaper than sending regullar SMS :-).

To use this service, add following URL to your goText services:

http://www.gotext.org/pub/cz-vodafone.php

New version of simple todo manager called Ukolovnik has been just released. Version 0.7 comes with following changes:

• Development switched to subversion.
• Added support for CSV export.
• Added support for vCalendar export.

Steve, I know that caching usually helps. But from all test I've made with MediaWiki installation on phpMyAdmin wiki, I can see that enabling any caching slows things down quite a lot.

With memcached based caching, each request takes about 15 seconds, if I add XCache backend for variables, it slows down things to 10 seconds and I receive same results with file based caching. So it doesn't seem to be problem with memcached, but rather MediaWiki issue. Unfortunately it's quite a lot of PHP code to investigate and I didn't find anything obvious in low level caching interface.

So say good bye to variable caching, opcode cache is enough for now and MySQL seems to handle the load quite well.

Same time I complained about not up to date xcache package, I wrote to current maintainer whether I can adopt it. As he promptly agreed to this, we now have up to date package in Debian archives.

The only thing I messed down is that I did upload in hurry and didn't check which all bugs I can close by it. Today it turned out that all opened bugs could be closed, so I made another upload just to close those bugs ;-).

As I wrote in previous post, memcached didn't help as expected with wiki performace, so it was time to look for something else.

Since I'm using lighttpd, I still considered using of XCache, but the Debian package is not really up to date (now it even does not install with current php), so it was too much effort.

But today I decided it's time to try it. I built package from current version (you can get it on debian.cihar.com), installed and it seems to increase performance a lot. Most visible it is on wiki pages, which now take about quarter of time to process. I only hope it won't have any stability impacts, but it survived fine stress tests I did so far.

I've been using memcached for some time on phpMyAdmin wiki and it seemed to help. But recently the performance went down. I didn't have time to investigate it till today and it turns out to be caused by memcached. When I switch it off, page load time goes from 16 seconds to 0.25 seconds.

I tried to search for some similar problems on the net, but I didn't succeed. So for now wiki runs without memcached and I will look at other possibilities of caching later.

Update: If you're interested about versions, it is MediaWiki: 1.7.1, PHP: 5.2.3-1 (cgi-fcgi), MySQL: 5.0.41-Debian_2-log and I have enabled persistent connections to MySQL.

Thing I longer wanted to happen is now becoming reality - Gammu is getting new public API. Man reasons for the change are:

• ABI stability. - Till now almost every change in internals of any driver lead to ABI change. If we would correctly increase so name on each ABI change, we would be somewhere near 200, what is not something we could be proud of.

• Centralisation of variables cleanup. - Currently all phone drivers have to do some common things in each function. New API allows to centralise those operations in one place.

• Exposing of internals. - Old API exposed too much of Gammu internals, what could be misused by programmers and could lead to unexpected behaviour when some internals are changed.

The work is not yet still complete, but the API is getting final shapes right now, so you still have chance to change something. While doing such big change, I decided also to complete documentation of all public functions. This is also currently work in progress, but arising API documentation is already available.

If you are developer using Gammu library, you will have to change some things in your code. To document these changes, new Porting guide has been created. I suggest you not to start with this right now as the API is really not complete and might change a lot during next days.

For long time I use few Python scripts to manage mails on IMAP account. On Tuesday I decided to put them under version control and improve them a bit, because I didn't want to have login information hardcoded in every script.

I made several improvements, added support for distutils and then I decided that I could also release this to public. I've named them IMAP utils, I made announcement to freshmeat.net and Python Cheese Shop (thanks to ./setup.py register) and now I wonder that these simple scripts have over hundred downloads and eight subscribers. Maybe it is not that useless as I thought :-).

You might have noticed that current Wammu is not available for Windows. The reason is simple - I was not able to produce working binary in reasonable time. Python-gammu itself seems to work, but when I run Wammu, it crashes quite soon. I tried to debug it for few evenings, but I was not successful.

Therefore I again ask for somebody who would be willing to maintain Windows build. Somebody with Windows experiences will be probably able to resolve problems much faster and will help creating application that will behave good on Windows. I am not able to achieve this as I do not know Windows much and do not use it at all.

I just released version 0.20 of Wammu. This is mostly bugfix release, which fixes various bugs reported by users.

• Improve locales behaviour (and require wxPython with Unicode).
• Show time in call listings.
• Fixed infinite loop in searching.
• Fixed displaying of messages with special chars.
• Fixed locale detection on system without configured locales.
• Use different approach for detecting home directory on Windows.
• Now requires PyWin32 on Windows.

While working on Gammu, I still wonder why people who want to connect their phone to computer buy phones from vendors who use own proprietary protocol or do not share any documentation. Then they come to Gammu mailing list and/or bug tracker and want Gammu to support their phone. Sometimes the fix is easy, but usually it is quite lot of work to debug unknown protocol.

I know this situation quite good from past. I had Alcatel phone, which was using proprietary protocol for access to in phone contacts and events. Fortunately Alcatel released synchronisation software for these phones (it of course runs only on Windows) which had enabled debugging and it was quite easy to understand protocol thanks to logs it could produce. But as newer phones with some extensions appeared, maintaining this became harder and harder.

When I looked for new phone, I decided to buy Sony-Ericcsson K750i phone. Writing support for most of functionality (well in fact all I need) was just matter of few days. The reason why it was so fast was that this phones is using open standards (e.g. OBEX, IrMC) and vendor specific AT commands are documented in freely available documents.

It's your choice how good will your phone interoperate with computer. If you buy well documented piece of hardware, chance to have it fully supported is much higher.

python-gammu 0.20 has been just released. New features:

• Fixed call information conversion.
• Support for last modified field.
• Support for latest Gammu version.
• Added python wrapper around C module, C module is now gammu.Core.
• Moved some generic data from Wammu.
• Added asynchronous wrapper around StateMachine.
• Added example for async operations, see examples/worker.py.
• Main library is now gammu.Core, but gammu.* still works.
• Support for service numbers.
• New example for service numbers, see examples/service-numbers.py.

I just released new version of Gammu, you can download it from download server. Changes to latest testing version are relatively small:

• Do not use asynchronous I/O on Windows.
• Make USSD work with AT phones.
• Fix setting date on Siemens MC75 (Rudolf Cejka).
• Config file on Windows is now searched in Application Data folder.
• Avoid corrupting vCard when some fields are empty.
• Properly handle OBEX error codes (bug #298).

Stay tuned, release evening will continue ;-).

python-gammu now supports service numbers dialogue. How it is useful for you depends on services your network provides. For example with Vodafone in Czech republic, you can use it to configure various services:

$ ./service-numbers.py 
This example shows interaction with network using service codes
Enter code (empty string to end): *111#
Talking to network...
Network reply:
Status: ActionNeeded
Vitejte v Kapesni Samoobsluze!
1 Muj ucet
2 Me sluzby
3 Nastaveni telefonu
4 Info a zabava
5 Oblibene

Enter code (empty string to end): 1
Talking to network...
Network reply:
Status: ActionNeeded
Muj ucet
1 Vyuct. a platby
2 Vydaje pod kontrolou
3 Osobni udaje
4 PIN a PUK kod
------
0 Zpet
00 Uvod

Enter code (empty string to end):

Python code which can be used for this is available as example. It will be probably one time integrated to Wammu, but it needs a bit more time than hacking this simple script :-).

If you want to follow SVN commits on some of my SVN repositories, you can subscribe to appropriate maling list, which gets notification on each SVN commit. I hope I set up mailman correctly and everything will work as I expect :-).

This list is also automatically forwared to packages.qa.debian.org, so you can also subscribe there for Debian package changes.

As phpMyAdmin themes for trunk are not compatible with 2.10.x releases, there was created 2.10 branch for themes compatible with those releases.

Today I added 2.10 themes to demo server and they are also available as subversion snapshots. I hope this will help fixing possible issues with them.

Yesterday I hacked asynchronous wrapper for python-gammu. It is start of progress to make Wammu behave better while talking to phone. My plan is that all operations will go through this asynchronous wrapper and will be completely non blocking to Wammu. This means no progress bar will pop up to front, but entries will appear in window as they are being read. Progress will be of course still shown, but only in status bar.

The current code is only first step, next thing which has to be implemented in python-gammu are virtual commands. Those will be used for things like getting all calendar events from phone, so all you will have to do for getting all is enqueue GetAllCalendar event and worker will do all for you including progress reporting.

If you want to write own application using this new wrapper, you can look at example which is in sources. It shows all current possibilities, but it will be enhanced in future.

I just spent some time on improving my website. Well I'm not graphic designer, so the site won't ever be according to latest trends and won't look sexy.

Anyway I tried to improve visibility of some things a bit. The main page now includes small screenshots of some programs I want to point out, small screenshots on application page are clickable and there is more visible download link on some pages as users complained they have problems finding this. While working with stylesheets, I also added maximal width of main content. This probably does not work with all browsers, but I don't care, the text is still visible.

Other thing I updated is information about last year talk about Gammu, which now contains links to videos. I haven't seen them yet, but they might contain interesting information for Czech speaking people :-).

Current Gammu versioning scheme confuses people a lot. For some time 1.x.0 versions are stable and 1.x.y are testing. However when somebody sees 1.10.5, he things this is updated version of Gammu 1.10.0. So it's probably time to move to something more obvious starting from next development cycle.

To stay as much compatible as possible with current scheme, I thing about minor change - so start testing releases on something like 1.11.90. I thing this is also quite often used (I've seen such versions in PyGTK in past) and nobody will confuse it with patch releases. Or should I switch to something completely different? I currently do not consider 1.11.0-rc1 as I'd like to keep version only numeric.

I just released new version of Gammu, you can download it from download server. The biggest change is improved gnappled and it's support in Gammu. It also had to became incompatible with previous versions, so you have to upgrade it in your phone. Full list of changes:

• Backup file uses ; instead of # (bug #85, patch by Marcin).
• Updated shell script gammu-config to generate configuration.
• Install gammu-config script to ease generating configuration.
• Correctly encode text in iCalendar (bug #262).
• Correctly encode recurrance in iCalendar (bug #262).
• Read last modified field from vCalendar.
• Support for last modified field from vCard.
• Use unsigned char to avoid trimming at 60 chars (bug #274).
• Properly handle vCard style continuation (bug #268).
• Detect Motorola specific error while reading SMSes (bug #288).
• Fix OBEX listing to behave according to specs (bug #290).
• Fix crash on invalid SMS reply from phone (bug #293).
• Improved gnapplet and support for it (Matthias Lechner).

I currently do not plan further testing release before next stable one, but in case some serious problem is found, this can change. This time really :-).

This was feature which I always hate - phone made strange sound when taking photo. Fortunately I'm not alone who hates this and there are several howtos on net which should guide you to do this.

The only way which worked for me at the end was replacing sounds in phone filesystem, so that it plays silent sounds. For Sony-Ericsson K750i with CID 49 BROWN you need specially hacked FAR from SE-tuning.cz and just replace files in filesystem (/tpa/preset/system/sound). Now phone magically makes no sounds when taking photos :-).

When introducing gettext to Gammu, I hoped that next release will be completely translated to Czech. Unfortunately there is too much to translate and too few free time. After hard work on translating, I got to 27%. That's not much.

Anyway it helped a lot in proofreading some texts and I hope I cleaned up a bit language used in Gammu. Also I introduced several calls to ngettext, what means that plurals in different languages should be now better. But this will only help, if somebody will be interested in translating Gammu. If you have free time, go ahead and improve translation to your language!

I just looked on server statistics and I was shocked by amount of spam and viruses rejected during last three days. Is there some new virus for Windows which tries random email addresses on all servers it can reach? I didn't analyse those messages, as they are rejected at RCPT time, because sender tries random addresses which do not work on my server. For now I try to reduce them by blacklisting in iptables, but this doesn't seem to help much as IP addresses change too often.

I think that graph shows best what big change has happened:

Update: It seems to be new Sober variant causing this mess.