awe – Page 2 – awe's blog

T-Mobile G1 aneb jak jsem k Androidovi přišel

Bylo nebylo, objevila se možnost mít za dobrých podmínek G1 od T-Mobile CZ. Sice jsem byl doteď relativně spokojený se Symbianovskou Nokii E66, ale rozhodl jsem se dát novému operačnímu systému šanci – buď budu spokojen a přejdu na Androida, nebo ne a pak zůstanu u Symbianu. Vlastně jsem o Androidu uvažoval už delsí dobu, ale pořád jsem čekal, až začne vyrábět takové telefony moje oblíbena Nokia. Léta ve vývoji mobilních telefonu jsou u téhle firmy znát a jejich “business” telefony řady Exx jsou nejen relativně hezké ale hlavně poměrně robustní a disponují velice slušnou výdrží na baterii.

V pátek dorazila černá G1. Po rozbalení jsem trochu zápasil s krytem, který se u nového telefonu sundavá docela špatně – než se trochu oběhá. Vložil jsem SIM kartu a baterii a dal telefon poprvé nabíjet. Asi po hodině jsem si na něj vzpomněl, zapnul ho a začal zkoumat nový operační systém. Musím konstatovat, že jsem byl velice mile překvapen – celý systém je poměrně intuitivní, návod není víceméně potřeba. Za největší rozdíly proti E66 bych označil:

relativně malý display s vysokým rozlišením (320×480)
trackball výrazně zrychlující ovládání telefonu
výborný GPS přijímač
rychlý “full featured” browser a podpora copy/paste libovolného textu
automatická synchronizace kontaktů a kalendáře s google mail

Naopak největší nedostatky spatřuji v:

menší než malá výdrž na baterie
chybějící podpora některých bluetooth profilů (zejména OBEXu)
mechanické provedení
rozměry a hmotnost

Tohle jsou moje poznatky z víkendu – stay tuned for more.

How to render Dzongkha texts in Swing properly, take II

Some time ago I wrote an article about problems with Dzongkha (Bhutanese) texts in Swing environment. At that time I thought it was fixed for good but nothing was further from the truth…

We’re not living in an ideal world where everything just works and if not it can be fixed in no time. Not at all. Some time ago I wrote an article about problems with Dzongkha (Bhutanese) texts in Swing environment. At that time I thought it was fixed for good but nothing was further from the truth.

While rendered texts looks all right on first sight we later found out that some of the characters were not rendered properly. Sometimes. I did more research and came to conclusion that characters composed from more “strokes” seem to be put together in a wrong way. But – properly rendered combinations do exist in fonts as one character too – so you could’ve two similar characters, one apparently correct and the other one fault, written next to each other in the same editor.

As OS/fonts are usually the usual suspect in those cases, I started investigation there – but it was a dead end. Even notepad/kate rendered composed characters properly. I went more low-level and tried to render it using WinAPI calls – again flawless. At this moment I was thinking about writing a native library for font rendering, but I’d need to have three of them (Windows, MacOSX, Linux) and it’s not exactly piece of cake. As a last experiment I tried to render it in SWT – it worked. Well then, I thought, this is going to be PITA as well, but still better than native library.

I spent two days analysing SWT and Swing sources, looking for differences, trying to find real issue behind Swing’s unability to render texts properly. And guess what – I was successful.

For tests I’ve selected character composed from four parts – 0x0f62, 0x0f92, 0x0fb1 and 0x0f74 (ICU browser), if you have Tibetan script support you should be able to see it directly as རྒྱུ. If not see screenshots in next paragraph.

Following image illustrates the issue – on the left there’s a properly rendered character, on the right bad one. Unfortunately the bad variant is what you get if you just put it into JTextPane.

left_or_right

On top of that I found out that none of Jomolhari fonts works in Java properly. The only font able to render my test character properly was Tibetan Machine Uni.

So how do you get properly rendered Tibetan script in Java? There are more steps required:

download and install Tibetan Machine Uni
edit fontconfig.properties as described in first article about Dzongkha How to render Dzongkha (Bhutan) texts in Swing properly using Tibetan Machine Uni instead of Jomolhari
add property “i18n” with value of true to every JTextComponent’s Document you want to display Tibetan script properly

To give you some basic idea here is the code used to produce image above:

package dzongkhatest;

import java.awt.Dimension;
import java.awt.Font;
import javax.swing.BoxLayout;
import javax.swing.JFrame;
import javax.swing.JTextPane;

public class DzongkhaTest {

    static class Frame extends JFrame {

        public Frame() {
            setTitle("Dzongkha rendering test");
            
            getRootPane().setLayout(new BoxLayout(getRootPane(), BoxLayout.X_AXIS));

            JTextPane pane1 = new JTextPane();
            pane1.setFont(new Font(Font.SANS_SERIF, Font.PLAIN, 100));
            pane1.getDocument().putProperty("i18n", true);
            pane1.setText("\u0f62\u0f92\u0fb1\u0f74");
            getRootPane().add(pane1);

            JTextPane pane2 = new JTextPane();
            pane2.setFont(new Font(Font.SANS_SERIF, Font.PLAIN, 100));
            pane2.setText("\u0f62\u0f92\u0fb1\u0f74");
            getRootPane().add(pane2);

            setSize(new Dimension(200, 200));

            setDefaultCloseOperation(EXIT_ON_CLOSE);
        }
        
    }
    
    public static void main(String[] args) {
        new Frame().setVisible(true);
    }

}

This solution seems to work fine on Windows and Linux. Nothing is perfect, however – it fails miserably on MacOSX – every composed character is shown as it’s parts one next to each other;(

Flashing Nokia E66 from VirtualBox

For last couple of years I don’t have Windows as main operating system. There are many reasons behind this decision, one for all would be it’s reliability. While using different operating system as the main one, I still keep couple of Windows virtual machines, I need them for testing of our software as well as for running applications, that can’t run anywhere else. Such as Nokia PC Suite.

I bought my Nokia E66 as one of the first people here in Prague. I had to wait for branded T-Mobile version because it was part of RMA transaction of my old E65. From the time I got my new E66 I was waiting for first firmware update. There were few annoying bugs, bugs that old E65 didn’t have. But as this was a branded version, update took quite a long time to appear in Nokia’s firmware update service. But it happened this week.

Having used PC Suite for backups in virtual machine I though firmware update would be piece of cake; it wasn’t. If you have ever used VirtualBox you probably know that there a device filter has to be added in order to “see” USB device inside virtual machine. Normally this is done on the fly – you connect a device, go to configuration, add it to filters, disconnect, connect and voilà…

In order to flash E66 successfully, two devices have to be added before flashing starts. In theory you could add them on the fly, but I’d have to be quick in order to keep flashing process running. I did this risky stuff for you so you can now flash safely having added these two devices to your Windows machine’s USB filter:

0421:0106 0421:0105

One of them is serial device of some kind, the other identifies itself as loader. Have fun!

Logging and OOME

Following text comes with limited warranty – I’m not saying that what we did was the best solution to problem but it was the simplest one. At least we thought so.

We recently replaced our in-house logging for something more “standard”; log4j-based based logger. The original code developed by out former employee was a mixture of spaghetti code and huge amount of non-threadsafe WTFs. We spent some time fixing it but finally we’ve decided to replace it. In order to incorporate our additional features that had been part of the original logger, we extended java.util.logging.Logger. So far, so good; but then the real hell started. There comes the first mistake one should learn from – never ever replace anything that works with something that hadn’t been even tested properly.

After a while with new logger we noticed our server application started to die with OutOfMemoryException. As there was a lot of changes made everywhere no one really suspected logger. But nothing could be further from the truth; wrong usage of ByteArrayOutputStream which was never reset() caused the trouble, we found that after couple of hours of profiling as no one would suspect “runtime” classes to eat the memory.

Having fixed something so obvious we thought we were done. Well, not really. Server did not crash in two days, it went down after a week. Same exception, same source – new logger.

In order to explain the situation I have to start with out new features. We wanted to have similar log files separated by given parameter, all our topics are members of huge enum, some of the values can have an annotation specifying destination filename and default parameter. If such parameter is present in log record, it’s written in file based on given data; this helps us to sort data according to their source, even if log entries are created by the same code; quick example will explain it faster – imagine application reading data from email accounts – such an application would log with our logger to N+1 files named:

application.log application_HOTMAIL.log application_GMAIL.log ...

I guess you got the idea. In order to do that we needed a place where those parameters are stored and instances of java.util.logging.Logger class seemed as a logical place. We extended this class and our implementation added support for those additional data.

Simple, but… there comes the catch. Loggers live in hierarchical structure, each logger has a parent. Whenever new logger is created it’s assigned to a parent (parent could be root logger, of course). Each logger keeps its children in a java.util.List, see following excerpt from Sun’s Logger.java:

    private Logger parent;    // our nearest parent.
    private ArrayList kids;   // WeakReferences to loggers that have us as parent

    public void setParent(Logger parent) {
	if (parent == null) {
	    throw new NullPointerException();
  	}
	(...)
	doSetParent(parent);
    }

  private void doSetParent(Logger newParent) {
	synchronized (treeLock) {
	    // Remove ourself from any previous parent.
	   (...)

	    // Set our new parent.
	    parent = newParent;
	    if (parent.kids == null) {
	        parent.kids = new ArrayList(2);
	    }
	    parent.kids.add(new WeakReference(this));
	}
    }

There are weak references, good. But wait – who’s cleaning up kids collection? Nobody? There we go…

This class is a joke. All important fields are private, it’s not implementing any interface, there is no simple way how to fix the problem. As a logger I wanted to get rid of myself when I’m finalized – no way, you can’t even call setParent(null), it’s gonna throw an exception and fail to do anything else. Saddest thing of all is that kids collection is not used anywhere for anything reasonable – it’s just sitting there eating heap. And you can’t do anything about it. Damn you, Sun.

How to render Dzongkha (Bhutan) texts in Swing properly

As you might know we’re developing a specialised information system. It’s been widely used in Asia, in fact one of our first international customers was (and still is) located in India. Thanks to this geographical variety we’re facing issues with exotic language/script support quite often. Last time we had an issue with Urdu script, that’s been used in Pakistan. It’s still not supported in mainstream operating systems (first support came with Vista/Office2007), a special commercial software is needed for that, it doesn’t to seem to adhere to Microsoft (or any) standards, it sort of works but there are still issues. The biggest problem with Urdu is it’s height which can really vary depending on what’s written

Yesterday I was asked about Dzongkha support. Dzongkha is the official language of the Kingdom of Bhutan. It’s using Tibetan script for written form. The good news is that there are free fonts available. I installed one of them (Jomolhari) and started experimenting with it. Just to run into complete failure – no part of the system was able to show anything else than “boxes” – at least on Windows I started with. There were only two parts of the system being able to show Dzongkha properly – but only thanks to their ability to change font used for them.

Imagining how complicated/annoying it would be to change all Swing controls to use Jomolhari font (ie. adding configuration, making sure that setFont() is called on every single JComponent, testing it….) made me feel really bad. On top of that latin script rendered by Jomolhari was really ugly (especially in small sizes like 10, 12pts).

I remembered that there was a way how to specify font substitutions in Java fortunately and after couple of minutes I got the right article http://java.sun.com/j2se/1.5.0/docs/guide/intl/fontconfig.html on Sun’s website.

I started reading the document but even after finishing I was not exactly sure what should be done – there’s a paragraph about substitution, about character subsets but – Tibetan, Dzonghka was not there. Having studied Unicode pages I knew that Tibetan characters are occupying their designated range starting with 0x0F00 yet there didn’t seem to be support for this subset. I tried adding line with:

sansserif.plain.tibetan=Jomolhari

but it didn’t seem to change anything, boxes again. While getting ready to leave the office I got one last idea – what if China somehow owns Tibetan range? They keep claiming that Tibet belongs to them, so why not the Unicode range too? So I changed line with Chinese subset:

sansserif.plain.chinese-ms950=MingLiU

to Jomolhari font

sansserif.plain.chinese-ms950=Jomolhari

and – voilà, there we go – it worked as a charm. But – what if this disables Chinese, I thought. Quick test revealed that my worries were needless, Kanji characters worked too. And latin as well, rendered in usual quality.

So, what exactly is needed? Just follow steps below:

Install appropriate fonts (ie. Jomolhari)
Locate fontconfig.properties.src ($JAVA_HOME/lib)
Rename it to fontconfig.properties (remove .src suffix)
Replace all occurrences of .chinese-ms950=(P/MingLiU) with .chinese-ms950=Jomolhari
Quit all Java processes and start your Swing application again
Every single Swing component should be capable of displaying Tibetan script properly – if not check if you’re using “logical” fonts and not forcing physical ones (there are five logical fonts, read the Javadoc for Font class)

Having solved problem on Windows I moved to Mac – just to see that on MacOSX Leopard (10.5.6) fonts for Tibetan are installed by default and working in Java out of the box. Good work, Steve.

JTextPane and “continuous” cursor

Well,
it’s not easy to work with Swing components. I’m working on a special editor, it consists from N JTextPanes glued together. They resize according to amount of text they have inside on order to avoid scrollbars. So far it wasn’t so complicated. But now I got complicated – I wanted to be able to “walk” freely through the JTextPanes – using only keyboard arrows.

First I implemented jumping between current and next/previous with TAB key, this was not a big deal. What’s the problem with arrows? Well – in order to have it working as expected you need to be able to find out whether you’re at the last line or not. One would guess that reading document structure would work but if you have lines longer then JTextPane width the “internal” lines and visual ones differ. One “internal” line could split into multiple visual lines.

In cases like this I simply go and study – study the libraries – this kind of check needs to be somewhere, right? So we only have to find it. After half an hour of searching I found something that could work:

int newpos = text.getUI ().getNextVisualPositionFrom (text, pos, Position.Bias.Forward, direction, bias);

(complete reference for UI is here:
http://java.sun.com/…/MultiTextUI.html)

text is JTextComponent, pos is current position, Position.Bias defines the direction (back/forward), direction is int defined in SwingUtils (SOUTH, NORTH, etc…) and bias is Bias[1] returned back. I don’t care about Biases – I just call it with desired direction and get new position. And if it’s the same like the original one we probably hit the border. Voila!

WiFi is not always the easiest way…

We recently moved into new offices. One of the problems we had was internet connectivity – we rely pretty much on stable and fast connection and our good old provider (UPC) was able to connect us on new location. Fortunately there is a fiber optics in a building nearby (480m / 1440ft). Logical solution to this kind of problem is WiFi technology, right? We wanted to have something stable so we went for 802.11a (aka 5Ghz).

Next important step was to choose the devices, having read almost every resource available and being recommended it we’ve decided to try Chinese OEM devices made by Wistron NeWeb Corporation (WNC) – namely model called RDAA-81 which is, for some unknown reason, called AirCA8-PRO in Europe. They are quite cheap, the price is around $100 per one (including VAT).

One day it took us to mount antennae to both ends, it was quite easy on the “fibre optics” building – it’s just behind the window while it took some time on our office bulding, we had to use long cable from the antenna to device, drill several holes etc.

Next day I came with the devices preconfigured for wds bridge, which is basically a combination that should behave in a transparent way like normal ethernet cable. We connected everything and it started to run. It ran for half an hour and then it suddenly dropped and didn’t come up again. I connected to the other end through the internet, reseted it and it started to work. For twenty minutes.

At this point I’d like to point at another bad aspect of those devices – they’re booting 3.5 minutes. WNC developers must’ve been joking! Next time even web administration of the device was dead. It was during Saturday and I didn’t have the keys to the room where the device was.

I searched the internet and found some people having same problem with wds mode hangups, so I’ve decided to reconfigure devices to standard accesspoint – client setup. I had to wait till Sunday until someone was able to reset it by pulling it out from the wall. For this setup I had to flash another firmware because the original one is accesspoint/wds only. I also had to login first through telnet interface and issue something like z_debug signature disable, the firware recommended by reseller is apparently meant to run in another hardware 😉

New setup was somehow better – now the link seemed to be more stable but when I tried to ping router behind the LAN interface it only worked for some time and then there was each time same delay of 40 seconds when I was not working. I also tried to ping internal IP on WiFi interface and that worked. To make long story short – this client firmware works fine until it receives ARP who-has from device with another MAC. When this happens it stops working until ARP with MAC of gateway (or what you’re communicating with) comes or delay of 40 seconds ends.

In client mode I was also able to see signal strength (this is not available in wds mode, there is no way how to check if the devices are associated to each other!), scanning for all accesspoints took around 3 minutes, however.

At this point I was ready to rma the devices and buy something more expensive but working.

But I’ve decided to give it a last chance – with OpenWRT firmware. Standard OpenWRT doesn’t support those devices but I found a fork just for them – Atheros OpenWRT managed by coincidence by someone from Prague (binaries are at http://openwrt.dlabac.net/snapshots/atheros/).

Again the same problem with getting into locked room at the other side but on Wednesday both devices were flashed and connected together in simple, not encrypted, link. It took us some time to talk to developers and configure it propertly- apparently it’s newer version of OpenWRT which is not documented on the web – all the configuration files are different etc.).

We had a link running but no encryption, which is simply unacceptable for “business” use. If there was nearly no documentation for “standard” features there certainly was zero documentation for WPA.

Fortunately I wasn’t linux newbee and moreover I was experienced in wireless extensions and specifically hostapd (I helped to build wireless network in our neighbourhood, it was/is still running mostly on 802.11b, majority of accesspoints are linux machines with hostapd running:) ).

So even without any documentation (and support in configuration files) I managed in three hours to get WPA running. Unfortunately things like that can not be just configured, one has to write his own starting scripts for hostapd on accesspoint side, wpa_supplicant on client side and static routes (if needed).

Pros:

OpenWRT boots faster (30 seconds compared to 3.5 minutes)
fully configurable – it’s standard linux (2.4) with ssh
it’s stable compared to the original firmwares

Cons:

way too much time spent on the project

Beware of null #TEXT nodes

I recently stumbled on NullPointerException in Transformer.transform (Source, Result) method, my application was working fine until yesterday, there was no update, no change in configuration of anything…

What I was doing there is a textbook example of usage of Transformer class:

	    Source source = new DOMSource (doc);
	    Result result = new StreamResult (new File (file));
	    Transformer transformer = TransformerFactory.newInstance ().newTransformer ();
	    transformer.transform (source, result);

Transformer is a part of JVM and it’s used mostly for saving Document to file, so first thoughts were to check the parameters passed. Yes, there was a valid path inside File, canWrite () returned true, the file was created indeed (albeit having zero size) but that was all. The only message I got came to stderr and read:

ERROR: ''

Very helpful, isn’t it? What next? I went to check the stacktrace:
javax.xml.transform.TransformerException: java.lang.NullPointerException at com.sun.org.apache.xalan.internal.xsltc.trax. TransformerImpl.transform(TransformerImpl.java:651) at com.sun.org.apache.xalan.internal.xsltc.trax. TransformerImpl.transform(TransformerImpl.java:281) at xxx.Main.writeDocumentToFile(Main.java:151) at xxx.Main.main(Main.java:332) Caused by: java.lang.NullPointerException at com.sun.org.apache.xml.internal.serializer. ToUnknownStream.characters(ToUnknownStream.java:312) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:229) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:215) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:215) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:215) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:121) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:85) at com.sun.org.apache.xalan.internal.xsltc.trax. TransformerImpl.transformIdentity(TransformerImpl.java:596) at com.sun.org.apache.xalan.internal.xsltc.trax. TransformerImpl.transform(TransformerImpl.java:642) ... 3 more --------- java.lang.NullPointerException at com.sun.org.apache.xml.internal.serializer. ToUnknownStream.characters(ToUnknownStream.java:312) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:229) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:215) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:215) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:215) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:121) at com.sun.org.apache.xalan.internal.xsltc.trax. DOM2TO.parse(DOM2TO.java:85) at com.sun.org.apache.xalan.internal.xsltc.trax. TransformerImpl.transformIdentity(TransformerImpl.java:596) at com.sun.org.apache.xalan.internal.xsltc.trax. TransformerImpl.transform(TransformerImpl.java:642) at com.sun.org.apache.xalan.internal.xsltc.trax. TransformerImpl.transform(TransformerImpl.java:281) at imdbparser.Main.writeDocumentToFile(Main.java:151) at imdbparser.Main.main(Main.java:332)

Fortunately I’m not afraid of going deep into the code, so I grabbed sources and checked the line where everything got started:
Caused by: java.lang.NullPointerException at com.sun.org.apache.xml.internal.serializer. ToUnknownStream.characters(ToUnknownStream.java:312)

the code there looks like this:

    public void characters(String chars) throws SAXException
    {
        final int length = chars.length();
        (...)

Gotcha! The only possible cause of that is chars being null, fine but… why? Previous line in stacktrace finally helped to resolve the problem:

case Node.TEXT_NODE:
	    _handler.characters(node.getNodeValue());
	    break;

So there’s one or more #TEXT nodes in the document having null value! It was quite easy to find the problem having this information, it was on following ternary expression:

m != null ? m.name : getMName (files[i].getName ())

At first sight it looks good – there is a test on null value but what if mis not null and m.name is?

The Owls Are Not (always) What They Seem (Twin Peaks)

Začínáme …

Ahoj,
tak jsem se konečně dostal k založení vlastního blogu. Plánoval jsem to už docela dlouho (doslova pár let) a teď na to došlo. Proč blog? Za poslední dobu se stalo hodně věcí a ne vždycky jsem je chtěl x-krát vyprávět, na “sdílení obsahu” je blog ideální. Druhým důvodem je, že si potřebuju občat odkládat texty a odkazy do nějakého “zápisníčku”. A i tady se blog bude hodit.

Drobná schizofrenie panuje u výběru jazyka – z profesních důvodů chci psát anglicky ale současně mám i potřebu používat svůj mateřský jazyk. Pro začátek jsem rozdělil kategorie podle jazyků a počítám, že v té anglické bude spíš technika a počítače, zatímco v té české kultura, tedy muzika, filmy, knížky. Česky se budu snažit psát s diakritikou (ač mne to stojí hodně přemáhání).

Abych nezapomněl – přeju vám všem úspěšný rok 2007.

Starting …

Hi there!

Something finally pushed me (and I still don’t know what it was) to realize my idea I’ve been having for last couple of years. I’m starting my blog – now, when blogs are slowly going out of the fashion 😉

I’ll try to write about something I understand – computers – and about other things I like. The later is mostly music, books and photography.

I also have to apologize for little schizophrenia at the beginning – this blog is going to be mixed Czech and English, I’m from Czech but due to my profession I adopted English as secondary language and … well some things can’t be translated 😉

I almost forgot – Happy New Year 2007 to you all.