OpenGL 4.0 — am I allowed to implement it or not?

Reading the second page of the opengl version 4.0 specifications www.opengl.org, I thought something was strange: the first paragraph state

(…) You may use this specification for implementing the functionality therein, without altering or removing any trademark, copyright or other notice from the specification, but the receipt or possession of this specification does not convey any rights to reproduce, disclose, or distribute its contents, or to manufacture, use, or sell anything that it may describe, in whole or in part. (…)

Emphasis added by me. Notice the first part of the bold text. It clearly states that I (or anyone else) am allowed to implemented the functionality, but the rest of the bold text clearly state that I am not allowed to manufacture anything that it describes — implementing the functionality is manufacturing what the specifications describe. So which is it? Am I allowed to implement it or not? This may seem like a trivial issues — of course I am allowed to! — but the problem is that someone might, some day, decide that anyone who has implemented OpenGL, in full or in part, is violating copyright and those people (like, for example, one of the Khronos group’s voting member who would have interests in shutting down OpenGL) might start suing them for (imaginary) damages.

The Khronos group should change this ASAP to state the obvious: the document itself is protected by copyright, but you can use it to implement OpenGL 4.0 if you want to without violating any copyrights.

Free nVidia drivers on Fedora 13

I have been running Fedora 13 on my computers since the day it was released, but have only just now realized that the experimental drivers it includes for nvidia gpus actually work. I initially thought it was just 2d that was accelerated and everything was done in gallium (it’s based on gallium ) fallbacks (which AFAIR is not technically correct, but I can’t quite recall what the technical correct term is) when the driver doesn’t have it’s hardware accelerated implementation ready. This was based on running glxgears and comparing the result with what I have gotten with the blob from nvidia — where the nouveau driver (the free one) is outperformed by a factor 10 to 1 (that is nvidia does 10 times better than nouveau). Which just shows how good a benchmark glxgears is (it’s not a good benchmark). Running a few (World of Padman and OpenArena) free games with a 3d engine, I got two working which had performance in the 90fps-100fps range with everything set to the best quality (highest resolution, best filters, antialiasing, etc.). The only game I couldn’t start was Tremulous, for reasons unknown. I tried to compile the game myself but the Makefile doesn’t include an install target and I’m lazy, so I haven’t been able to test it.

Anyways — thumbs up to the guys working on nouveau. It certainly did pay off to give them all the data I collected for them through out the upgrades.

Why I will not be getting a SPARC based computer

I like SPARCs. I find them to be far superior to the x86 processors on, at least, a theoretical level, if not also on a practical level. Having said that I will likely never get one, unless a meteor hit me in the head with one. The reason is simple — they are, by far, too expensive. The cheapest SPARC based computer I could find anywhere costs $14,750.00, which is about 4 times more than I would have to pay for a computer I could assemble using off the shelve components (and not the cheapest ones), resulting in a computer which is superior to the SPARC in every way. The SPARC I’m refering to is the cheapest one from SUN (or is that Oracle now-adays?): Listing price is $14,795.00, which buys you one (1) 2.75 GHz SPARC64 VII 2-core processor with a total of 4 threads, with 8 GiB of RAM, a total of 292 GB of HDD on two drives @ 10k RPM and a few other things (including Solaris 10 — but who cares?). I’m assuming some sort of support plan is included — if not, I can’t really justify the price at all and would consider this theft at broad daylight.

The system I propose as a superior replacement consists of the following components: two (2) Quad core, hyper threaded Xeon L5630 @ 2.33 GHz. This is slightly slower (in hertz) than the SPARC64 VII, but on the up side, there are two of them, and both have twice as many threads as the SPARC, for a total of 4 times as many threads. This won’t cost you more than $1500 total. The motherboard is an Intel Server Board S5500BC, which should cost you about $400 — this lacks a SAS controller, so you will have to get one. A SAS controller can be had for about $150. Faster, bigger drives with more cache can be had for about $200 per drive — if you want to use them for RAID 1, which is what I would recommend, that will be a total of $400 or so. 2 GiB sticks of paired DDR3 RAM modules running @ 1800 MHz can be bought for $150 per pair — a total of 8 GiB costs $600. The only thing missing is a case and a power supply — the case can, again, be had for about $100-$150 (I’m assuming $150) and the power supply will cost about $300 — the total is ($1500 + $400 + $150 + $400 + $600 + $150 + $300) $3500, or about a quarter of what the SPARC was listed at. Personally, I would much rather have four servers than one, and if those four servers are all better than the one server I could get otherwise, I would definitely pick the four servers.

Components I used to find the price:

Intel Xeon L5630, Intel Server Board S5500, Kingston HyperX 2 x 2 GB @ 1800 MHz DDR3, Fujitsu 450 GB (SAS HDD @ 1000 RPM), LSI MegaRAID SAS (SAS Controller), Ikonik Zaria A10 (case), OCZ Z Series 1KW power supply.

Public const, private non-const variables in C++

Have you have thought to yourself that you wanted to provide a readable version of a variable in the public area of a class in C++, but without providing a getter function? Well, that’s not possible, since you an only grant full access or no access to a variable, but you can do a hack that makes it look like you haven’t provided a getter function — the key is to use operator overloading.

The concept is something similar to this:

class example {
private:
int c;
public:
const int c;
};

The idea is to make c fully accessible in the example class, but only readable in the public. There are a few options to doing this — the union way and the other way.
The union way has the nice advantage that it doesn’t use extra memory (and it can even be combined with the other ways) but it’s not always a good option because you cannot inherit from unions nor can unions inherit anything. Unions therefor are only usefull if you are writing a class that you know will never be derived.

The union way would be:

union example {
private:
int c_;
public:
class { operator int() { return ((example*)(this))->c_; } c;
};

Reading example::c will return the value of example::c_, but doesn’t allow example::c_ to be written to. The union way is simple and neat. It doesn’t mess up readability of the code a lot. The only reason why it’s not perfect is that you can’t inherit from a union or let classes/structs/unions inherit from unions. While I’m still at unions, you can imbed a struct in the union if you need more private variables, like so:

union example {
private:
struct { int c_; int b_; };
public:
class { operator int() { return ((example*)(this))->c_; } c;
class { operator int() { return ((example*)(this))->b_; } b;
};

This will do the same as above, only with two members.

A more extensible trick, which is also far less readable and a lot more convoluted, has to be used if you want something you can derive from. I haven’t fully explored the effects of this yet (such as what happens if you derive from it), but here goes:

class example
{
unsigned p_;
public:
class p_traits {
example * o;
public:
p_traits(example * e) : o(e) { };
operator unsigned() { return (o->p_); }
} p;
example(int i) : p_(i), p(this) { };
};

This is not nice code, and it can be made nice, but then it will be a lot more convoluted and less readable (but has more features).
The code is a single class which has a single unsigned private variable p_ and a public member p. p is, just like the example above, readable from outside the class, but is only writeable inside the class. I have not found a way to have several readable entries but without having full object overhead. The real problem however is that you must have a pointer to the owner of the class, which, to me, is a waste (this is why I prefer the union way in all the cases I can do it).

Both these techniques can be expanded to allow assigning values to the public portion or doing anything to them really, such as calling doing calculations or sending a message to a server. It could for example be used to change the size of a window in a graphical environment in a more “natural” way than the regular window.resize(100,100) — instead you could do window.width = 100, window.height=100. This is, of course just bells and whistles and doesn’t make the language more powerful, but I do like the idea of setting values, rather than calling functions (even if there’s no real difference) — it feels more object oriented this way.

Oh, and I don’t use the above non-union way exactly. Instead I have the p_traits class outside of the class which inherits from a common base class. The base class is a friend of the owner class and so the traits can access anything they want to.

HTC Hero or why Android doesn’t (really) matter

I got an HTC Hero in December — it’s a nice phone. I like it. I knew that Android 2.0 would be out not longer after I got my phone, so I checked the web to see if it would be possible to update the phone to the latest version (since frankly Android 1.6 is not that big a deal). Lo and behold — HTC promised that the Hero would get an update in December, so I figured it would be out in January or February. The update still hasn’t arrived and Android is now up to 2.2. HTC has, in the meantime, promised to provide 2.1 updates, but I’m not holding my breath for any update from them.

The main rant of this post however is not so much about the Hero or HTC, but more about Android and specifically why it won’t matter.

The Android operating system is in many ways similar to Windows. No, it doesn’t run your favourite first person shooter or Microsoft Office, and that’s not what I’m talking about. Android is an operating system and nothing else. It is not a list of hardware specifications, it doesn’t contain any details on how things are booted or how memory is managed or accessed the processor. It doesn’t say anything about how discs are organized or — and this is the important bit — how to install a new version of it. In those ways it is very similar to Windows.

Android cannot simply be installed on a phone or a device, and this one thing is the one thing that is lacking for Android to really matter. Since I cannot, in a simple way, brew my own version and install it on my phone, it doesn’t matter if it’s Android or Windows Phone 7 Series (or what ever permutation of those words make up the name of the latest version of the worst phone OS out there). Using Linux as the kernel specifically doesn’t matter for a number of reasons, the most obvious being that I, as a programmer, have no way of communicating with kernel.

If Android included a boot mechanism it would fix all the problems at once — but being that it’s google who’s developing the operating system it’s not likely that we will ever have that.

Kernels for 32 bit x86 — compiling the kernel

This is the second part of my introduction on kernels for 32 bit x86 machines. In this part I will be looking into how to make a kernel, which takes advantage of a boot loader which complies with the multiboot specifications as defined by GNU[1]. This allows us to avoid messing about with moving into 32 bit mode and clears the interrupt flag as well as a few other things — it will also allow us to do a few other things with the hardware before the kernel starts, but that will be something I will not be looking into now. Before I go into the kernel itself, I will be looking into how to build the kernel, which to me, was far more troublesome than writing the kernel. I will, of course, still be using GCC 4.4.2 (I believe any version will do — I’m not doing anything fancy) and make. Using the built-in rules of make the end result is this:

OBJECTS = kernel.o boot.o
CFLAGS = -Wall -ffree-standing -pedantic -m32 -std=c99
ASFLAGS = $(CFLAGS)
LDFLAGS =  -melf_i386 -Ttext 1000000

kernel: $(OBJECTS)
	$(LD) $(LDFLAGS) $(OBJECTS) -o $@
clean:
	$(RM) *~ *.o kernel

Simply save that to the file Makefile

and it should be able to compile the kernel (that I am about to supply the code for. About the Makefile: The makefile contains three symbol definitions at the top CFLAGS, ASFLAGS and LDFLAGS. The first two (being identical) are used when compiling C source (CFLAGS) and assembler source (ASFLAGS). The final symbol is used when the kernel is linked. The CFLAGS symbol contains flags for the C compiler, which tell it that the source code is not to be linked to anything other than what is explicitly given — this is what -ffree-standing is for — and is by far the most important flag given. It is this flag which allows the kernel to run at all. The flag -m32 is only important if you are compiling the kernel on a x86-64 operating system. If you’re compiling on a i386 operating system it won’t make any difference and on any other machine the compile will utterly fail if you have it or not — in that case you need a cross compiler (which is far beyond the scope of this article, yet a simple thing to get done once you know what you’re doing). The last couple of flags are ultimately unimportant, except that -std=c99 and -pedantic changes how we can express things in the source.

The kernel it builds is the sample kernel provided in the multiboot specifications — it is a simply thing that simply prints some information to the video display and then halts the computer. It contains three files: multiboot.h, boot.S and kernel.c. The files can be found here. The file is a gzip compressed tarball of the four files specified here. I have set up and environment where the commands to build the disk image explained in the first example has been compiled into a makefile which allows me to erase the image, recreate it, compile the kernel, copy it onto the disk and boot it with Qemu with one command. The makefile in question contains the following commands:

QEMU_FLAGS = -vga std

boot: disk.img
	make -C kernel
	sudo mount -o loop,offset=16384 -t ext2 disk.img mnt
	cp kernel/kernel mnt
	sudo umount mnt
	qemu $(QEMU_FLAGS) -hda disk.img

commit:
	SVN_EDITOR=vim svn commit

clean:
	make -C kernel clean
	$(RM) -r mnt *~

disk.img: grub.input parted.input mnt grub.conf
	qemu-img create disk.img 20M
	sudo parted disk.img < parted.input
	sudo mount -o loop,offset=16384 -t ext2 disk.img mnt
	sudo chown `whoami` mnt -R
	mkdir mnt/grub -p
	cp /boot/grub/stage{1,2} /boot/grub/e2fs_stage1_5 mnt/grub
	grub --device=/dev/null < grub.input
	cp grub.conf mnt/grub
	sudo umount mnt

mnt:
	mkdir mnt

The files grub.input parted.input simply contains the commands given to the shell as specified in the first part.

That’s it. You don’t need anything else to compile a kernel for 32 bit x86 computers. For the next part I will be dissecting the source for this minimalistic, and ultimately, useless kernel.

Kernels for 32 bit x86

The following is a short introduction to writing kernels for 32 bit x86 computers. I have taken up the topic over the weekend and have found that most of the introduction and documentation I have been able to get my hands on and lacking on the most basic things — it seems they expect you to know everything before you start learning it. Some of the details in here might be a bit too low for some — they were added for completeness and to make sure I don’t leave anything out that might be vital to someone.

This part will guide you through setting up the environment which allows you to test the kernel.

I expect the following things from you and your operating system.

You:

• You know some basic C — you don’t have to be an expert, but it helps.

If you know x86 assembler, it helps too, but since we won’t be doing a whole lot of x86 assembler it won’t hurt a lot if you don’t know any. Besides, we’re doing this in GNU as, so chances are you’ll be confused if you are used to use {M,N}ASM or anything else that’s not AT&T syntax.

The following software is installed on your computer

• GCC (I’ve only tested this with GCC 4.4.2 on Fedora 12)
• Make
• Qemu
• binutils (a linker and an assembler)
• parted
• mount and umount

Your tool chain must, ofcourse, be able to produce 32 bit x86 ELF files. I have no idea if you can get all of that installed on a Windows machine, so if you don’t want to run a GNU distribution (on a real or virtual machine) then you may not be able to build and test the kernel.

Parts you need

To build and test the kernel you need the following things

1. A boot loader
2. A kernel
3. A disk to boot from

The bootloader of choise in this case is GRUB legacy (I plan to move to GRUB 2 at some later point — if I ever get around to it, this article will be updated). You can’t install it before you have the disk image however, so that’s the first thing to do.

Creating the disk image

The disk image doesn’t have to be large — for the kernel I am describing here, a few hundred kilobytes will suffice. I’ll be making a 20 MiB image however. The command to do so is

qemu-img create disk.img 20M

This command will create a 20MiB file filled with zero bytes — in order for it to be usefull a partition table has to be added and a partition with a usefull filesystem has to be created. To do both, parted is used. When parted is running it will open a shell where the user can enter commands — the commands (in order) to enter are:

mklabel msdos
mkpartfs primary ext2 0 20
set 1 boot on
quit


The first command (mklabel msdos) creates a label on the disk. The second command (mkpartfs primary ext2 0 20) first creates a partition and then installs an ext2 filesystem on the parition. GRUB can now be installed, but to be able to boot on it, the third command has to be run — this makes the disk bootable. The last command simply exits parted.

Installing GRUB Legacy

To install GRUB you need to copy a few files to the disk image — in order to do that, you must first mount it. On non-UNIX like operating systems, this is likely to be very tricky — I know of no way of mounting the filesystem if mount is not present. As always you will need root access (or an entry in /etc/fstab) to mount the disk image. The command for mounting the filesystem — which does not reside directly at the beginning of the file, you need to specify an offset, for where the filesystem begins. In this case (because it’s simple) the filesystem begins after the first 16KiB. The command for mounting the filesystem is

mount -o loop,offset=16384 -t ext2 disk.img mnt


where mnt is the directory you wish to mount it on (I just use a local directory relative to the disk image). Umounting is a simple matter of running umount the usual way:

umount mnt


On the disk image, you need the directory grub to store the files GRUB need to boot.

mkdir mnt/grub

Now copy the files to the image — you need ext2 support and stage 1 and 2 files.

cp /boot/grub/stage{1,2} /boot/grub/e2fs_stage1_5 mnt/grub


The ext2 file may have a slightly different name on some systems — likely something that makes more sense (try ext2fs_stage1_5). The next step is making grub the bootloader on the image. To do so, run the following command

grub --device=/dev/null


When grub is running it will (like parted) open a shell for you to enter commands in. The commands you need to enter are:

device (hd0) disk.img
root (hd0,0)
setup (hd0)
quit


The first command maps the first BIOS disk to be disk.img — this is required for grub since it depends on the BIOS mapping of disks. The second command sets the root to be the first partition of the first disk (the partition you created earlier). The third command installs grub to the first BIOS disk (that is the image) and the final command stops grub.

Configuring GRUB

You should now have grub installed on the image. If you boot it with qemu, grub should start up and offer you a shell. In order to make grub boot the kernel (which has yet to be made and copied to the image) create the file mnt/grub/grub.conf and let it have the following content

timeout 10
default 0
title kernel
root (hd0,0)
kernel /kernel


This file will instruct grub to wait for 10 seconds and the boot entry 0 in the list of operating systems. In this case entry 0 is the only entry. This entry instructs grub to use the first partition of the first disk as the root disk and to use the file kernel, found at the root of the root disk as the kernel and then boot that kernel. Once you have that file on the image you’re done setting it up and is then ready to create the kernel.

A better make

I’m implementing my own ‘make’-like build system in C — I call it builder. I’m wondering why people haven’t done it they way I (want to) do it. Instead of manually telling the builder what kind of project it is (library or executable) I determine that through the name of the project (if the first three characters is lib then it’s a library — otherwise it’s an executable).

I plan to (but haven’t worked it out entirely yet) to allow automation of compiler flags and what libraries to link to. The plan is to make it possible to link to a library simply by including a header file.

To morfik or to make things work

I think it’s about time I bitch about Morifk (www.morfik.com).

Morfik is a RAD (Rapid Application Development) tool, designed to make it easy to produce applications that reside on the net (in a browser to be accurate), allowing the programmer to do all the code in either Object Pascal, Visual Basic or C#. That idea is fine and if they got it right it would be a great thing to have. But, as you may have guesse already, Morfik doesn’t live up to it’s own promises.

The first thing you may notice when you start using Morfik is that you often have to resort to using custom built data extraction routines, unless you are doing stock webpages (in which case it would have been smarter to simply have made a template webpage and sell that) — if you do not use one  of the (few) standard layouts, you are simply not going to have any use for the database aware components of Morfik. This means using webmethods and rebuilding what is essentially the same routine over and over again (code sharing is not a strong point of Morfik).

Another thing you will soon notice is that the date type is handled differently on the server and in the browser — the (to me) most odd thing is that it’s the browser that handles dates the best. In the end I had to code my own date type that doesn’t make use of server or browser specific code (of course I still need a browser and a server implementation, but since they are identical, that’s a minor issue more than a problem).

The company behind Morfik calls Morfik the WebOS, but in reality it feels more like you’re writing software for a set of operating systems that have to communicate over the most complex channel they could think of — raw socket programming in C is less complex (not really, I’m just frustrated).

Another strangeness of morfik is how text is handled. You cannot do things like casting a character to an integer and then print the result (that is the numeric value of the character) — that means that I, for example, am not able to search and replace anything out side the basic English alphabet, since I have no idea what morfik is using nor what external datasources are using. Which brings me to another point. Morfik uses unicode. Or it claims to do. Or that is, Morfik claims to be using unicode when it doesn’t claim not to do so. That is, Morfik doesn’t really do anything with unicode. In fact, Morfik doesn’t seem to be doing much really. If you try to print the danish letters ‘æ’, ‘ø’ or ‘å’ (or the capital equivelant) you get garbage — and since you have no idea what the numeric values are and (strangely) you cannot compare those letters with themselves once they have been outside of morfik, there is no way to replace them with the html codes for those letters (nor should you have to — morfik should just do things right). In order to replace these letters I had to copy the result of printing them a HTML page and use those results to search and replace all instances with the HTML codes for the letters — of course, this doesn’t work with Reports since they use an entirely different character set. In reports I can’t use either trick (casting or copying) to get it fixed.

These issues and problems are not related to the Morfik program though — the problems with the IDE are enough to make you cry. Random crashes, lock-ups and plain weirdness is abundant. For example, I often get an error message that says “Cannot copy to clipboard”, when I hit CTRL+X. The content is always copied but the original is never deleted. Also when viewing datatables (either internal or external) I often get no result at all, which locks up the database components of Morfik forcing me to shut down and restart the IDE to restore it.

I wish I could be allowed to do things in PHP. Sigh.

How to save the Playstation 3 platform

The PS3 has been around for some time now, and still it doesn’t have a lot of games. More annoyingly, most games for the platform are crap — either fully or compared to the Windows and Xbox versions of the games. A few of the games are decent — but they are not all over the place. A lot of developers have been complaining about the console — which may be one of the reasons why the PS3 doesn’t have a lot of games.

I think I have a solution to the problem. First of all Sony has to realise that to gain control over the market, they have to release (some) control over the console. This control will actually not be lost, but shifted, if my proposal is followed. One thing that’s bugging a lot of people is that you have to pay money to develop games for the console. That may seem like a minor problem as you also have to pay money to develop stuff for the XBox and the Wii. The difference (from the XBox) is that the XBox in many ways is just a stripped down Windows machine running on a triple cored Power 970. You can make a game for Windows and be reasonable comfortable that porting won’t be a too big issue. In some cases, it will be very close to just doing a recompile for the XBox. The PS3 is completely different. As far as I know, it doesn’t look like anything else. I have no details on the SDK, but there is no reason that it should look like any other operating system and the fact that it has a completely new type of processor, may seem frightening to some people. Of course the Cell processor should just be seen as yet another “GPU” like processor and the SPEs should just be treated as an easily accessable set of GPU cores — not like a real part of the CPU.

The first part of my suggestion is this: release the SDK to the public. Allow anyone to develop and publish software for the PS3. Then, make all game disitribution go through PlaystationStore. It is extremely convenient to buy games of PlaystationStore and it adds that you don’t have to have disks lying around or change them to play another game. Further it means that Sony is the only middleman from the developers to the consumer, which can result in two things: cheaper games or increased profit (both for Sony and for developers). If the cheaper games option is chosen, it’s not unlikely that it will also increase profits.

The second part, which is a bit more drastic is to revise the Playstation 3 console itself. Add a lot of memory — at least a total of 1GiB — and increase the harddrive capacity to at least 500 GiB. The extra memory reduces the problems developers have had with the PS3 operating system hogging up to 84 MiB of the preciouse 256MiB of memory it has. Furthermore it will make it feaseable to run GNU/Linux on the machine, which is rather painful at the moment.